Hi Weights and Biases Community,

I have a question that I was hoping someone more experienced could point me towards documentation for or mention a solution for.

I have a table with model predictions for a classification problem, the columns for my table are as follows:

[epoch, sample_idx, action_idx, logit, predicted_probability, target]

For various samples in each epoch I am recording the logit/predicted probability as well as the target value (0/1).

I am looking to plot an overlayed histogram that looks at the distribution of the predicted probabilities versus the target for action indices for a given sample and epoch. What is the best method of accomplishing this?

I have attempted

- creating the chart as a custom chart
- Using Weave to create the chart.

For the custom chart I got as far as starting to look into using a custom vega spec inspired by this. I am not overly familiar with the syntax so I didn’t spend long on this as I suspected there may be a quicker method via the UI.

When using a weave panel, I have been able to apply groupby operations on a table such that I get individual histograms for the columns predicted probability and target, however, the histograms are not what I am looking for as they are bins with value counts.

Ideally I want to create an overlayed histogram where the x-axis is the action index grouped by epoch and sample and the y-axis is the predicted_probability versus target value in overlayed histogram.

Any pointers would be greatly appreciated, if it is necessary to learn the spec for vega to accomplish this I would be willing to do so but hoping there is a quicker way to accomplish this.