Hi Weights and Biases Community,
I have a question that I was hoping someone more experienced could point me towards documentation for or mention a solution for.
I have a table with model predictions for a classification problem, the columns for my table are as follows:
[epoch, sample_idx, action_idx, logit, predicted_probability, target]
For various samples in each epoch I am recording the logit/predicted probability as well as the target value (0/1).
I am looking to plot an overlayed histogram that looks at the distribution of the predicted probabilities versus the target for action indices for a given sample and epoch. What is the best method of accomplishing this?
I have attempted
- creating the chart as a custom chart
- Using Weave to create the chart.
For the custom chart I got as far as starting to look into using a custom vega spec inspired by this. I am not overly familiar with the syntax so I didn’t spend long on this as I suspected there may be a quicker method via the UI.
When using a weave panel, I have been able to apply groupby operations on a table such that I get individual histograms for the columns predicted probability and target, however, the histograms are not what I am looking for as they are bins with value counts.
Ideally I want to create an overlayed histogram where the x-axis is the action index grouped by epoch and sample and the y-axis is the predicted_probability versus target value in overlayed histogram.
Any pointers would be greatly appreciated, if it is necessary to learn the spec for vega to accomplish this I would be willing to do so but hoping there is a quicker way to accomplish this.