How to plot performance metrics against non-summary stats?
I am running sweeps over a few different datasets within the same job (each for a different location, it’s a multi-task learning problem) and I am making wandb logging calls such as:
wandb.log({
"false_negative_rate" : 0.043,
"false_positive_rate" : 0.261,
"location": "Boston",
"number_training_instances": 50
})
...
wandb.log({
"false_negative_rate" : 0.017,
"false_positive_rate" : 0.145,
"location": "Boston",
"number_training_instances": 100
})
...
wandb.log({
"false_negative_rate" : 0.076,
"false_positive_rate" : 0.334,
"location": "Miami",
"number_training_instances": 50
})
...
wandb.log({
"false_negative_rate" : 0.048,
"false_positive_rate" : 0.172,
"location": "Miami",
"number_training_instances": 100
})
After logging, I’d like to be able to use the dashboard to create various scatter plots such as
Plot false_negative_rate versus false_positive_rate
where location == "Boston"
and number_training_instances == 50
Plot false_negative_rate versus number_training_instances
where location == "Washington"
Plot false_negative_positve_rate versus location
where number_training_instances == 100
However, the filters in the dashboard only provide me the ability to filter/plot the last values logged (as opposed to any of the values logged). E.g., I can only filter on
- number_training_instances == 100
- location == "Miami"
because these were the last values logged for these attributes in each sweep.
Is there a way to get the plotting flexibility I want using wandb’s existing features?
Cheers