I’m currently performing a hyperparameter search (I’m not using wandb’s sweeep feature for that) on a GPU cluster. I group the runs into different categories, let’s say that’s simply “Group A”, “Group B”, etc.
Now, since jobs can crash for various reasons, I would love to have a panel that reports the number of successful runs for each group, so I know how each group is doing (“Group A has 200 successful runs, while Group B only has 50, so I need to start some more jobs for Group B”). I know I can get there by using the general filter and group feature in my project’s run-table, but this is rather tedious and it would be more convenient for me to have it as a panel for quick access (e.g., inside a report).
I’ve fiddled around with the “scalar chart” and the “weave” panel, but without success – any ideas?
Good question. By the sounds of it, W&B Reports would solve this for you.
You could create a W&B Report and add your panel, do the grouping and filtering there, and then you could see these each time you run something new. It’ll create sections for each of your groups, and subsections for each state.
Here I’ve grouped by learning_rate and a secondary group by State, you could use your group name in place of learning_rate.
I was hoping for a way to compile these sorts of information inside dedicated panels (like a “scalar chart”), since this would generally allow for more concise reports – but this would be a “nice to have”, rather than a “must have”
