Average multiple runs over one parameter

W&B Help
,
1

I am doing a sweep over multiple parameters. For every configuration, I also have a seed to control randomness.
For instance, let’s say I am sweeping over learning rate in [0.1, 0.01] and batch size in [16, 32]. This will give me 4 runs, and for each I will have two runs with seed 1 and 2, treated as hyperparameter, for a total of 8 runs.
I’d like to average ALL plots over the 2 seeds. So, instead of having 8 curves per plot I’ll have only 4.
I can do this manually by grouping over learning rate and batch size. However, in practice, I have way more hyperparameters than 2, and I don’t want to manually add all of them one by one to the group.

Is there a way to, basically, “group over all but one” hyperparameter?

Average performance over seeds for Bayesian hyperparameter optimizer
2

Hello @parisi !

Thank you for bringing this up! We do not support this currently but I can make a feature request for you for our internal team to look at.

3

That would be great, thanks!

4

Yes, I do this as well and more tooling would help.

More broadly, when doing a HPO it would be great to be able to tell it to run a set number of seeds at each hyperparameter value and then use the median (or whatever) as the HPO objective value.

5

This topic was automatically closed 60 days after the last reply. New replies are no longer allowed.