Average performance over seeds for Bayesian hyperparameter optimizer

[kyrylo-boiko]

Dear W&B Support Team,

In ML, a single seed’s performance is not a reliable metric to feed to a Bayesian optimizer. This is especially pronounced in RL, where the instability is so high that a SOTA method can perform worse than the baseline on an unlucky run, and vice versa[1]. This renders the W&B Bayesian optimizer currently unusable (or rather, unscientific to use) for tuning RL hyperparams.

I believe the community has voiced support for this feature numerous times in many threads (below), but no updates or even workarounds were ever provided. I hope this post becomes a +1 and a summary of previous requests, and finally moves the needle on this feature.

Personally, not being able to use the W&B bayes optimizer means I have to settle for the random option, or spend considerable time implementing a seed-aggregate wrapper.

Previous requests:

Sweeps with multiple seeds for the same config values

Hi, Sometimes (for example in RL) agents are very unstable and you only know how a config behaves if you tested it on 5-10 seeds. So I was wondering if there is a feature in wandb sweeps that allows the aggregation of a metric over multiple seeds (but the same config values)? I know one solution is to define a for loop in my own training script that repeats the same config, but I would like these runs to be executed in parallel, and possibly even on different machines. Thanks, Tom

I wonder if the utility to aggregate over multiple seeds was added or not in later releases

Based on this question posted 2 years back : Sweeps with multiple seeds for the same config values, I wonder if the utility to sweep over different parameters meanwhile also being able to aggregate over different seeds was added to wandb or not. If yes, is there a tutorial/helper link which I can follow to use this facility. Thanks

Averaging objective over various seeds in sweeps

Hi, In RL and ML, we often evaluate the performances of algorithms over various seeds. However, I think this is currently not supported in sweeps. What I want is the sweep to run multiple times the same hyper-parameter values over various seeds and then use an aggregation of the objective values as objective function of the sweep. Of course, I’d like to have control over the seeds, e.g. be able to specify them. I believe this has been requested already here: Sweeps with multiple seeds for the …

Average multiple runs over one parameter

I am doing a sweep over multiple parameters. For every configuration, I also have a seed to control randomness. For instance, let’s say I am sweeping over learning rate in [0.1, 0.01] and batch size in [16, 32]. This will give me 4 runs, and for each I will have two runs with seed 1 and 2, treated as hyperparameter, for a total of 8 runs. I’d like to average ALL plots over the 2 seeds. So, instead of having 8 curves per plot I’ll have only 4. I can do this manually by grouping over learning r…

Sweeps with a set number of seeds for the same hyperparameter values

Hi, I would like to run a random sweep of my hyperparameters. However, I want to have 5 runs (with varying seeds) for each hyperparameter configuration. In other words, I want a random search strategy for my hyperparameters but a grid search strategy with respect to the seeds. This is a pretty standard way of performing hyperparameter tuning in my field (computing science). There is an old thread from 2021 that mentions this feature may be added to W&B, but it is now 2023 and I don’t see any …

Three points of improvement for Sweeps

Hi all, I’ve started using Sweeps, and after having wrestled with it for some time, I see three points of improvement. A “repeat” parameter. It is pretty standard practice to run a particular hyperparameter choice for a couple different seeds. To do this now, we have to add a few different seeds to the optimized hyperparameters, and the results are not aggregated across different seeds. More generally, it would be great if there was a way to create multiple runs for one agent’s function call …

Reference:

[1] - Eimer, Theresa, Marius Lindauer, and Roberta Raileanu. ‘Hyperparameters in Reinforcement Learning and How To Tune Them’. arXiv:2306.01324. Preprint, arXiv, 2 June 2023. [2306.01324] Hyperparameters in Reinforcement Learning and How To Tune Them .

[ml-reports]

I’m sending this off to the support team to see what can be done.

[kiante]

We have a solution in the Kempner Institute handbook; feel free to submit an issue to our GitHub repository if you encounter any problems. github/KempnerInstitute/optimizing-ml-workflow/tree/main/workshop_exercises/wandb_aggregate