Averaging objective over various seeds in sweeps

W&B Help
1

Hi,

In RL and ML, we often evaluate the performances of algorithms over various seeds. However, I think this is currently not supported in sweeps. What I want is the sweep to run multiple times the same hyper-parameter values over various seeds and then use an aggregation of the objective values as objective function of the sweep. Of course, I’d like to have control over the seeds, e.g. be able to specify them.

I believe this has been requested already here: Sweeps with multiple seeds for the same config values
then here:
I wonder if the utility to aggregate over multiple seeds was added or not in later releases
But closed due to no follow-up from the requester.

Thanks in advance,

Florian

2

Hello @florian-felten !

I can certainly make make a feature request for you in order to have this added to sweeps. It seems that since the last post was closed without response that no feature request was made so I would be happy to make this for you.

3

Hi @raphael-sanandres,

Yeah, I’d like to see that in the backlog and prioritized accordingly if possible. I think this feature makes sense for a lot of use cases.

Cheers,

Florian

4

This topic was automatically closed 60 days after the last reply. New replies are no longer allowed.