# Sweeps with multiple seeds for the same config values **URL:** \ **Category:** W&B Help\ **Tags:** wandb\ **Created:** [October 24, 2021, 2:31pm UTC](https://community.wandb.ai/t/sweeps-with-multiple-seeds-for-the-same-config-values/1077 "2021-10-24T14:31:01Z")\ **Posts on this page:** 9\ **Page:** 1
**Author:** ![tomjur](https://avatars.discourse-cdn.com/v4/letter/t/e68b1a/32.png) [@tomjur](https://community.wandb.ai/u/tomjur)\ **Post date:** [October 24, 2021, 2:31pm UTC](https://community.wandb.ai/t/sweeps-with-multiple-seeds-for-the-same-config-values/1077/1 "2021-10-24T14:31:01Z")
Hi, Sometimes (for example in RL) agents are very unstable and you only know how a config behaves if you tested it on 5-10 seeds. So I was wondering if there is a feature in wandb sweeps that allows the aggregation of a metric over multiple seeds (but the same config values)? I know one solution is to define a for loop in my own training script that repeats the same config, but I would like these runs to be executed in parallel, and possibly even on different machines. Thanks, Tom ---
**Author:** ![\_scott](https://sea2.discourse-cdn.com/flex020/user_avatar/community.wandb.ai/_scott/32/95_2.png) [@\_scott](https://community.wandb.ai/u/_scott)\ **Post date:** [October 25, 2021, 12:49pm UTC](https://community.wandb.ai/t/sweeps-with-multiple-seeds-for-the-same-config-values/1077/2 "2021-10-25T12:49:35Z")
Hi @tomjur, this is an interesting question! This is possible if you use the seed as a parameter in your sweep config and then group based on the parameters you care about. There are a bunch of ways to group your runs and aggregate your metrics. You can click the group button above your runs and choose the metrics you care about, or you can group within each plot by editing a plot and clicking the Group tab and choosing the parameter you want to group on. > **[Group Runs](https://docs.wandb.ai/guides/track/advanced/grouping#grouping-dynamically-in-the-ui)** > > Group training and evaluation runs into larger experiments ---
**Author:** ![tomjur](https://avatars.discourse-cdn.com/v4/letter/t/e68b1a/32.png) [@tomjur](https://community.wandb.ai/u/tomjur)\ **Post date:** [October 25, 2021, 1:05pm UTC](https://community.wandb.ai/t/sweeps-with-multiple-seeds-for-the-same-config-values/1077/3 "2021-10-25T13:05:01Z")
Hi @_scott thanks for the replay! Sorry, but I still don’t understand the solution. Maybe I can try to be more explicit in my description of the problem (in the following I assume bayes \ random search since I do not have the budget to do a grid search): Let’s say I defined a distribution over all parameters, and specifically, I defined 3 uniform values for seeds. Now if an agent samples a configuration with seed 1, what forces the hyper-parameter optimization process (in the wandb controller) to select the same configuration again but with seed no. 2? If the optimization is random, it is possible but not likely (especially with a continuous random variable), if the optimization is bayes it is unlikely but might also have the bad side-effect of preferring easy seeds. I think this might be a common pain point in RL (and possibly GAN) sweeps. Thanks again! Tom ---
**Author:** ![\_scott](https://sea2.discourse-cdn.com/flex020/user_avatar/community.wandb.ai/_scott/32/95_2.png) [@\_scott](https://community.wandb.ai/u/_scott)\ **Post date:** [October 25, 2021, 1:49pm UTC](https://community.wandb.ai/t/sweeps-with-multiple-seeds-for-the-same-config-values/1077/4 "2021-10-25T13:49:16Z")
Thanks for sharing more detail about your question. I understand now. In essence, you want a grid search over random seeds while also doing a bayes / random search for your other configs, and make sure the configs are the same for each Sweep. It isn’t currently possible to do this, but this is definitely a feature we will try to support in the future because it’s a common workflow for people like you trying to deterministically run Sweeps. Thank you for adding another +1 to this feature request. Unfortunately, my only suggestion for now is doing a grid search for each of the configurations you want to test, including the random seed in that search. ---
**Author:** ![tomjur](https://avatars.discourse-cdn.com/v4/letter/t/e68b1a/32.png) [@tomjur](https://community.wandb.ai/u/tomjur)\ **Post date:** [October 25, 2021, 2:16pm UTC](https://community.wandb.ai/t/sweeps-with-multiple-seeds-for-the-same-config-values/1077/5 "2021-10-25T14:16:33Z")
Thank you for adding this as a feature request! BTW I am a new user, how can I track the progress on this feature? ---
**Author:** ![\_scott](https://sea2.discourse-cdn.com/flex020/user_avatar/community.wandb.ai/_scott/32/95_2.png) [@\_scott](https://community.wandb.ai/u/_scott)\ **Post date:** [October 25, 2021, 3:36pm UTC](https://community.wandb.ai/t/sweeps-with-multiple-seeds-for-the-same-config-values/1077/6 "2021-10-25T15:36:30Z")
Our CLI is Open Source so you can track the release notes here: [Releases · wandb/client · GitHub](https://github.com/wandb/client/releases) And here’s the code for Sweeps: [GitHub - wandb/sweeps: W&B Hyperparameter Sweep Engine. File sweeps related issues at the W&B client: https://github.com/wandb/client](https://github.com/wandb/sweeps) ---
**Author:** ![\_scott](https://sea2.discourse-cdn.com/flex020/user_avatar/community.wandb.ai/_scott/32/95_2.png) [@\_scott](https://community.wandb.ai/u/_scott)\ **Post date:** [October 25, 2021, 3:37pm UTC](https://community.wandb.ai/t/sweeps-with-multiple-seeds-for-the-same-config-values/1077/7 "2021-10-25T15:37:43Z")
> BTW I am a new user Welcome! Hope you’re enjoying using W&B, here to help if you have anymore questions or issues. 🙂 ---
**Author:** ![tomjur](https://avatars.discourse-cdn.com/v4/letter/t/e68b1a/32.png) [@tomjur](https://community.wandb.ai/u/tomjur)\ **Post date:** [October 26, 2021, 2:13pm UTC](https://community.wandb.ai/t/sweeps-with-multiple-seeds-for-the-same-config-values/1077/8 "2021-10-26T14:13:20Z")
Yes W&B is great (: thanks for all the help so far ---
**Author:** ![system](https://us1.discourse-cdn.com/flex020/uploads/wandb/original/1X/366b649231631dbab896843020da0056074ac79d.png) [@system](https://community.wandb.ai/u/system)\ **Post date:** [April 20, 2022, 6:02pm UTC](https://community.wandb.ai/t/sweeps-with-multiple-seeds-for-the-same-config-values/1077/9 "2022-04-20T18:02:07Z")
Closed on April 20, 2022, 6:02pm UTC This topic was automatically closed 60 days after the last reply. New replies are no longer allowed.