How do I store a sweep_id in a cli environment variable so that it runs in a wandb agent later?

W&B Help
1

how do I run a wandb sweep in a cli/terminal/bash and store the sweep id in a env variable to use it later in the wandb agent cli command?

Is this really correct/recommended way?

export SWEEP_ID=$(wandb sweep sweep.yaml --project <your-project-name> | awk '/ID:/{print $2}')
wandb agent --count $N $SWEEP_ID
2

@corey-strausman do you know the answer? (sorry for the direct ping)

It’s really annoying to copy paste the sweep_id directly when using the cli terminal.

3

Hi @brando

To store a sweep_id in a CLI environment variable and run it with wandb agent later, follow these steps:

  1. Initialize the sweep using the wandb sweep command and provide the name of the YAML file. Optionally, provide the name of the project for the project flag (–project):
bash
wandb sweep --project your_project_name config.yaml

This command will return a sweep ID.

  1. Store the sweep ID in an environment variable:
bash
export SWEEP_ID=your_sweep_id

Replace your_sweep_id with the actual sweep ID you got from the previous step.

3.Start the sweep agent using the stored environment variable:

bash
wandb agent --count $NUM your-entity/your_project_name/$SWEEP_ID

Replace your-entity and your_project_name with the appropriate values, and $NUM with the maximum number of runs the sweep agent should try.

Here are some docs to reference:

I hope this helps!

4

I see what you’re trying to do. You want to run a sweep after creating it within a bash script. There’s no way currently to have just the ID as the output and use it for this purpose so your awk solution should suffice for now, but I agree it’s not very elegant and I’ll forward this feedback to the relevant team.

You can use the Python sweeps API to do this better programmatically, as the Sweep ID is returned as a variable in python when you call wandb.sweep. But this may not suit you for your workflow.

5

Thanks @corey-strausman and @_scott for the answers!

@brando Was your question answered or is there something else that you are having issues with?

6

sorry I’m puzzled, can it not be done?

7

Sorry for the confusion. It can be done programmatically in Python, but not in bash. Your way of getting the ID from the logs is an admittedly inelegant workaround in bash. I’ve passed this feedback to the team, and requested some way to get the ID of the sweep so you can assign it to a variable in bash.

1 Like
8

Hello Brando! Did that answer your question or do you have any other concerns?

9

This topic was automatically closed 60 days after the last reply. New replies are no longer allowed.