# Sweep on accelerate script

**URL:** <https://community.wandb.ai/t/sweep-on-accelerate-script/6768>\
**Category:** W&B Help\
**Tags:** sweeps\
**Created:** [May 23, 2024, 5:33pm UTC](https://community.wandb.ai/t/sweep-on-accelerate-script/6768 "2024-05-23T17:33:23Z")\
**Posts on this page:** 6\
**Page:** 1

<div class="post-metadata">

**Author:** ![kerkathy](https://avatars.discourse-cdn.com/v4/letter/k/ee7513/32.png) [@kerkathy](https://community.wandb.ai/u/kerkathy)\
**Post date:** [May 23, 2024, 5:33pm UTC](https://community.wandb.ai/t/sweep-on-accelerate-script/6768/1 "2024-05-23T17:33:23Z")

</div>

I’m using Huggingface Accelerate library and also using Wandb to trace everything.  
I use a script regulated by Accelerate to start my python script, where I create a wandb\_tracker using accelerator inside. How can I sweep over a hyperparamter that I set in config.yaml, and keep logging everything onto wandb?

```auto
accelerate launch \
    --gpu_ids 0 \
    my_script_with_accelerator_and_wandb.py \
    --config_file config.yaml

```

Here is how I interact with Wandb

```auto
    accelerator = Accelerator(
        # device_placement='auto',  
        gradient_accumulation_steps=args.gradient_accumulation_steps,
        log_with= 'wandb', 
        mixed_precision='bf16', # turn on bf16
        kwargs_handlers=[kwargs]
    )
        wandb_tracker = accelerator.get_tracker("wandb")

```

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/flex020/uploads/wandb/original/1X/366b649231631dbab896843020da0056074ac79d.png) [@system](https://community.wandb.ai/u/system)\
**Post date:** [May 23, 2024, 5:44pm UTC](https://community.wandb.ai/t/sweep-on-accelerate-script/6768/2 "2024-05-23T17:44:12Z")

</div>

Hello, This a reply from our support bot designed to assist you with your Weights & Biases related queries. To reach a human please reply to this message.

To integrate Hugging Face Accelerate with Weights & Biases for hyperparameter sweeping and configure a W&B sweep using a `config.yaml` file, follow these steps:

1. **Define Your Sweep Configuration** : First, you need to define your sweep configuration in a YAML file. This file will specify the hyperparameters you want to sweep over, the method of sweeping (e.g., grid, random), and any other sweep settings.

2. **Initialize the Sweep** : Use the W&B CLI to initialize the sweep. This step creates the sweep on the W&B server and gives you a sweep ID.

3. **Modify Your Script for W&B and Accelerate** : Ensure your script is set up to use both W&B and Accelerate. When using Accelerate, you typically initialize it as shown in your question. To integrate W&B sweeps, you’ll need to fetch the sweep configuration from W&B inside your script and use those values to set up your model, data, etc.

4. **Launch the Sweep** : Use the `accelerate launch` command along with the W&B agent command to start your sweep. You’ll need to replace `sweep_id` with the ID you received when you initialized the sweep.

**Troubleshooting Tips** :

- Ensure your `config.yaml` is correctly formatted and includes all necessary hyperparameters.
- Check that your script correctly uses the hyperparameters from `wandb.config`.
- If you encounter issues with re-initializing the Accelerator, ensure you’re not inadvertently creating multiple instances within a single run or across sweeps.

For more detailed guidance on setting up sweeps and integrating with Hugging Face Accelerate, refer to the [Weights & Biases documentation on sweeps](https://docs.wandb.ai/guides/sweeps) and the [Hugging Face Accelerate documentation](https://docs.wandb.ai/guides/integrations/accelerate).

To reach a human please reply to this message.

-WandBot 🤖

---

<div class="post-metadata">

**Author:** ![kerkathy](https://avatars.discourse-cdn.com/v4/letter/k/ee7513/32.png) [@kerkathy](https://community.wandb.ai/u/kerkathy)\
**Post date:** [May 25, 2024, 3:49am UTC](https://community.wandb.ai/t/sweep-on-accelerate-script/6768/3 "2024-05-25T03:49:52Z")

</div>

Hi WandBot,

Thanks for trying to reply but that doesn’t solve my problem. My main concern is that wandb sweep agents should be started via CLI, but now my CLI needs to be used to start an Accelerate agent.

---

<div class="post-metadata">

**Author:** ![artsiom](https://sea2.discourse-cdn.com/flex020/user_avatar/community.wandb.ai/artsiom/32/1001_2.png) [@artsiom](https://community.wandb.ai/u/artsiom)\
**Post date:** [June 3, 2024, 10:42pm UTC](https://community.wandb.ai/t/sweep-on-accelerate-script/6768/4 "2024-06-03T22:42:47Z")

</div>

Hi @kerkathy! Thank you so much for elaborating! We do have an option of creating a sweep and calling it using the Python SDK. We have our docs on it [here](https://docs.wandb.ai/guides/sweeps/walkthrough). Would this fit your workflow?

---

<div class="post-metadata">

**Author:** ![artsiom](https://sea2.discourse-cdn.com/flex020/user_avatar/community.wandb.ai/artsiom/32/1001_2.png) [@artsiom](https://community.wandb.ai/u/artsiom)\
**Post date:** [June 6, 2024, 3:06pm UTC](https://community.wandb.ai/t/sweep-on-accelerate-script/6768/5 "2024-06-06T15:06:27Z")

</div>

Hi there, I wanted to follow up on this request. Please let us know if we can be of further assistance or if your issue has been resolved.

---

<div class="post-metadata">

**Author:** ![artsiom](https://sea2.discourse-cdn.com/flex020/user_avatar/community.wandb.ai/artsiom/32/1001_2.png) [@artsiom](https://community.wandb.ai/u/artsiom)\
**Post date:** [June 10, 2024, 5:11pm UTC](https://community.wandb.ai/t/sweep-on-accelerate-script/6768/6 "2024-06-10T17:11:17Z")

</div>

Hi, since we have not heard back from you, we are going to close this request. If you would like to reopen the conversation, please let us know! Unfortunately, at the moment, we do not receive notifications if a thread reopens on Discourse. So, please feel free to create a new ticket regarding your concern if you’d like to continue the conversation.