# Notebook/full code for "Hyperparameter Optimization for HuggingFace" **URL:** \ **Category:** W&B Help\ **Tags:** sweeps\ **Created:** [May 21, 2023, 10:38am UTC](https://community.wandb.ai/t/notebook-full-code-for-hyperparameter-optimization-for-huggingface/4434 "2023-05-21T10:38:02Z")\ **Posts on this page:** 6\ **Page:** 1
**Author:** ![bramvanroy](https://avatars.discourse-cdn.com/v4/letter/b/b9e5f3/32.png) [@bramvanroy](https://community.wandb.ai/u/bramvanroy)\ **Post date:** [May 21, 2023, 10:38am UTC](https://community.wandb.ai/t/notebook-full-code-for-hyperparameter-optimization-for-huggingface/4434/1 "2023-05-21T10:38:02Z")
Hello I want to parallellize my sweeps with WANDB over different GPUs and I came across this post below. It looks promising but unfortunately a full notebook is missing. > **[Hyperparameter Optimization for HuggingFace Transformers](https://wandb.ai/amogkam/transformers/reports/Hyperparameter-Optimization-for-HuggingFace-Transformers--VmlldzoyMTc2ODI)** > > This article explains three strategies for hyperparameter optimization for HuggingFace Transformers, using W&B to track our experiments. Can you please share this code? Thanks ---
**Author:** ![fantauzzi](https://avatars.discourse-cdn.com/v4/letter/f/cdc98d/32.png) [@fantauzzi](https://community.wandb.ai/u/fantauzzi)\ **Post date:** [May 22, 2023, 6:44pm UTC](https://community.wandb.ai/t/notebook-full-code-for-hyperparameter-optimization-for-huggingface/4434/2 "2023-05-22T18:44:48Z")
The content of the blog post seems obsolete as it mentions `WandbLogger` and `@wandb_mixin` , but if you try to use them (even with the examples provided in wandb documentation/GitHub) you get errors saying they are deprecated. See the Example Code at the bottom of [Ray Tune | Weights & Biases Documentation](https://docs.wandb.ai/guides/integrations/ray-tune) Looking at the source code it seems to me that `WandbLogger` and `@wandb_mixin` are being reworked/replaced, and the change is not yet documented. Also see [[CLI]: Ray Tune documentation is out of date and raise ImportError · Issue #4829 · wandb/wandb · GitHub](https://github.com/wandb/wandb/issues/4829) The most puzzling part to me is how the blog author has made the parallel coordinate charts. As far as I can tell, they require to use wandb sweeps, I am not aware of any way to obtain them doing the sweeps with, say, Hyperopt or Ray. ![Screenshot from 2023-05-22 20-37-23](https://us1.discourse-cdn.com/flex020/uploads/wandb/original/2X/0/02e1935037a43511ab07e0a0e0bdb7bb8d0a896e.png) By clicking on the runs tabulated in the blog article it is possible to go to the author’s project, but from there I cannot find any parallel coordinate chart, and the sweeps section is empty. ---
**Author:** ![thanos-wandb](https://avatars.discourse-cdn.com/v4/letter/t/4af34b/32.png) [@thanos-wandb](https://community.wandb.ai/u/thanos-wandb)\ **Post date:** [May 25, 2023, 8:41am UTC](https://community.wandb.ai/t/notebook-full-code-for-hyperparameter-optimization-for-huggingface/4434/3 "2023-05-25T08:41:01Z")
Hi @bramvanroy thanks for writing in! Are you referring to W&B Ray Tune integration in that article? There’s a [Colab here](https://colab.research.google.com/github/wandb/examples/blob/master/colabs/raytune/RayTune_with_wandb.ipynb#scrollTo=iUi06ExGDuAz) if that helps. Could you please provide us with some more details of your compute infra, and your training environment for multi-gpu training? If you’re using PyTorch DDP, we also have an example in our repo [here](https://github.com/wandb/examples/tree/master/examples/pytorch/pytorch-ddp). ---
**Author:** ![fantauzzi](https://avatars.discourse-cdn.com/v4/letter/f/cdc98d/32.png) [@fantauzzi](https://community.wandb.ai/u/fantauzzi)\ **Post date:** [May 25, 2023, 5:04pm UTC](https://community.wandb.ai/t/notebook-full-code-for-hyperparameter-optimization-for-huggingface/4434/4 "2023-05-25T17:04:01Z")
> [@thanos-wandb](#): > > Hi @bramvanroy thanks for writing in! Are you referring to W&B Ray Tune integration in that article? There’s a [Colab here](https://colab.research.google.com/github/wandb/examples/blob/master/colabs/raytune/RayTune_with_wandb.ipynb#scrollTo=iUi06ExGDuAz) The Colab you linked stops with an error. I believe it is because it uses an implementation of the integration with Ray Tune which has been deprecated. I couldn’t find any documentation nor example for the new integration. Also see [[Q] What is the correct way to file issues in the documentation? · Issue #5595 · wandb/wandb · GitHub](https://github.com/wandb/wandb/issues/5595#issuecomment-1557871523) ---
**Author:** ![thanos-wandb](https://avatars.discourse-cdn.com/v4/letter/t/4af34b/32.png) [@thanos-wandb](https://community.wandb.ai/u/thanos-wandb)\ **Post date:** [May 31, 2023, 10:53pm UTC](https://community.wandb.ai/t/notebook-full-code-for-hyperparameter-optimization-for-huggingface/4434/5 "2023-05-31T22:53:00Z")
Hi @fantauzzi thanks for flagging this, indeed some imports and calls have been deprecated. I also saw you’ve raised this issue in our wandb/examples repo and the Colabs update should be addressed there. We have also an internal ticket filed to update our Docs. We will keep you posted here once these are fixed. ---
**Author:** ![system](https://us1.discourse-cdn.com/flex020/uploads/wandb/original/1X/366b649231631dbab896843020da0056074ac79d.png) [@system](https://community.wandb.ai/u/system)\ **Post date:** [July 30, 2023, 10:53pm UTC](https://community.wandb.ai/t/notebook-full-code-for-hyperparameter-optimization-for-huggingface/4434/6 "2023-07-30T22:53:17Z")
This topic was automatically closed 60 days after the last reply. New replies are no longer allowed.