Hello
I want to parallellize my sweeps with WANDB over different GPUs and I came across this post below. It looks promising but unfortunately a full notebook is missing.
Can you please share this code? Thanks
The content of the blog post seems obsolete as it mentions WandbLogger and @wandb_mixin , but if you try to use them (even with the examples provided in wandb documentation/GitHub) you get errors saying they are deprecated. See the Example Code at the bottom of Ray Tune | Weights & Biases Documentation
Looking at the source code it seems to me that WandbLogger and @wandb_mixin are being reworked/replaced, and the change is not yet documented. Also see [CLI]: Ray Tune documentation is out of date and raise ImportError · Issue #4829 · wandb/wandb · GitHub
The most puzzling part to me is how the blog author has made the parallel coordinate charts. As far as I can tell, they require to use wandb sweeps, I am not aware of any way to obtain them doing the sweeps with, say, Hyperopt or Ray.
By clicking on the runs tabulated in the blog article it is possible to go to the author’s project, but from there I cannot find any parallel coordinate chart, and the sweeps section is empty.
Hi @bramvanroy thanks for writing in! Are you referring to W&B Ray Tune integration in that article? There’s a Colab here if that helps. Could you please provide us with some more details of your compute infra, and your training environment for multi-gpu training? If you’re using PyTorch DDP, we also have an example in our repo here.
The Colab you linked stops with an error. I believe it is because it uses an implementation of the integration with Ray Tune which has been deprecated. I couldn’t find any documentation nor example for the new integration.
Also see [Q] What is the correct way to file issues in the documentation? · Issue #5595 · wandb/wandb · GitHub
Hi @fantauzzi thanks for flagging this, indeed some imports and calls have been deprecated. I also saw you’ve raised this issue in our wandb/examples repo and the Colabs update should be addressed there. We have also an internal ticket filed to update our Docs. We will keep you posted here once these are fixed.
1 Like
This topic was automatically closed 60 days after the last reply. New replies are no longer allowed.