I am using Wandb with Ray Tune, via WandbLoggerCallback method. Because it was running on a HPC and multiple projects were streaming logs to wandb, it triggered the rate limit, and now I can not even log in to my dashboard and see earlier results. My latest run has stopped and the dashboard is still not available. What can I do in these situations? Is there a way to control the sync rate with the WandbLoggerCallback method? Can I increase the rate limit or somehow apply wandb to a project running in 10-100 parallel versions at the same time?
Related topics
Topic | Replies | Views | Activity | |
---|---|---|---|---|
"rate limit exceeded" blocking access to wandb dashboard | 4 | 646 | July 9, 2023 | |
Rate limit exceeded -- wandb | 7 | 512 | December 22, 2023 | |
Rate limit | 1 | 13 | August 18, 2024 | |
Rate limit exceeded - no access to wandb dashboard | 1 | 5 | December 12, 2024 | |
Wandb: 429 encountered (Filestream rate limit exceeded, retrying in 4.7 seconds.), retrying request | 2 | 8 | August 30, 2024 |