Encountering 429 Error: FileStream Rate Limit Exceeded

W&B Help
1

Hello everyone,

I’ve encountered a recurring issue with a 429 error stating “FileStream rate limit exceeded” while using wandb. The same code was working perfectly a few hours ago, and I am only trying to run a single experiment at this moment.

Here’s the error message I keep receiving:

wandb: 429 encountered (Filestream rate limit exceeded, retrying in 2.5 seconds.), retrying request
wandb: 429 encountered (Filestream rate limit exceeded, retrying in 4.8 seconds.), retrying request
wandb: 429 encountered (Filestream rate limit exceeded, retrying in 8.6 seconds.), retrying request
wandb: 429 encountered (Filestream rate limit exceeded, retrying in 16.9 seconds.), retrying request
wandb: 429 encountered (Filestream rate limit exceeded, retrying in 36.2 seconds.), retrying request
wandb: 429 encountered (Filestream rate limit exceeded, retrying in 77.2 seconds.), retrying request
wandb: 429 encountered (Filestream rate limit exceeded, retrying in 154.6 seconds.), retrying request

This issue seems to be related to rate limits on file streaming, but I haven’t changed my usage pattern significantly to explain why it’s happening now.

Has anyone else experienced this recently? Any advice on how to mitigate this or adjust settings in wandb to avoid hitting these limits would be greatly appreciated.

Thank you!

2

Hi,

I am encountering the exact same thing. My code is simply:

run = wandb.init(entity = "xxx", project="xxx", job_type="train", config=config)
config = wandb.config

# Load the DataFrames
train_artifact = run.use_artifact('train_df:latest')
train_dir = train_artifact.download()

which is the exact same code I have been running for months without issue and I am now getting these rate limitting issues and am unable to finish a run because it won’t allow me to sync data to wandb.

I am not running any other runs in the background this is my only wandb run.

I also posted on the discord aswell and will link this post to the discord to give this issue a bit of visibility.

Cheers!

1 Like
3

Seems like an issue hitting multiple accounts right based on other threads im seeing

4

I also encounter this problem, how to fix this?

5

Having the same issue here.

6

Hi, having the same issue here.

7

Having the same issue here!!

8

Thank you for posting on the discord!

9

Hello @pguo and everyone,

Apologies as we encountered some issues this weekend that resulted to Rate Limit errors. Our engineers were aware of this and took care of it. Could you please try again and confirm if it is now working on your end?

Thanks,
Joana Marie

10

Hi there, since we have not heard back from you we are going to close this request. If you would like to re-open the conversation, please let us know!