OOM memory issue

W&B Help
1

Hi! I’m getting the following issue when trying to run my hyperparameter sweep.

wandb: Starting wandb agent 🕵️
2024-08-18 00:46:19,379 - wandb.wandb_agent - INFO - Running runs: []
wandb: ERROR Error while calling W&B API: OOM command not allowed when used memory > 'maxmemory'. (<Response [500]>)
wandb: ERROR Error while calling W&B API: OOM command not allowed when used memory > 'maxmemory'. (<Response [500]>)
wandb: ERROR Error while calling W&B API: OOM command not allowed when used memory > 'maxmemory'. (<Response [500]>)
wandb: Network error (HTTPError), entering retry loop.
wandb: ERROR Error while calling W&B API: OOM command not allowed when used memory > 'maxmemory'. (<Response [500]>)
^Cwandb: Ctrl-c pressed. Waiting for runs to end. Press ctrl-c again to terminate them.
wandb: Terminating and syncing runs. Press ctrl-c to kill.

Is this on my end or the server side end?

2

Hi @soniajoseph thank you for reporting this issue and we sincerely apologize for any inconvenience caused by the recent issue with Filestream rate limit errors. Our engineers have thoroughly investigated and resolved the problem. As of now, the system should be functioning normally, and you should no longer experience any related disruptions.

I am closing this ticket as the incident appears to be resolved, also noted in our status page: https://status.wandb.com/. However, if you continue to encounter any issues, please don’t hesitate to reach out. We will be more than happy to investigate further.