BrokenPipeError on GCP

W&B Help
1

I am running my model on a GCP workbench, sometimes we get a BrokenPipeError, I got it this time on epoch 11, it is really annoying and makes us reconsider using wandb. Does anyone have any fix?

venv/lib/python3.10/site-packages/wandb/sdk/lib/sock_client.py", line 130, in _sendall_with_error_handle
sent = self._sock.send(data)
BrokenPipeError: [Errno 32] Broken pipe

2

Hi @borach ,

Thank you for reaching out for support. I’ll be happy to assist you with this. I’ll investigate this and get back to you. For the mean time, can you provide the following for me please:

  • WandB Version you are using
  • Full stack trace error that you are seeing on your end
  • debug.log and debug-internal.log. These files are under your local folder wandb/run-<date>_<time>-<run-id>/logs in the same directory where you’re running your code.
  • Could you also send the code that you are using on running on GCP

Regards,
Carlo Argel

3

Hi @borach , since we have not heard back from you we are going to close this request. If you would like to re-open the conversation, please let us know!

4

This topic was automatically closed 60 days after the last reply. New replies are no longer allowed.