Command I run and errors I get below
mlflow run . -P steps=download
wandb: While tearing down the service manager. The following error has occurred: [Errno 32] Broken pipe
wandb: ERROR Abnormal program exit
wandb: ERROR Internal wandb error: file data was not synced
I get several errors like the following as well but won’t post all right now.
File "/home/adduser/miniconda3/envs/mlflow-c1beca2aef7a523d139ba0209b68828b5e298901/lib/python3.10/site-packages/wandb/sdk/internal/internal_util.py", line 49, in run
self._run()
Any insight into why mlflow isn’t communicating with wandb even after being logged in would be helpful.
Hi @samueltwjohnson Good day and thank you for reaching out to us. Happy to help you on this!
May I know what is the host URL you are using for your instance? Can you share what you have set for WANDB_BASE_URL
?
Thanks,
Paulo
Hi Paulo,
I think I got what you are looking for. Please see below.
https://api.wandb.ai
If this is not what you are looking for, I apologize. Mlflow and Wandb are new to me.
Thank you,
Samuel
Hi @samueltwjohnson Good day and thanks for your update! Would you mind changing that URL into https://app.wandb.ai and let me know if this fixes the issue?
Hi Paulo,
Unfortunately, I get the below error.
wandb: ERROR https://app.wandb.ai is not a valid server address, did you mean https://api.wandb.ai?
Thank you,
Samuel
Hi @samueltwjohnson Thank you for checking this for me, you may now revert the changes back using the original setting. We will need to further investigate this and I may need to request some more information that might help us. Could you please provide us the following?
- Full stack trace of the initial errors you are getting
- debug-internal.log and debug.log for the affected run. These files are under your local folder wandb/run-_-/logs in the same directory where you’re running your code. These files will help us with more details about this error.
- A code snippet on how you are trying to execute your MLflow command
Thank you!
Hi @samueltwjohnson , since we have not heard back from you we are going to close this request. If you would like to re-open the conversation, please let us know!