I’m using wandb on a cluster that does not have external internet, so I have to do a
rsync after all the results are collected in offline mode, but then I realize the artifact is using absolute path, so as a result after
rsync the absolute path is changed and I got the following errors
Syncing: https://wandb.ai/rogerluo/duck-rg/runs/2x4xugws ... wandb: ERROR Error uploading "/home/rogerluo/.local/share/wandb/artifacts/staging/tmp3j_0ic4j": FileNotFoundError, [Errno 2] No such file or directory: '/home/rogerluo/.local/share/wandb/artifacts/staging/tmp3j_0ic4j'
I’m wondering if there is a way to store the artifacts in the local directory with a path that relative to the project root?
Hi @rogerluo Thank you for writing in!
In order to potentially solve your concern, you will need to use
WANDB_DIR env vars to have control over where you store artifact files.
Here are some of the important wandb artifact directories:
- the staging directory, which is
WANDB_DATA_DIR/artifacts/staging/ (defaults to
~/.local/share/wandb/artifacts/staging/ on unix). This is where copies of files that are about to be uploaded live
- the artifact directory, which is the default location that artifacts will be downloaded to; that’s set by
WANDB_ARTIFACT_DIR and defaults to
- the cache directory,
WANDB_CACHE_DIR/artifacts (defaults to
~/.cache/artifacts on unix)
- there is also
WANDB_DIR and you may set this to an absolute path to store all generated files instead of the
wandb directory relative to your training script. be sure this directory exists and you have permissions to write to it
Hi there, I wanted to follow up on this request. Please let us know if we can be of further assistance or if your issue has been resolved.
Hi Roger, since we have not heard back from you, we are going to close this request. If you would like to reopen the conversation, please let us know! Unfortunately, at the moment, we do not receive notifications if a thread reopens on Discourse. So, please feel free to create a new ticket regarding your concern if you’d like to continue the conversation.
This topic was automatically closed 60 days after the last reply. New replies are no longer allowed.