How to make artifact use relative path to the project root?

W&B Help
1

I’m using wandb on a cluster that does not have external internet, so I have to do a rsync after all the results are collected in offline mode, but then I realize the artifact is using absolute path, so as a result after rsync the absolute path is changed and I got the following errors

Syncing: https://wandb.ai/rogerluo/duck-rg/runs/2x4xugws ... wandb: ERROR Error uploading "/home/rogerluo/.local/share/wandb/artifacts/staging/tmp3j_0ic4j": FileNotFoundError, [Errno 2] No such file or directory: '/home/rogerluo/.local/share/wandb/artifacts/staging/tmp3j_0ic4j'

I’m wondering if there is a way to store the artifacts in the local directory with a path that relative to the project root?

2

Hi @rogerluo Thank you for writing in!

In order to potentially solve your concern, you will need to use WANDB_ARTIFACT_DIR or WANDB_DIR env vars to have control over where you store artifact files.

Here are some of the important wandb artifact directories:

  • the staging directory, which is WANDB_DATA_DIR/artifacts/staging/ (defaults to ~/.local/share/wandb/artifacts/staging/ on unix). This is where copies of files that are about to be uploaded live
  • the artifact directory, which is the default location that artifacts will be downloaded to; that’s set by WANDB_ARTIFACT_DIR and defaults to ./artifacts/
  • the cache directory, WANDB_CACHE_DIR/artifacts (defaults to ~/.cache/artifacts on unix)
  • there is also WANDB_DIR and you may set this to an absolute path to store all generated files instead of the wandb directory relative to your training script. be sure this directory exists and you have permissions to write to it
1 Like
3

Hi there, I wanted to follow up on this request. Please let us know if we can be of further assistance or if your issue has been resolved.

4

Hi Roger, since we have not heard back from you, we are going to close this request. If you would like to reopen the conversation, please let us know! Unfortunately, at the moment, we do not receive notifications if a thread reopens on Discourse. So, please feel free to create a new ticket regarding your concern if you’d like to continue the conversation.

5

This topic was automatically closed 60 days after the last reply. New replies are no longer allowed.