Add file to artifact without downloading it

W&B Help
1

I have a large artifact containing a table with 6K images and other columns.
Is it possible to add another file, for instance a csv, without having to download the artifact, get the table, add the file and the table to a new version of the artifact and log it to wandb?

A possibility would be to create a new version of the artifact with only the csv file and then merge the two versions (from the UI?), but I am sure this is possible.

Thanks

2

Hi @tommasodelorenzo thanks for writing in! It sounds like that you’re looking for the incremental argument that would allow you to add files to existing artifacts. Please have a look to this example, but also more specifically for your case the following snippet should work:

import wandb

run = wandb.init(entity=ENTITY, project=PROJECT)
artifact = wandb.Artifact('ARTIFACT-NAME', type='ARTIFACT-TYPE', incremental=True)
artifact.add_file('/path/to/file.format') #add_dir works too
run.log_artifact(artifact)
run.finish()

This will create a new version with the new files, plus the files from previous version. Would this work for you?

3

@thanos-wandb Yes, exactly! I didn’t know about this argument. Super useful.
I get a warning about it being experimental. Is it going to become a stable feature soon?

4

Hi @tommasodelorenzo glad to hear this works for you! This is indeed an experimental feature, hence the warning message, and it’s on our roadmap to be further developed. As of right now it’s not possible to remove or update files in existing artifacts, but we will be adding that soon. I hope this helps! Let me know if you have any further questions about this.

1 Like
5

@thanos-wandb . I posted a somewhat related question here. Hope you can solve that too :slight_smile:

6

Hey @tommasodelorenzo feel free to ask us any questions or report any issues, and someone from our team will be happy to investigate and help you with this! I will now close this ticket, and feel free to reopen it if you had any follow-up questions on this.

7

This topic was automatically closed 60 days after the last reply. New replies are no longer allowed.