# Strategy for adding referenced files to an artifact **URL:** \ **Category:** W&B Help\ **Tags:** artifacts\ **Created:** [September 11, 2022, 5:29am UTC](https://community.wandb.ai/t/strategy-for-adding-referenced-files-to-an-artifact/3094 "2022-09-11T05:29:45Z")\ **Posts on this page:** 6\ **Page:** 1
**Author:** ![kevinashaw](https://sea2.discourse-cdn.com/flex020/user_avatar/community.wandb.ai/kevinashaw/32/1016_2.png) [@kevinashaw](https://community.wandb.ai/u/kevinashaw)\ **Post date:** [September 11, 2022, 5:29am UTC](https://community.wandb.ai/t/strategy-for-adding-referenced-files-to-an-artifact/3094/1 "2022-09-11T05:29:45Z")
We are new to WandB and working out best-practices for using referenced-artifacts. We have an S3 bucket were we keep our data corpus, so that it can be shared between machines. We want to use WandB to track these files and to use it to download/synchronize copies of the datasets to local machines. There seem to be two ways to add files to an artifact: 1. By-Group: Create the file locally and put it to S3. Repeat with all other data files until done. Then use the `artifact.add_reference()` command and point it to the S3 prefix/directory for the files. This will add the directory and its files to the artifact. The artifact will report that only a single “file” exists (since we only added the directory) – which I think is weird, by the way – but all the files seem to be there. 2. One-by-one: create the file locally, put it to S3 and immediately add the S3 file to the artifact. Repeat until all files are done and then close the artifact and the run. The artifact will now properly note that n-files have been added. The real question is, when I later execute a `download(root=my-local-path)` operation, will I be able to cleanly load the files from the artifact to my local directory. That is, without having to fight a path mismatch between the S3 paths and my local paths. That is, if the S3 path is: /really/deep/s3/path/to/my/dataset/files And my local path is: /Users/user/ Can the files end up here: /Users/user/dataset/files Thank you, Kevin ---
**Author:** ![mohammadbakir](https://avatars.discourse-cdn.com/v4/letter/m/a9a28c/32.png) [@mohammadbakir](https://community.wandb.ai/u/mohammadbakir)\ **Post date:** [September 14, 2022, 6:15pm UTC](https://community.wandb.ai/t/strategy-for-adding-referenced-files-to-an-artifact/3094/2 "2022-09-14T18:15:56Z")
Hi @kevinashaw , when [downloading](https://docs.wandb.ai/ref/python/artifact#download) the Artifact, set the argument `recursive= True`and define your `root` directory to download Artifacts to. The artifacts will download cleanly. ---
**Author:** ![mohammadbakir](https://avatars.discourse-cdn.com/v4/letter/m/a9a28c/32.png) [@mohammadbakir](https://community.wandb.ai/u/mohammadbakir)\ **Post date:** [September 20, 2022, 6:17pm UTC](https://community.wandb.ai/t/strategy-for-adding-referenced-files-to-an-artifact/3094/3 "2022-09-20T18:17:33Z")
Hi @kevinashaw, since we have not heard back from you we are going to close this request. If you would like to re-open the conversation, please let us know! ---
**Author:** ![kevinashaw](https://sea2.discourse-cdn.com/flex020/user_avatar/community.wandb.ai/kevinashaw/32/1016_2.png) [@kevinashaw](https://community.wandb.ai/u/kevinashaw)\ **Post date:** [September 20, 2022, 6:40pm UTC](https://community.wandb.ai/t/strategy-for-adding-referenced-files-to-an-artifact/3094/4 "2022-09-20T18:40:26Z")
@mohammadbakir Thank you for the response. However my core question still stands. What sets the base path or root path for a given file. If I add the files one by one how does it know the root path of the file? For example: If the S3 path for a file is: /really/deep/s3/path/to/my/dataset/files/my\_file And my local path is: /Users/user/ Do the files end up here: /Users/user/dataset/files/my\_file Or do I get: /Users/user/really/deep/s3/path/to/my/dataset/files/my\_file How do I control this? ---
**Author:** ![kevinashaw](https://sea2.discourse-cdn.com/flex020/user_avatar/community.wandb.ai/kevinashaw/32/1016_2.png) [@kevinashaw](https://community.wandb.ai/u/kevinashaw)\ **Post date:** [October 5, 2022, 4:28am UTC](https://community.wandb.ai/t/strategy-for-adding-referenced-files-to-an-artifact/3094/5 "2022-10-05T04:28:17Z")
@mohammadbakir Do you have any thoughts on this? Thank you. ---
**Author:** ![system](https://us1.discourse-cdn.com/flex020/uploads/wandb/original/1X/366b649231631dbab896843020da0056074ac79d.png) [@system](https://community.wandb.ai/u/system)\ **Post date:** [December 4, 2022, 4:28am UTC](https://community.wandb.ai/t/strategy-for-adding-referenced-files-to-an-artifact/3094/6 "2022-12-04T04:28:55Z")
This topic was automatically closed 60 days after the last reply. New replies are no longer allowed.