Invalid artifact

W&B Help
,
1

When trying to add a reference to an object stored on s3 I got a :

CommError: Invalid artifact path: s3://my_bucket_names/my_lang_folder/my_subfolder/my_file.json

I used :

 self.run = wandb.init(
     project="my_project",
     entity="my_group_name",
      dir=self.temp_dir,
      resume=bool(config.previous_model),
 )

 for dataset in config.datasets:
    self.run.use_artifact(
        f"s3://my_bucket_name/{make_path(dataset['lang'],dataset['type'],[],dataset['name'],'json')}",
        type="train_dataset",
    )

I’m sure the dataset exists in this bucket. However, it’s a private bucket but I have access to it with my credentials.

Thanks in advance for any help.

2

Note for the posterity:

I managed to make it work by firstly creating an Artifact like so :

artifact = wandb.Artifact("dataset_name", type="train_dataset")
artifact.add_reference(f"s3://my_bucket/{make_path(dataset_metas['lang'],dataset_metas['type'],[],dataset_metas['name'],'json')}", name="dataset_name")
self.run.use_artifact(artifact)
3

I’m glad you were able to solve your problem. Have a good rest of the day

1 Like
4

This topic was automatically closed 60 days after the last reply. New replies are no longer allowed.