Update artifact of finished run

W&B Help
1

Hi,

I have a bunch of runs where I made an error and did not record all relevant information to reproduce it. I want to edit the artifacts and add the information. My code looks something like this:

artifact_dir = artifact.download(
    os.path.abspath("./data/artifacts/" + artifact.name.replace(":", "_"))
)
file_name = artifact_dir + "/" + artifact.files()[0].name
state_dict = torch.load(file_name, map_location=torch.device('cpu'))
model_state_dict = state_dict["model_state_dict"]
# fix missing constraints
if "constraints" not in model_state_dict:
    model_state_dict["constraints"] = dataset.create_constraints().expression
    print("hi")       
    artifact_new = artifact.new_draft()
    artifact_new.remove(artifact.files()[0].name)
    torch.save(state_dict, file_name)
    artifact_new.add_file(file_name)
    artifact_new.save()
    with wandb.init(id=run.id, resume="allow") as r:
                    r.log_artifact(artifact_new)

but this is not working? The project get’s messed up (“ValueError: Artifact init exists in project ‘constrained_distributions’; it can’t be moved to 'constrained-prob-ml-constrained_prob_ml”). I can’t fix this if I add project=run.project in init. Is there something I am doing wrong? The project is constrained_prob_ml.

2

Thank you for providing the context and code snippet. I can see a few issues that might be causing problems. Let’s go through them and suggest some improvements:

  1. Project name mismatch:
    The error message suggests there’s a mismatch between the project names. It seems the artifact is trying to be saved in a project called “constrained_distributions” while your actual project is “constrained_prob_ml”. This could be causing the “can’t be moved” error.

  2. Resuming a run:
    When you’re trying to resume a run, you should use the same project name as the original run. Make sure you’re specifying the correct project name when initializing the run.

  3. Modifying artifacts:
    Keep in mind that artifacts are designed to be immutable once they’re logged. Instead of modifying an existing artifact, it’s generally better to create a new version of the artifact.

Here’s a revised version of your code that should address these issues:

import wandbimport torchimport osdef update_artifact(run, artifact, dataset):# Download the artifactartifact_dir = os.path.abspath(os.path.join("./data/artifacts", artifact.name.replace(":", "_")))artifact.download(root=artifact_dir)file_name = os.path.join(artifact_dir, artifact.files()[0].name)state_dict = torch.load(file_name, map_location=torch.device('cpu'))model_state_dict = state_dict["model_state_dict"]# Check if constraints need to be addedif "constraints" not in model_state_dict: print("Adding missing constraints...") model_state_dict["constraints"] = dataset.create_constraints().expression # Create a new artifact version new_artifact = wandb.Artifact(artifact.name.split(":")[0], type=artifact.type) # Save updated state dict torch.save(state_dict, file_name) new_artifact.add_file(file_name) # Log the new artifact version with wandb.init(id=run.id, resume="must", project=run.project) as resumed_run: resumed_run.log_artifact(new_artifact) print(f"Updated artifact logged: {new_artifact.name}")else: print("Constraints already present. No update needed.")Usagerun = wandb.init(project="constrained_prob_ml") # Make sure this is your correct project nameartifact = run.use_artifact("your_artifact_name:latest")dataset = YourDatasetClass() # Initialize your datasetupdate_artifact(run, artifact, dataset)```

Key changes and explanations:

  1. We’re creating a new artifact version instead of modifying the existing one.
  2. We’re using wandb.init() with resume="must" to ensure we’re continuing the existing run.
  3. We’re explicitly specifying the project name when resuming the run to avoid any project mismatch issues.
  4. We’re only updating the artifact if the constraints are missing.
    Make sure to replace "your_artifact_name" with the actual name of your artifact, and YourDatasetClass with your actual dataset class.

This approach should avoid the project mismatch error and properly update your artifacts with the missing information. If you’re still encountering issues, please provide more details about your project setup and any error messages you’re seeing.

Let me know if this helps and if there is anything else at all I can assist you with :slightly_smiling_face:

3

Hi Leander,

We wanted to follow up with you regarding your support request as we have not heard back from you. Please let us know if we can be of further assistance or if your issue has been resolved.

Best,
Weights & Biases

4

Hi Leander, since we have not heard back from you we are going to close this request. If you would like to re-open the conversation, please let us know!