How to Retrieve and Compare Code for Two WandB Experiments with Git State?

Hi all,

I have two experiments logged in WandB with nearly identical configurations, but one crashed, and the other completed successfully. Each experiment logs a Git state with a branch name and commit hash, like this:

git checkout -b "branch_name" <commit_hash>

However:

  1. I can’t retrieve one of the commit hashes locally—git checkout gives me a “bad object” error, and it’s not found with git fetch --all.
  2. In the WandB code directory, I only see the main module that launches the experiment. In my case, it’s just:
from trainer import train
train()

I suspect the branch and commit were only created in the environment where the training job ran and were not pushed to the remote repository.

Question:

  • Can I retrieve the full code or Git state directly from WandB or the environment?
  • Does the branch/hash information in WandB mean the code is accessible, and if so, how do I retrieve it?
  • Are there best practices to ensure such states are reproducible for debugging?

Thanks in advance!