Track train script version along with hyperaparams (ideally automated)

I’m just getting started with experiments, my goal is to ablation-study some potentially new methods for finetuning NLP models. I use HuggingFace trainer and the W&B integration works flawlessly, so the hyperparams are super easy to track. I’m going to be doing some experimentation within the training code though - is there an easy way to track the training script version/content along with the experiments, ideally integrated with Github? Thanks! Darek

Hi Darek,
They sound like interesting experiments, we’re happy you’ve chosen W&B to track them!

You can track your code by turning on code saving in your settings:

You can also call".") after calling wandb.init() to capture all python source code files in the current directory and all subdirectories as an artifact.

Soon you should be able to use wandb Sweeps as a backend of the hyperparameter_search call in HuggingFace if this PR gets merged: Add W&B backend for hyperparameter sweep by AyushExel · Pull Request #14582 · huggingface/transformers · GitHub

1 Like

Thanks Scott, this is exactly what I needed! Also looking forward to the HF sweep PR :slight_smile: