I’m just getting started with experiments, my goal is to ablation-study some potentially new methods for finetuning NLP models. I use HuggingFace trainer and the W&B integration works flawlessly, so the hyperparams are super easy to track. I’m going to be doing some experimentation within the training code though - is there an easy way to track the training script version/content along with the experiments, ideally integrated with Github? Thanks! Darek
They sound like interesting experiments, we’re happy you’ve chosen W&B to track them!
You can track your code by turning on code saving in your settings: https://docs.wandb.ai/ref/app/features/panels/code
You can also call
wandb.run.log_code(".") after calling
wandb.init() to capture all python source code files in the current directory and all subdirectories as an artifact.
Soon you should be able to use
wandb Sweeps as a backend of the
hyperparameter_search call in HuggingFace if this PR gets merged: Add W&B backend for hyperparameter sweep by AyushExel · Pull Request #14582 · huggingface/transformers · GitHub
Thanks Scott, this is exactly what I needed! Also looking forward to the HF sweep PR