HuggingFace Trainer Doesn't Log Validation Loss to WandB

I’m trying to use HuggingFace’s Trainer API to fine-tune BERT:

    # Skip trainer till we can figure out how to specify MSE as loss
    trainer = Trainer(
        model=model,  # the instantiated 🤗 Transformers model to be trained
        args=training_args,  # training arguments, defined above


However, the validation loss is not logged to WandB:

This problem was observed by someone else on the HuggingFace forum: No loss being logged, when running MLM script (Colab) - 🤗Transformers - Hugging Face Forums

Can someone please tell me how to ensure the validation loss is logged to WandB?

1 Like

Hi Rylan, what do you have stored in the training_args variable? There are several ways to integrate W&B for HuggingFace. One is to make sure that training_args is a transformers.TrainingArguments object with TrainingArguments(report_to='wandb').

Hope this helps! Full W&B documentation for HuggingFace can be found here:

1 Like

Yep! report_to='wandb' is successfully logging the training loss to W&B, but the validation loss isn’t being logged…

It seems that adding the following arguments to the TrainingArguments solved the problem :slight_smile:


:+1: Nice to hear that you solved it!

1 Like