I’m trying to use HuggingFace’s Trainer API to fine-tune BERT:
# Skip trainer till we can figure out how to specify MSE as loss
trainer = Trainer(
model=model, # the instantiated 🤗 Transformers model to be trained
args=training_args, # training arguments, defined above
train_dataset=train_dataset,
eval_dataset=eval_dataset)
trainer.train()
However, the validation loss is not logged to WandB:
Hi Rylan, what do you have stored in the training_args variable? There are several ways to integrate W&B for HuggingFace. One is to make sure that training_args is a transformers.TrainingArguments object with TrainingArguments(report_to='wandb').
Hope this helps! Full W&B documentation for HuggingFace can be found here: