Table metrics are not logged when killing a wandb run early

W&B Help
1

Hi I have the following problem. I log several statistics, e.g. with

run.log({"train-loss": loss.item()}
run.summary["best_accuracy"] =  best_acc1

and then I kill my wandb run early in the terminal. Unfortunately the metric “train-loss” and “best_accuracy” is not logged in the table in wandb. The plots are available. It seems that summary metrics aren’t. Does anyone know how I can fix this? For early prototyping, I kill jobs quite often.

Thanks and cheers!

Stefan

2

Hey @skolek,

How are you killing your jobs? If you are using something like a KeyboardInterrupt, I would recommend using something like a try/except block to enclose your code, and have a call to wandb.finish() in your except block.

Calling wandb.finish() starts a cleanup process for the run and makes sure all the data from your run gets synced.

Please let me know if this solves your issue.

Thanks,
Ramit
Weights and Biases Support

3

Hi @skolek,

We wanted to follow up with you regarding your support request as we have not heard back from you. Please let us know if we can be of further assistance or if your issue has been resolved.

Best,
Weights & Biases

4

This topic was automatically closed 60 days after the last reply. New replies are no longer allowed.