Table metrics are not logged when killing a wandb run early

Hi I have the following problem. I log several statistics, e.g. with

run.log({"train-loss": loss.item()}
run.summary["best_accuracy"] =  best_acc1

and then I kill my wandb run early in the terminal. Unfortunately the metric “train-loss” and “best_accuracy” is not logged in the table in wandb. The plots are available. It seems that summary metrics aren’t. Does anyone know how I can fix this? For early prototyping, I kill jobs quite often.

Thanks and cheers!

Stefan

Hey @skolek,

How are you killing your jobs? If you are using something like a KeyboardInterrupt, I would recommend using something like a try/except block to enclose your code, and have a call to wandb.finish() in your except block.

Calling wandb.finish() starts a cleanup process for the run and makes sure all the data from your run gets synced.

Please let me know if this solves your issue.

Thanks,
Ramit
Weights and Biases Support

Hi @skolek,

We wanted to follow up with you regarding your support request as we have not heard back from you. Please let us know if we can be of further assistance or if your issue has been resolved.

Best,
Weights & Biases