I am running wandb with pytorch Lit. After running the training mode and log all the metric in the dashboard, I want to close the session by running
wandb.finish(). After that I want to execute the test mode so I ativated the mode by
trainer.test(...) but i get an error says “The wandb backend process has shutdown”. it seems logic to me that it can not log to the project to save the test loss because I already close it.
My question here, is there any possible way to reopen the same session with the same project name and ID ?
FYI, i uses the resume mode with the same name and ID but didn’t not work wit me. Each time, it create a new run but in the same project.
Thanks for joining our community
wandb.init between training and testing fix the issue?
great to hear from you (again )
No, I tried
wandb.init but I always get a new run!
I even used the same name of the project and the resume option
you can check with this screen how I get a new run after the
Thanks for that additional context.
So the typical workflow for logging training and testing metrics would be to have a different run for each. You can use run groups if you’d like to group these runs.
You can do this dynamically in the UI or by passing in
Do you have any particular reason for wanting the run for both to be the same?
Edit: Here’s the docs for run groups: https://docs.wandb.ai/guides/track/advanced/grouping
Thank you for your quick replay.
the reason behind this ‘re-run’ that is when I finish the training of my model I dont need wandb so I do
wandb.finish() but after that I want to test my model (either with a test dataset or a new data). In this way, I can’t do the test . so i need to leave the session open. (please excuse me if I did not describe the problem in a clear way)
Ah I understand. You want to disable
wandb entirely while you’re running your test script.
You can disable wandb which will “noop” the wandb.log calls by initialising with
Hope that helps