I’m getting an error when I try to call wandb.finish()
This is also the case when I switch to an older commit.
The bug is also triggered when I simply run the following:
import wandb
wandb.init(project="test", entity="tim-kuipers", job_type="train")
wandb.log({"acc": 0.9, "loss": 1.0})
wandb.finish()
My WandB seems to be in a broken state and it cannot escape.
I tried killing the wandb service, so that it would restart, but to no avail.
I tried deleting the last couple of runs.
How do I clean my working directory and get wandb working again?
Here’s the error:
TypeError Traceback (most recent call last)
Cell In[5], line 1
----> 1 wandb.finish()
File ~/mambaforge-pypy3/envs/fastai_env/lib/python3.10/site-packages/wandb/sdk/wandb_run.py:3711, in finish(exit_code, quiet)
3701 """Mark a run as finished, and finish uploading all data.
3702
3703 This is used when creating multiple runs in the same process.
(...)
3708 quiet: Set to true to minimize log output
3709 """
3710 if wandb.run:
-> 3711 wandb.run.finish(exit_code=exit_code, quiet=quiet)
File ~/mambaforge-pypy3/envs/fastai_env/lib/python3.10/site-packages/wandb/sdk/wandb_run.py:390, in _run_decorator._noop..wrapper(self, *args, **kwargs)
387 wandb.termwarn(message, repeat=False)
388 return cls.Dummy()
--> 390 return func(self, *args, **kwargs)
File ~/mambaforge-pypy3/envs/fastai_env/lib/python3.10/site-packages/wandb/sdk/wandb_run.py:331, in _run_decorator._attach..wrapper(self, *args, **kwargs)
329 raise e
330 cls._is_attaching = ""
--> 331 return func(self, *args, **kwargs)
File ~/mambaforge-pypy3/envs/fastai_env/lib/python3.10/site-packages/wandb/sdk/wandb_run.py:1879, in Run.finish(self, exit_code, quiet)
1865 @_run_decorator._noop
1866 @_run_decorator._attach
1867 def finish(
1868 self, exit_code: Optional[int] = None, quiet: Optional[bool] = None
1869 ) -> None:
1870 """Mark a run as finished, and finish uploading all data.
1871
1872 This is used when creating multiple runs in the same process. We automatically
(...)
1877 quiet: Set to true to minimize log output
1878 """
-> 1879 return self._finish(exit_code, quiet)
File ~/mambaforge-pypy3/envs/fastai_env/lib/python3.10/site-packages/wandb/sdk/wandb_run.py:1892, in Run._finish(self, exit_code, quiet)
1890 for hook in self._teardown_hooks:
1891 if hook.stage == TeardownStage.EARLY:
-> 1892 hook.call()
1894 self._atexit_cleanup(exit_code=exit_code)
1895 if self._wl and len(self._wl._global_run_stack) > 0:
File ~/mambaforge-pypy3/envs/fastai_env/lib/python3.10/site-packages/wandb/sdk/wandb_init.py:430, in _WandbInit._jupyter_teardown(self)
428 assert self.notebook
429 ipython = self.notebook.shell
--> 430 self.notebook.save_history()
431 if self.notebook.save_ipynb():
432 assert self.run is not None
File ~/mambaforge-pypy3/envs/fastai_env/lib/python3.10/site-packages/wandb/jupyter.py:435, in Notebook.save_history(self)
433 """This saves all cell executions in the current session as a new notebook."""
434 try:
--> 435 from nbformat import v4, validator, write
436 except ImportError:
437 logger.error("Run pip install nbformat to save notebook history")
File ~/mambaforge-pypy3/envs/fastai_env/lib/python3.10/site-packages/nbformat/__init__.py:12
9 from traitlets.log import get_logger
11 from . import v1, v2, v3, v4
---> 12 from ._version import __version__, version_info
13 from .sentinel import Sentinel
15 __all__ = [
16 "versions",
17 "validate",
(...)
32 "Sentinel",
33 ]
File ~/mambaforge-pypy3/envs/fastai_env/lib/python3.10/site-packages/nbformat/_version.py:28
12 # matches tbump regex in pyproject.toml
13 _version_regex = re.compile(
14 r"""
15 (?P\d+)
(...)
25 re.VERBOSE,
26 )
---> 28 _version_fields = _version_regex.match(__version__).groupdict() # type:ignore
29 version_info = tuple(
30 field
31 for field in (
(...)
38 if field is not None
39 )
TypeError: expected string or bytes-like object
Not sure if this helps:
wandb verify
Default host selected: https://api.wandb.ai
Find detailed logs for this test at: /tmp/tmpoolua1ny/wandb
❌
Cannot run wandb verify against api.wandb.ai
There are no logs in that location. The dir is empty.