Traceback error

pthielge · August 25, 2022, 10:14pm

Hey guys,

I am totally new to W&B. I am getting a Traceback error when I want to run “wandb.init(project=”…“)”. Last week it still did work. Any tips what to do?? Thank you so much.

Traceback (most recent call last):
  File "/home/p/pthielge/.local/lib/python3.6/site-packages/wandb/sdk/wandb_init.py", line 999, in init
    run = wi.init()
  File "/home/p/pthielge/.local/lib/python3.6/site-packages/wandb/sdk/wandb_init.py", line 651, in init
    backend.cleanup()
  File "/home/p/pthielge/.local/lib/python3.6/site-packages/wandb/sdk/backend/backend.py", line 246, in cleanup
    self.interface.join()
  File "/home/p/pthielge/.local/lib/python3.6/site-packages/wandb/sdk/interface/interface_shared.py", line 475, in join
    super().join()
  File "/home/p/pthielge/.local/lib/python3.6/site-packages/wandb/sdk/interface/interface.py", line 666, in join
    _ = self._communicate_shutdown()
  File "/home/p/pthielge/.local/lib/python3.6/site-packages/wandb/sdk/interface/interface_shared.py", line 472, in _communicate_shutdown
    _ = self._communicate(record)
  File "/home/p/pthielge/.local/lib/python3.6/site-packages/wandb/sdk/interface/interface_shared.py", line 226, in _communicate
    return self._communicate_async(rec, local=local).get(timeout=timeout)
  File "/home/p/pthielge/.local/lib/python3.6/site-packages/wandb/sdk/interface/interface_shared.py", line 231, in _communicate_async
    raise Exception("The wandb backend process has shutdown")
Exception: The wandb backend process has shutdown
wandb: ERROR Abnormal program exit
---------------------------------------------------------------------------
Exception                                 Traceback (most recent call last)
~/.local/lib/python3.6/site-packages/wandb/sdk/wandb_init.py in init(job_type, dir, config, project, entity, reinit, tags, group, name, notes, magic, config_exclude_keys, config_include_keys, anonymous, mode, allow_val_change, resume, force, tensorboard, sync_tensorboard, monitor_gym, save_code, id, settings)
    998         try:
--> 999             run = wi.init()
   1000             except_exit = wi.settings._except_exit

~/.local/lib/python3.6/site-packages/wandb/sdk/wandb_init.py in init(self)
    650                     # we don't need to do console cleanup at this point
--> 651                     backend.cleanup()
    652                     self.teardown()

~/.local/lib/python3.6/site-packages/wandb/sdk/backend/backend.py in cleanup(self)
    245         if self.interface:
--> 246             self.interface.join()
    247         if self.wandb_process:

~/.local/lib/python3.6/site-packages/wandb/sdk/interface/interface_shared.py in join(self)
    474     def join(self) -> None:
--> 475         super().join()
    476 

~/.local/lib/python3.6/site-packages/wandb/sdk/interface/interface.py in join(self)
    665             return
--> 666         _ = self._communicate_shutdown()
    667 

~/.local/lib/python3.6/site-packages/wandb/sdk/interface/interface_shared.py in _communicate_shutdown(self)
    471         record = self._make_record(request=request)
--> 472         _ = self._communicate(record)
    473 

~/.local/lib/python3.6/site-packages/wandb/sdk/interface/interface_shared.py in _communicate(self, rec, timeout, local)
    225     ) -> Optional[pb.Result]:
--> 226         return self._communicate_async(rec, local=local).get(timeout=timeout)
    227 

~/.local/lib/python3.6/site-packages/wandb/sdk/interface/interface_shared.py in _communicate_async(self, rec, local)
    230         if self._process_check and self._process and not self._process.is_alive():
--> 231             raise Exception("The wandb backend process has shutdown")
    232         future = self._router.send_and_receive(rec, local=local)

Exception: The wandb backend process has shutdown

The above exception was the direct cause of the following exception:

Exception                                 Traceback (most recent call last)
<ipython-input-49-e3734aa09c65> in <module>
      1 #Login to wandb
      2 # #! wandb login config_dict["wandb_key"]
----> 3 wandb.init()
      4 #run_name = wandb.run.name

~/.local/lib/python3.6/site-packages/wandb/sdk/wandb_init.py in init(job_type, dir, config, project, entity, reinit, tags, group, name, notes, magic, config_exclude_keys, config_include_keys, anonymous, mode, allow_val_change, resume, force, tensorboard, sync_tensorboard, monitor_gym, save_code, id, settings)
   1035             if except_exit:
   1036                 os._exit(-1)
-> 1037             raise Exception("problem") from error_seen
   1038     return run

Exception: problem

ramit_goolry · August 29, 2022, 11:11pm

Hi @pthielge!

Thanks for writing in, and I am sorry you are facing issues over here. There could be a few issues that could cause this error to show up in your program. Could you share some more information with me so that we could narrow down the scope of this error?

What version of wandb are you using?
What operating system are you running this script on?
There should be a folder called wandb in your working directory with multiple sub-folders of the format run-<DATETIME>-<ID>, could you share the debug.log and debug-internal.log files from the folder corresponding to this run ID?

Thanks,
Ramit

pthielge · August 30, 2022, 8:53am

Hi Ramit,

thanks for your reply. I’m using version ‘0.13.2’ in JupyterLab in Windows 11. Somehow, there is only a debug.log and no debug-internal.log file, so I will append it here.

Thanks,

Philip

2022-08-30 10:44:49,612 INFO    MainThread:420 [wandb_setup.py:_flush():76] Configure stats pid to 420
2022-08-30 10:44:49,612 INFO    MainThread:420 [wandb_setup.py:_flush():76] Loading settings from /home/p/pthielge/.config/wandb/settings
2022-08-30 10:44:49,612 INFO    MainThread:420 [wandb_setup.py:_flush():76] Loading settings from /home/p/pthielge/3D/S2_Top/wandb/settings
2022-08-30 10:44:49,612 INFO    MainThread:420 [wandb_setup.py:_flush():76] Loading settings from environment variables: {'_require_service': 'True'}
2022-08-30 10:44:49,612 INFO    MainThread:420 [wandb_setup.py:_flush():76] Inferring run settings from compute environment: {'program': '<python with no main file>'}
2022-08-30 10:44:49,612 INFO    MainThread:420 [wandb_init.py:_log_setup():461] Logging user logs to /home/p/pthielge/3D/S2_Top/wandb/run-20220830_104449-5v6f9ano/logs/debug.log
2022-08-30 10:44:49,613 INFO    MainThread:420 [wandb_init.py:_log_setup():462] Logging internal logs to /home/p/pthielge/3D/S2_Top/wandb/run-20220830_104449-5v6f9ano/logs/debug-internal.log
2022-08-30 10:44:49,613 INFO    MainThread:420 [wandb_init.py:init():495] calling init triggers
2022-08-30 10:44:49,613 INFO    MainThread:420 [wandb_init.py:init():499] wandb.init called with sweep_config: {}
config: {}
2022-08-30 10:44:49,613 INFO    MainThread:420 [wandb_init.py:init():548] starting backend
2022-08-30 10:44:49,613 INFO    MainThread:420 [wandb_init.py:init():552] setting up manager
2022-08-30 10:44:49,631 ERROR   MainThread:420 [wandb_init.py:init():1070] error
Traceback (most recent call last):
  File "/home/p/pthielge/.local/lib/python3.6/site-packages/wandb/sdk/wandb_init.py", line 1043, in init
    run = wi.init()
  File "/home/p/pthielge/.local/lib/python3.6/site-packages/wandb/sdk/wandb_init.py", line 553, in init
    manager._inform_init(settings=self.settings, run_id=self.settings.run_id)
  File "/home/p/pthielge/.local/lib/python3.6/site-packages/wandb/sdk/wandb_manager.py", line 161, in _inform_init
    svc_iface._svc_inform_init(settings=settings, run_id=run_id)
  File "/home/p/pthielge/.local/lib/python3.6/site-packages/wandb/sdk/service/service_sock.py", line 39, in _svc_inform_init
    self._sock_client.send(inform_init=inform_init)
  File "/home/p/pthielge/.local/lib/python3.6/site-packages/wandb/sdk/lib/sock_client.py", line 140, in send
    self.send_server_request(server_req)
  File "/home/p/pthielge/.local/lib/python3.6/site-packages/wandb/sdk/lib/sock_client.py", line 84, in send_server_request
    self._send_message(msg)
  File "/home/p/pthielge/.local/lib/python3.6/site-packages/wandb/sdk/lib/sock_client.py", line 81, in _send_message
    self._sendall_with_error_handle(header + data)
  File "/home/p/pthielge/.local/lib/python3.6/site-packages/wandb/sdk/lib/sock_client.py", line 61, in _sendall_with_error_handle
    sent = self._sock.send(data[total_sent:])
BrokenPipeError: [Errno 32] Broken pipe

ramit_goolry · September 14, 2022, 4:28pm

Hey @pthielge,

Thanks for the logs! Is this an error that you see consistently across the runs or is this just a one-off event? Looks like the communication we set up between your program and our server was dropped, so I don’t expect this to happen consistently.

Thanks,
Ramit

ramit_goolry · September 20, 2022, 5:07pm

Hi @pthielge,

We wanted to follow up with you regarding your support request as we have not heard back from you. Please let us know if we can be of further assistance or if your issue has been resolved.

pthielge · September 20, 2022, 5:23pm

Hi,

thanks for your reply @ramit_goolry. It seems you’re right, some temporary connection problem. I had this issue for 2 days, and then suddenly it was no longer there, without changing the code.

Thank you so much. Best regards!

ramit_goolry · September 20, 2022, 5:28pm

Got it! In that case, I’ll go ahead and close out this support request. In case this issue shows up again, please let us know!

Thanks,
Ramit

system · November 19, 2022, 5:28pm

This topic was automatically closed 60 days after the last reply. New replies are no longer allowed.

Topic		Replies	Views
ERROR Abnormal program exit W&B Help wandb	3	1672	January 21, 2023
Permission Error W&B Help	3	1190	July 18, 2022
Wandb fails at init (assert ports_found) W&B Help wandb	8	4014	June 3, 2023
Unable to login W&B Help wandb	16	5097	September 5, 2023
NotFoundError() W&B Help sweeps , wandb	3	580	October 31, 2022

Traceback error

Related Topics