# Initial timeout after 90 sec

**URL:** <https://community.wandb.ai/t/initial-timeout-after-90-sec/8041>\
**Category:** W&B Help\
**Created:** [October 1, 2024, 7:16am UTC](https://community.wandb.ai/t/initial-timeout-after-90-sec/8041 "2024-10-01T07:16:27Z")\
**Posts on this page:** 12\
**Page:** 1

<div class="post-metadata">

**Author:** ![ahamedrobin45](https://avatars.discourse-cdn.com/v4/letter/a/8baadc/32.png) [@ahamedrobin45](https://community.wandb.ai/u/ahamedrobin45)\
**Post date:** [October 1, 2024, 7:16am UTC](https://community.wandb.ai/t/initial-timeout-after-90-sec/8041/1 "2024-10-01T07:16:28Z")

</div>

I cannot start a project. whenever i call wandb.init, it runs for 90 seconds and then it automatically stops with an error “ commerror: run initialization has timed out after 90.0 sec.”

---

<div class="post-metadata">

**Author:** ![jason-arkens17](https://avatars.discourse-cdn.com/v4/letter/j/f07891/32.png) [@jason-arkens17](https://community.wandb.ai/u/jason-arkens17)\
**Post date:** [October 1, 2024, 3:52pm UTC](https://community.wandb.ai/t/initial-timeout-after-90-sec/8041/2 "2024-10-01T15:52:06Z")

</div>

Hi there,

Thanks for reaching out about the init issues you’re experiencing. I’d love to help you get to the bottom of this. To better understand what’s going on, could you share a few more details?

1. What system are you running this on? (e.g., Notebook, local, could etc)
2. Which version of our software are you using?
3. Do you have a code snippet that demonstrates the problem?

This information will really help me pinpoint the cause and find a solution for you. Feel free to share whatever you’re comfortable with.

Looking forward to hearing back from you!

Best,  
Jason

---

<div class="post-metadata">

**Author:** ![ahamedrobin45](https://avatars.discourse-cdn.com/v4/letter/a/8baadc/32.png) [@ahamedrobin45](https://community.wandb.ai/u/ahamedrobin45)\
**Post date:** [October 1, 2024, 9:05pm UTC](https://community.wandb.ai/t/initial-timeout-after-90-sec/8041/3 "2024-10-01T21:05:00Z")

</div>

1. ipynb, local machine, linux
2. 0.17.8, also tried the latest one. same problem
3. 

wandb.init(project=EXP\_NAME, entity=‘ahamedrobin45’)  
wandb.config.update(config. **dict** )

error: CommError: Run initialization has timed out after 90.0 sec.

---

<div class="post-metadata">

**Author:** ![jason-arkens17](https://avatars.discourse-cdn.com/v4/letter/j/f07891/32.png) [@jason-arkens17](https://community.wandb.ai/u/jason-arkens17)\
**Post date:** [October 2, 2024, 4:32pm UTC](https://community.wandb.ai/t/initial-timeout-after-90-sec/8041/4 "2024-10-02T16:32:17Z")

</div>

Hi there! Thanks for clarifying. It may also be useful to have a look at the `output.log` file and the `debug.log` and `debug-internal.log` files from the `./wandb/run-date_time-runid/` folder (located in files and logs subfolders) if you have access to the working directory where the experiment is running

---

<div class="post-metadata">

**Author:** ![jason-arkens17](https://avatars.discourse-cdn.com/v4/letter/j/f07891/32.png) [@jason-arkens17](https://community.wandb.ai/u/jason-arkens17)\
**Post date:** [October 4, 2024, 3:13pm UTC](https://community.wandb.ai/t/initial-timeout-after-90-sec/8041/5 "2024-10-04T15:13:28Z")

</div>

Hi there, I wanted to follow up on this request. Please let us know if we can be of further assistance or if your issue has been resolved.

---

<div class="post-metadata">

**Author:** ![pl03818948](https://avatars.discourse-cdn.com/v4/letter/p/bc8723/32.png) [@pl03818948](https://community.wandb.ai/u/pl03818948)\
**Post date:** [October 5, 2024, 7:42pm UTC](https://community.wandb.ai/t/initial-timeout-after-90-sec/8041/6 "2024-10-05T19:42:11Z")

</div>

Hi, I have the same error, I can provide with the `.log` file.

# First, code of a.py:

import wandb  
import random # for demo script  
import os

wandb\_api\_key = xxx  
wandb.login(key=wandb\_api\_key, relogin=True)

epochs = 10  
lr = 0.01

run = wandb.init(name = “jj”)

offset = random.random() / 5  
print(f"lr: {lr}")

for epoch in range(2, epochs):  
acc = 1 - 2\*\*-epoch - random.random() / epoch - offset  
loss = 2\*\*-epoch + random.random() / epoch + offset  
print(f"epoch={epoch}, accuracy={acc}, loss={loss}")  
wandb.log({“accuracy”: acc, “loss”: loss})

When I run “python a.py” in linux, it works. But when I run sbatch b.sh, it does not work!

# b.sh:

#!/bin/bash -l

# 

#SBATCH --nodes=1 # Resource requirements, job runtime, other options  
#SBATCH --ntasks-per-node=1 #All #SBATCH lines have to follow uninterrupted  
#SBATCH --time=24:00:00  
#SBATCH --job-name=zephyr-7b\_dpo\_4gpu  
#SBATCH --export=NONE # do not export environment from submitting shell  
#SBATCH --output=zephyr-7b\_dpo\_4gpu.txt  
#SBATCH --cpus-per-gpu=8  
#SBATCH --gres=gpu:a40:1

cd xxx  
source ~/.bashrc  
conda activate simpo  
echo “start to run”  
python -c “import torch; print(torch.cuda.is\_available())”

python a.py

---

<div class="post-metadata">

**Author:** ![pl03818948](https://avatars.discourse-cdn.com/v4/letter/p/bc8723/32.png) [@pl03818948](https://community.wandb.ai/u/pl03818948)\
**Post date:** [October 5, 2024, 7:48pm UTC](https://community.wandb.ai/t/initial-timeout-after-90-sec/8041/7 "2024-10-05T19:48:47Z")

</div>

debug.log:  
2024-10-05 21:38:57,704 INFO MainThread:1173237 [wandb\_setup.py:\_flush():79] Current SDK version is 0.18.3  
2024-10-05 21:38:57,704 INFO MainThread:1173237 [wandb\_setup.py:\_flush():79] Configure stats pid to 1173237  
2024-10-05 21:38:57,704 INFO MainThread:1173237 [wandb\_setup.py:\_flush():79] Loading settings from /home/hpc/v100dd/v100dd18/.config/wandb/settings  
2024-10-05 21:38:57,704 INFO MainThread:1173237 [wandb\_setup.py:\_flush():79] Loading settings from /home/hpc/v100dd/v100dd18/dpo/trl/wandb/settings  
2024-10-05 21:38:57,704 INFO MainThread:1173237 [wandb\_setup.py:\_flush():79] Loading settings from environment variables: {}  
2024-10-05 21:38:57,704 INFO MainThread:1173237 [wandb\_setup.py:\_flush():79] Inferring run settings from compute environment: {‘program\_relpath’: ‘fei.py’, ‘program\_abspath’: ‘/home/hpc/v100dd/v100dd18/dpo/trl/fei.py’, ‘program’: ‘/home/hpc/v100dd/v100dd18/dpo/trl/fei.py’}  
2024-10-05 21:38:57,704 INFO MainThread:1173237 [wandb\_setup.py:\_flush():79] Applying login settings: {‘api\_key’: ‘_ **REDACTED** _’}  
2024-10-05 21:38:57,704 INFO MainThread:1173237 [wandb\_setup.py:\_flush():79] Applying login settings: {‘api\_key’: ‘_ **REDACTED** _’}  
2024-10-05 21:38:57,704 INFO MainThread:1173237 [wandb\_setup.py:\_flush():79] Applying login settings: {}  
2024-10-05 21:38:57,704 INFO MainThread:1173237 [wandb\_init.py:\_log\_setup():532] Logging user logs to /home/hpc/v100dd/v100dd18/dpo/trl/wandb/run-20241005\_213857-9pvgjp31/logs/debug.log  
2024-10-05 21:38:57,704 INFO MainThread:1173237 [wandb\_init.py:\_log\_setup():533] Logging internal logs to /home/hpc/v100dd/v100dd18/dpo/trl/wandb/run-20241005\_213857-9pvgjp31/logs/debug-internal.log  
2024-10-05 21:38:57,704 INFO MainThread:1173237 [wandb\_init.py:init():617] calling init triggers  
2024-10-05 21:38:57,704 INFO MainThread:1173237 [wandb\_init.py:init():624] wandb.init called with sweep\_config: {}  
config: {}  
2024-10-05 21:38:57,704 INFO MainThread:1173237 [wandb\_init.py:init():667] starting backend  
2024-10-05 21:38:57,705 INFO MainThread:1173237 [wandb\_init.py:init():671] sending inform\_init request  
2024-10-05 21:38:57,706 INFO MainThread:1173237 [backend.py:\_multiprocessing\_setup():104] multiprocessing start\_methods=fork,spawn,forkserver, using: spawn  
2024-10-05 21:38:57,707 INFO MainThread:1173237 [wandb\_init.py:init():684] backend started and connected  
2024-10-05 21:38:57,708 INFO MainThread:1173237 [wandb\_init.py:init():779] updated telemetry  
2024-10-05 21:38:57,709 INFO MainThread:1173237 [wandb\_init.py:init():812] communicating run to backend with 90.0 second timeout  
2024-10-05 21:39:55,999 INFO Thread-1 (wrapped\_target):1173237 [retry.py: **call** ():172] Retry attempt failed:  
Traceback (most recent call last):  
File “/home/hpc/v100dd/v100dd18/miniconda3/envs/simpo/lib/python3.10/site-packages/urllib3/connection.py”, line 199, in \_new\_conn  
sock = connection.create\_connection(  
File “/home/hpc/v100dd/v100dd18/miniconda3/envs/simpo/lib/python3.10/site-packages/urllib3/util/connection.py”, line 85, in create\_connection  
raise err  
File “/home/hpc/v100dd/v100dd18/miniconda3/envs/simpo/lib/python3.10/site-packages/urllib3/util/connection.py”, line 73, in create\_connection  
sock.connect(sa)  
TimeoutError: timed out

The above exception was the direct cause of the following exception:

Traceback (most recent call last):  
File “/home/hpc/v100dd/v100dd18/miniconda3/envs/simpo/lib/python3.10/site-packages/urllib3/connectionpool.py”, line 789, in urlopen  
response = self.\_make\_request(  
File “/home/hpc/v100dd/v100dd18/miniconda3/envs/simpo/lib/python3.10/site-packages/urllib3/connectionpool.py”, line 490, in \_make\_request  
raise new\_e  
File “/home/hpc/v100dd/v100dd18/miniconda3/envs/simpo/lib/python3.10/site-packages/urllib3/connectionpool.py”, line 466, in \_make\_request  
self.\_validate\_conn(conn)  
File “/home/hpc/v100dd/v100dd18/miniconda3/envs/simpo/lib/python3.10/site-packages/urllib3/connectionpool.py”, line 1095, in \_validate\_conn  
conn.connect()  
File “/home/hpc/v100dd/v100dd18/miniconda3/envs/simpo/lib/python3.10/site-packages/urllib3/connection.py”, line 693, in connect  
self.sock = sock = self.\_new\_conn()  
File “/home/hpc/v100dd/v100dd18/miniconda3/envs/simpo/lib/python3.10/site-packages/urllib3/connection.py”, line 208, in \_new\_conn  
raise ConnectTimeoutError(  
urllib3.exceptions.ConnectTimeoutError: (\<urllib3.connection.HTTPSConnection object at 0x14e9b8a442e0\>, ‘Connection to [api.wandb.ai](http://api.wandb.ai) timed out. (connect timeout=20)’)

---

<div class="post-metadata">

**Author:** ![pl03818948](https://avatars.discourse-cdn.com/v4/letter/p/bc8723/32.png) [@pl03818948](https://community.wandb.ai/u/pl03818948)\
**Post date:** [October 5, 2024, 7:51pm UTC](https://community.wandb.ai/t/initial-timeout-after-90-sec/8041/8 "2024-10-05T19:51:10Z")

</div>

# debug-internal.log:

{“time”:“2024-10-05T21:38:57.72198004+02:00”,“level”:“INFO”,“msg”:“using version”,“core version”:“0.18.3”}  
{“time”:“2024-10-05T21:38:57.722000528+02:00”,“level”:“INFO”,“msg”:“created symlink”,“path”:“/home/hpc/v100dd/v100dd18/dpo/trl/wandb/run-20241005\_213857-9pvgjp31/logs/debug-core.log”}  
{“time”:“2024-10-05T21:38:57.726760521+02:00”,“level”:“ERROR”,“msg”:“dialing: google: could not find default credentials. See xx”}  
{“time”:“2024-10-05T21:38:57.756738939+02:00”,“level”:“INFO”,“msg”:“created new stream”,“id”:“9pvgjp31”}  
{“time”:“2024-10-05T21:38:57.756755881+02:00”,“level”:“INFO”,“msg”:“stream: started”,“id”:“9pvgjp31”}  
{“time”:“2024-10-05T21:38:57.756789084+02:00”,“level”:“INFO”,“msg”:“handler: started”,“stream\_id”:{“value”:“9pvgjp31”}}  
{“time”:“2024-10-05T21:38:57.756821766+02:00”,“level”:“INFO”,“msg”:“writer: Do: started”,“stream\_id”:{“value”:“9pvgjp31”}}  
{“time”:“2024-10-05T21:38:57.756815584+02:00”,“level”:“INFO”,“msg”:“sender: started”,“stream\_id”:{“value”:“9pvgjp31”}}  
{“time”:“2024-10-05T21:39:27.760853135+02:00”,“level”:“INFO”,“msg”:“api: retrying error”,“error”:“Post "xx”: context deadline exceeded (Client.Timeout exceeded while awaiting headers)“}  
{“time”:“2024-10-05T21:40:00.230482509+02:00”,“level”:“INFO”,“msg”:“api: retrying error”,“error”:“Post "[https://api.wandb.ai/graphql\](https://api.wandb.ai/graphql%5C)”: context deadline exceeded”}  
{“time”:“2024-10-05T21:40:29.742902224+02:00”,“level”:“INFO”,“msg”:“stream: closing”,“id”:“9pvgjp31”}  
{“time”:“2024-10-05T21:40:29.743215419+02:00”,“level”:“INFO”,“msg”:“handler: closed”,“stream\_id”:{“value”:“9pvgjp31”}}  
{“time”:“2024-10-05T21:40:29.743245806+02:00”,“level”:“INFO”,“msg”:“writer: Close: closed”,“stream\_id”:{“value”:“9pvgjp31”}}  
{“time”:“2024-10-05T21:40:29.743262648+02:00”,“level”:“ERROR”,“msg”:“sender: upsertRun:”,“error”:“failed to upsert bucket: api: failed sending: POST [https://api.wandb.ai/graphql](https://api.wandb.ai/graphql) giving up after 3 attempt(s): context canceled”}  
{“time”:“2024-10-05T21:40:29.743305961+02:00”,“level”:“ERROR”,“msg”:“runwork: ignoring record after close”,“work”:{“Record”:{“RecordType”:{“Request”:{“RequestType”:{“Defer”:{}}}},“control”:{“always\_send”:true}}}}  
{“time”:“2024-10-05T21:40:29.745116376+02:00”,“level”:“INFO”,“msg”:“sender: closed”,“stream\_id”:{“value”:“9pvgjp31”}}  
{“time”:“2024-10-05T21:40:29.745131004+02:00”,“level”:“INFO”,“msg”:“stream: closed”,“id”:“9pvgjp31”}

---

<div class="post-metadata">

**Author:** ![jason-arkens17](https://avatars.discourse-cdn.com/v4/letter/j/f07891/32.png) [@jason-arkens17](https://community.wandb.ai/u/jason-arkens17)\
**Post date:** [October 8, 2024, 11:34pm UTC](https://community.wandb.ai/t/initial-timeout-after-90-sec/8041/9 "2024-10-08T23:34:13Z")

</div>

Hi there - can you please provide me a link to the project/workspace in question? Thanks!

---

<div class="post-metadata">

**Author:** ![jason-arkens17](https://avatars.discourse-cdn.com/v4/letter/j/f07891/32.png) [@jason-arkens17](https://community.wandb.ai/u/jason-arkens17)\
**Post date:** [October 10, 2024, 7:57pm UTC](https://community.wandb.ai/t/initial-timeout-after-90-sec/8041/10 "2024-10-10T19:57:20Z")

</div>

Hi there, I wanted to follow up on this request. Please let us know if we can be of further assistance or if your issue has been resolved.

---

<div class="post-metadata">

**Author:** ![jason-arkens17](https://avatars.discourse-cdn.com/v4/letter/j/f07891/32.png) [@jason-arkens17](https://community.wandb.ai/u/jason-arkens17)\
**Post date:** [October 11, 2024, 5:03pm UTC](https://community.wandb.ai/t/initial-timeout-after-90-sec/8041/11 "2024-10-11T17:03:51Z")

</div>

Hi Rizwan, since we have not heard back from you we are going to close this request. If you would like to re-open the conversation, please let us know!

---

<div class="post-metadata">

**Author:** ![himanshu\_nitrr](https://avatars.discourse-cdn.com/v4/letter/h/3ab097/32.png) [@himanshu\_nitrr](https://community.wandb.ai/u/himanshu_nitrr)\
**Post date:** [April 30, 2025, 3:26pm UTC](https://community.wandb.ai/t/initial-timeout-after-90-sec/8041/12 "2025-04-30T15:26:02Z")

</div>

Getting the same error:  
[rank0]: File “/group/anantm-g00/hmaurya/envs/llava-fac/lib/python3.10/site-packages/wandb/sdk/wandb\_init.py”, line 1065, in init  
[rank0]: raise CommError(  
[rank0]: wandb.errors.errors.CommError: Run initialization has timed out after 120.0 sec. Please try increasing the timeout with the `init_timeout` setting: `wandb.init(settings=wandb.Settings(init_timeout=120))`.