# Wandb port unreachable from SLURM **URL:** \ **Category:** W&B Help\ **Tags:** wandb\ **Created:** [June 5, 2024, 9:17am UTC](https://community.wandb.ai/t/wandb-port-unreachable-from-slurm/6848 "2024-06-05T09:17:09Z")\ **Posts on this page:** 6\ **Page:** 1
**Author:** ![malla-bhavana26](https://avatars.discourse-cdn.com/v4/letter/m/c4cdca/32.png) [@malla-bhavana26](https://community.wandb.ai/u/malla-bhavana26)\ **Post date:** [June 5, 2024, 9:17am UTC](https://community.wandb.ai/t/wandb-port-unreachable-from-slurm/6848/1 "2024-06-05T09:17:09Z")
Hi, I was logging to Wandb server from the SLURM cluster and it was working without any issues till yesterday. But the connection has started failing suddenly. I tried to relogin but it doesnt seem to fix the issue. I tried to ping the server using “ping [api.wandb.ai](http://api.wandb.ai)”. I can see the server is trying to send data to the wandb remote server, but I see that “Destination Port Unreachable” message. PING [api.wandb.ai](http://api.wandb.ai) (35.186.228.49) 56(84) bytes of data. From xxxxxxx.xxxxx.xxxx icmp\_seq=1 Destination Port Unreachable ping: sendmsg: Operation not permitted I checked the debug logs and I see the connection error from urllib. Could you please help me solve the issue? ## Error Trace: Traceback (most recent call last): File “/xxxx/xxxx/anaconda3/envs/xxxxx/lib/python3.8/site-packages/urllib3/connection.py”, line 174, in \_new\_conn conn = connection.create\_connection( File “/xxxx/xxxx/anaconda3/envs/xxxxx/lib/python3.8/site-packages/urllib3/util/connection.py”, line 95, in create\_connection raise err File “/xxxx/xxxx/anaconda3/envs/xxxxx/lib/python3.8/site-packages/urllib3/util/connection.py”, line 85, in create\_connection sock.connect(sa) ConnectionRefusedError: [Errno 111] Connection refused During handling of the above exception, another exception occurred: Traceback (most recent call last): File “/xxxx/xxxx/anaconda3/envs/xxxxx/thesis/lib/python3.8/site-packages/urllib3/connectionpool.py”, line 715, in urlopen httplib\_response = self.\_make\_request( File “/xxxx/xxxx/anaconda3/envs/xxxxx/lib/python3.8/site-packages/urllib3/connectionpool.py”, line 404, in \_make\_request self.\_validate\_conn(conn) File “/xxxx/xxxx/anaconda3/envs/xxxxx/lib/python3.8/site-packages/urllib3/connectionpool.py”, line 1058, in \_validate\_conn conn.connect() File “/xxxx/xxxx/anaconda3/envs/xxxxx/lib/python3.8/site-packages/urllib3/connection.py”, line 363, in connect self.sock = conn = self.\_new\_conn() File “/xxxx/xxxx/anaconda3/envs/xxxxx/lib/python3.8/site-packages/urllib3/connection.py”, line 186, in \_new\_conn raise NewConnectionError( urllib3.exceptions.NewConnectionError: \: Failed to establish a new connection: [Errno 111] Connection refused During handling of the above exception, another exception occurred: Traceback (most recent call last): File “/xxxx/xxxx/anaconda3/envs/xxxxx/lib/python3.8/site-packages/requests/adapters.py”, line 486, in send resp = conn.urlopen( File “/xxxx/xxxx/anaconda3/envs/xxxxx/lib/python3.8/site-packages/urllib3/connectionpool.py”, line 799, in urlopen retries = retries.increment( File “/xxxx/xxxx/anaconda3/envs/xxxxx/lib/python3.8/site-packages/urllib3/util/retry.py”, line 592, in increment raise MaxRetryError(\_pool, url, error or ResponseError(cause)) urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host=‘[api.wandb.ai](http://api.wandb.ai)’, port=443): Max retries exceeded with url: /graphql (Caused by NewConnectionError(‘\: Failed to establish a new connection: [Errno 111] Connection refused’)) ---
**Author:** ![system](https://us1.discourse-cdn.com/flex020/uploads/wandb/original/1X/366b649231631dbab896843020da0056074ac79d.png) [@system](https://community.wandb.ai/u/system)\ **Post date:** [June 5, 2024, 9:23am UTC](https://community.wandb.ai/t/wandb-port-unreachable-from-slurm/6848/2 "2024-06-05T09:23:10Z")
Hello, This a reply from our support bot designed to assist you with your Weights & Biases related queries. To reach a human please reply to this message. It seems like you’re encountering a network connectivity issue that prevents your SLURM cluster from communicating with the Weights & Biases server. This could be due to several reasons, including network configuration changes, firewall rules, or issues with the Weights & Biases server itself. Here are some steps to troubleshoot and potentially resolve the issue: 1. **Verify Network Configuration** : Ensure that the network configuration on your SLURM cluster hasn’t changed. Sometimes, network policies or firewall settings are updated, which can block outgoing connections to specific ports or addresses. 2. **Check Firewall Rules** : Verify that your firewall rules allow outgoing connections to `api.wandb.ai` on port 443. You might need to work with your network administrator to ensure that these connections are not being blocked. 3. **Test Connectivity** : Use tools like `ping` and `traceroute` (or `tracert` on Windows) to test connectivity to `api.wandb.ai`. While `ping` might not always respond due to server configurations, `traceroute` can help identify where the connection is being blocked. 4. **Use `curl` or `wget`** : Try using `curl` or `wget` from your SLURM cluster to access `https://api.wandb.ai`. This can help confirm if the HTTPS connection can be established. For example: 5. **Review W&B Server Logs** : If you’re using a self-hosted W&B server, check the server logs for any errors or warnings that might indicate why connections are being refused. 6. **Check W&B Status** : Visit the Weights & Biases status page to ensure there are no ongoing issues with the W&B service that could be affecting connectivity. 7. **SLURM Configuration** : Ensure that your SLURM job scripts do not contain any settings that could restrict network access. Some clusters have specific partitions or configurations for jobs requiring external network access. 8. **Contact W&B Support** : If you’ve gone through these steps and are still facing issues, it might be helpful to contact Weights & Biases support for further assistance. You can reach out to them at [support@wandb.com](mailto:support@wandb.com). Given the error trace you provided, it’s clear that the issue lies with establishing a connection to the W&B API server. This is typically a network-level issue rather than a problem with your code or the W&B client library itself. Therefore, focusing on network troubleshooting steps is likely to be the most effective approach to resolving this issue. To reach a human please reply to this message. -WandBot 🤖 ---
**Author:** ![paulo-sabile](https://avatars.discourse-cdn.com/v4/letter/p/f14d63/32.png) [@paulo-sabile](https://community.wandb.ai/u/paulo-sabile)\ **Post date:** [June 14, 2024, 7:29am UTC](https://community.wandb.ai/t/wandb-port-unreachable-from-slurm/6848/3 "2024-06-14T07:29:58Z")
Hi Bhavana, Good day and thank you for reaching out to us! Happy to help you on this. Can you confirm if this issue is still happening on your instance? Did you try to relogin to your instance using this method? `wandb login --relogin --host ``http://your-host-url.com` I may also request the following so we can investigate if this issue is still happening: - W&B Local version, you can get this by running (`wandb --version`). - `debug.log` and `debug-internal.log` for the affected run? These files are under your local folder `wandb/run-_