# How to jump the W&B upload process when the network is not so good?

**URL:** <https://community.wandb.ai/t/how-to-jump-the-w-b-upload-process-when-the-network-is-not-so-good/3487>\
**Category:** W&B Help\
**Tags:** wandb, beginner-friendly\
**Created:** [December 2, 2022, 11:57am UTC](https://community.wandb.ai/t/how-to-jump-the-w-b-upload-process-when-the-network-is-not-so-good/3487 "2022-12-02T11:57:42Z")\
**Posts on this page:** 6\
**Page:** 1

<div class="post-metadata">

**Author:** ![yangze68](https://avatars.discourse-cdn.com/v4/letter/y/96bed5/32.png) [@yangze68](https://community.wandb.ai/u/yangze68)\
**Post date:** [December 2, 2022, 11:57am UTC](https://community.wandb.ai/t/how-to-jump-the-w-b-upload-process-when-the-network-is-not-so-good/3487/1 "2022-12-02T11:57:42Z")

</div>

My training process unsuccessfully ended, because of the failure of the uploading process for W&B.

 ![image](https://us1.discourse-cdn.com/flex020/uploads/wandb/original/2X/d/dbd70f7322f4960651abd3f6fae4f09d756e5380.png)  
I often use a script file to run multiple experiments at once. When one of it is tucked, others cannot be run.  
How to jump this?

---

<div class="post-metadata">

**Author:** ![mohammadbakir](https://avatars.discourse-cdn.com/v4/letter/m/a9a28c/32.png) [@mohammadbakir](https://community.wandb.ai/u/mohammadbakir)\
**Post date:** [December 2, 2022, 10:55pm UTC](https://community.wandb.ai/t/how-to-jump-the-w-b-upload-process-when-the-network-is-not-so-good/3487/2 "2022-12-02T22:55:15Z")

</div>

Hi @yangze68 , happy to help. From the image attached the `Network error (TransientError) points` to potential packet loss attributed to a network error on the users end. Event though a single experiment might fail, wandb would still execute subsequent runs, depending on how a user sets up their experiments.

- Could you please expand on how you are setting up your experiments and what additional errors you are seeing that stop subsequent runs
- Provide a `debug.log` and `debug-internal.log` file of the crashing run for us to get a better sense of anything else is happening with the run. These are located in the wandb/ folder of the working directory of the project.

Thank you

---

<div class="post-metadata">

**Author:** ![yangze68](https://avatars.discourse-cdn.com/v4/letter/y/96bed5/32.png) [@yangze68](https://community.wandb.ai/u/yangze68)\
**Post date:** [December 3, 2022, 3:41am UTC](https://community.wandb.ai/t/how-to-jump-the-w-b-upload-process-when-the-network-is-not-so-good/3487/3 "2022-12-03T03:41:43Z")

</div>

Hi @mohammadbakir, thanks for your reply. If I set the wand mode to offline, I don’t need to upload the file every time. But the new error accused. When I use the command `wandb sync --sync-all` to upload the offline file, the upload speed is very slow, I think this problem is related to the sync with tensorboad, which is mentioned in [[CLI] Slow uploads of offline runs · Issue #1972 · wandb/wandb · GitHub](https://github.com/wandb/wandb/issues/1972)  
How can I share the debug.log file with you? By e-mail or here?  
Thanks again for your help

When I copy the cached folder to another machine and upload it successfully.  
And it throws a warning

 ![image](https://us1.discourse-cdn.com/flex020/uploads/wandb/original/2X/7/786130bf568ecc3bfed9ba66b12e6dcb78f01713.png)  
I think there’s a bug with sync with tensorboad which slow the process.

---

<div class="post-metadata">

**Author:** ![mohammadbakir](https://avatars.discourse-cdn.com/v4/letter/m/a9a28c/32.png) [@mohammadbakir](https://community.wandb.ai/u/mohammadbakir)\
**Post date:** [December 8, 2022, 11:11pm UTC](https://community.wandb.ai/t/how-to-jump-the-w-b-upload-process-when-the-network-is-not-so-good/3487/4 "2022-12-08T23:11:29Z")

</div>

Thank you for the update @yangze68 . Please send them to `support@wandb.ai` and include my name in the subject line. I will perform some tests on the offline syncing of tensorboard output to wandb and get back to you with my findings.

---

<div class="post-metadata">

**Author:** ![mohammadbakir](https://avatars.discourse-cdn.com/v4/letter/m/a9a28c/32.png) [@mohammadbakir](https://community.wandb.ai/u/mohammadbakir)\
**Post date:** [December 14, 2022, 6:39pm UTC](https://community.wandb.ai/t/how-to-jump-the-w-b-upload-process-when-the-network-is-not-so-good/3487/5 "2022-12-14T18:39:07Z")

</div>

Hi @yangze68 , since we have not heard back from you we are going to close this request. If you would like to re-open the conversation, please let us know!

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/flex020/uploads/wandb/original/1X/366b649231631dbab896843020da0056074ac79d.png) [@system](https://community.wandb.ai/u/system)\
**Post date:** [February 12, 2023, 6:39pm UTC](https://community.wandb.ai/t/how-to-jump-the-w-b-upload-process-when-the-network-is-not-so-good/3487/6 "2023-02-12T18:39:26Z")

</div>

Closed on February 12, 2023, 6:39pm UTC

This topic was automatically closed 60 days after the last reply. New replies are no longer allowed.
