# Showing total loss in distributed computing **URL:** \ **Category:** W&B Help\ **Tags:** wandb, beginner-friendly\ **Created:** [August 13, 2022, 10:40pm UTC](https://community.wandb.ai/t/showing-total-loss-in-distributed-computing/2923 "2022-08-13T22:40:05Z")\ **Posts on this page:** 7\ **Page:** 1
**Author:** ![mkhoshle](https://sea2.discourse-cdn.com/flex020/user_avatar/community.wandb.ai/mkhoshle/32/809_2.png) [@mkhoshle](https://community.wandb.ai/u/mkhoshle)\ **Post date:** [August 13, 2022, 10:40pm UTC](https://community.wandb.ai/t/showing-total-loss-in-distributed-computing/2923/1 "2022-08-13T22:40:05Z")
Hello, I am trying to use wandb to monitor my loss functions and I am running my code on multiple GPU nodes. When logging my loss function I can see 4 different links for each node. However, I would like to see the total loss and have one process reporting that through wandb. How should I take care of this? I would appreciate any feedback as I am totally new to this community, Thanks, ---
**Author:** ![thanos-wandb](https://avatars.discourse-cdn.com/v4/letter/t/4af34b/32.png) [@thanos-wandb](https://community.wandb.ai/u/thanos-wandb)\ **Post date:** [August 17, 2022, 8:14pm UTC](https://community.wandb.ai/t/showing-total-loss-in-distributed-computing/2923/2 "2022-08-17T20:14:15Z")
Hi @mkhoshle welcome to W&B community! Is it one experiment distributed in 4 GPUs or are you running 4 different experiments? Which framework are you using? Any code snippet or link to the workspace would help to further investigate this issue. Would [this](https://github.com/wandb/examples/tree/master/examples/pytorch/pytorch-ddp#method-1-log-from-a-single-process) example with PyTorch DDP fit your case? You may also find useful our reference docs [here](https://docs.wandb.ai/guides/track/advanced/distributed-training) about distributed training. ---
**Author:** ![mkhoshle](https://sea2.discourse-cdn.com/flex020/user_avatar/community.wandb.ai/mkhoshle/32/809_2.png) [@mkhoshle](https://community.wandb.ai/u/mkhoshle)\ **Post date:** [August 17, 2022, 8:48pm UTC](https://community.wandb.ai/t/showing-total-loss-in-distributed-computing/2923/3 "2022-08-17T20:48:32Z")
Hi @thanos-wandb , Yes, I am running my experiment using multiple GPUs (e.g. 4 GPUs) and yes PyTorch DistributedDataParallel is my case. I just need some information regarding when I should log on all processes and when I should log using only one process. Which would be more useful and provide more insight? Thanks ---
**Author:** ![mohammadbakir](https://avatars.discourse-cdn.com/v4/letter/m/a9a28c/32.png) [@mohammadbakir](https://community.wandb.ai/u/mohammadbakir)\ **Post date:** [August 29, 2022, 7:04pm UTC](https://community.wandb.ai/t/showing-total-loss-in-distributed-computing/2923/4 "2022-08-29T19:04:56Z")
Hi @mkhoshle , this [document](https://docs.wandb.ai/guides/track/advanced/distributed-training#logging-distributed-training-experiments-with-w-and-b) may be helpful, listing two methods recommend by W&B for multiprocessing logging. Method 1, logging through the `rank0` process would work for you case of only wanting to log a single value from a single process. ---
**Author:** ![mohammadbakir](https://avatars.discourse-cdn.com/v4/letter/m/a9a28c/32.png) [@mohammadbakir](https://community.wandb.ai/u/mohammadbakir)\ **Post date:** [September 1, 2022, 10:12pm UTC](https://community.wandb.ai/t/showing-total-loss-in-distributed-computing/2923/5 "2022-09-01T22:12:25Z")
Hi @mkhoshle , do you still need assistance with W&B multiprocessing? ---
**Author:** ![mohammadbakir](https://avatars.discourse-cdn.com/v4/letter/m/a9a28c/32.png) [@mohammadbakir](https://community.wandb.ai/u/mohammadbakir)\ **Post date:** [September 9, 2022, 11:55pm UTC](https://community.wandb.ai/t/showing-total-loss-in-distributed-computing/2923/6 "2022-09-09T23:55:16Z")
Hi @mkhoshle , since we have not heard back from you we are going to close this request. If you would like to re-open the conversation, please let us know! ---
**Author:** ![system](https://us1.discourse-cdn.com/flex020/uploads/wandb/original/1X/366b649231631dbab896843020da0056074ac79d.png) [@system](https://community.wandb.ai/u/system)\ **Post date:** [November 8, 2022, 11:55pm UTC](https://community.wandb.ai/t/showing-total-loss-in-distributed-computing/2923/7 "2022-11-08T23:55:19Z")
This topic was automatically closed 60 days after the last reply. New replies are no longer allowed.