# Interprete gradient graphs

**URL:** https://community.wandb.ai/t/interprete-gradient-graphs/3052
**Category:** W&B Help
**Tags:** beginner-friendly, wandb, dashboard
**Created:** [September 2, 2022, 1:25pm UTC](https://community.wandb.ai/t/interprete-gradient-graphs/3052 "2022-09-02T13:25:01Z")
**Posts on this page:** 4
**Page:** 1

<div class="post-metadata">

### Author: ![thongnt](https://sea2.discourse-cdn.com/flex020/user_avatar/community.wandb.ai/thongnt/32/960_2.png) [@thongnt](https://community.wandb.ai/u/thongnt)
#### Post date: [September 2, 2022, 1:25pm UTC](https://community.wandb.ai/t/interprete-gradient-graphs/3052/1 "2022-09-02T13:25:01Z")

</div>

Hi everyone,  
Attached is the gradient histograms of my training. It seems that the gradient for lin1.weight and line2.weight are mostly zero everywhere. Does it mean that the model doesn’t learn anything from these parameters and should I exclude them my optimizer?

Thank you very much

 ![image](https://us1.discourse-cdn.com/flex020/uploads/wandb/original/1X/d4407acdc05fd1bbf26d360e1cc8c98ee256e367.png)

---

<div class="post-metadata">

### Author: ![mohammadbakir](https://avatars.discourse-cdn.com/v4/letter/m/a9a28c/32.png) [@mohammadbakir](https://community.wandb.ai/u/mohammadbakir)
#### Post date: [September 7, 2022, 8:41pm UTC](https://community.wandb.ai/t/interprete-gradient-graphs/3052/2 "2022-09-07T20:41:05Z")

</div>

Hi @thongnt , it’s difficult to say why your gradients are zeroed out. Assuming it’s not an error in your code, you may be encountering a vanishing gradient which could be leading to overflow / underflow issues. Here are some debugging steps I can suggest. 1) ensure the that you’re calling `optimizer.zero_grad()` before each batch 2). try normalizing the weights and inputs 3). Try implementing gradient clipping. Please let me know if any of these work for you.

---

<div class="post-metadata">

### Author: ![mohammadbakir](https://avatars.discourse-cdn.com/v4/letter/m/a9a28c/32.png) [@mohammadbakir](https://community.wandb.ai/u/mohammadbakir)
#### Post date: [September 12, 2022, 11:12pm UTC](https://community.wandb.ai/t/interprete-gradient-graphs/3052/3 "2022-09-12T23:12:34Z")

</div>

Hi @thongnt , since we have not heard back from you we are going to close this request. If you would like to re-open the conversation, please let us know!

---

<div class="post-metadata">

### Author: ![system](https://us1.discourse-cdn.com/flex020/uploads/wandb/original/1X/366b649231631dbab896843020da0056074ac79d.png) [@system](https://community.wandb.ai/u/system)
#### Post date: [November 11, 2022, 11:12pm UTC](https://community.wandb.ai/t/interprete-gradient-graphs/3052/4 "2022-11-11T23:12:37Z")

</div>

This topic was automatically closed 60 days after the last reply. New replies are no longer allowed.