New to Wandb making sense of the gradient dashboard, am I seeing exploding gradients?

W&B Help
, ,
1

Hi

I’m running my first experiments with weights and biases, I find it pretty cool and quite intuitive to use.

The dashboard is a little confusing at the moment though
I have random initialized weights, and they seem to gro pretty quickly and then stagnate.
The absolute value still seems very small so I’m not sure fi this is due to scaling, are these exploding gradients?

Thanks for any insights and help understanding and getting the most out of this awesome tool.

image
image1785×610 53.4 KB

2

Hi Oliver,

Here’s an article we have that explains what you are seeing in your gradients (https://wandb.ai/site/articles/exploring-gradients)

Warmly,
Leslie

3

Hi there, I wanted to follow up on this request. Please let us know if we can be of further assistance or if your issue has been resolved.

4

Hi Oliver, since we have not heard back from you we are going to close this request. If you would like to re-open the conversation, please let us know!