How to read parameter and gradient plots generated from wandb.watch()

Show the Community!
1

Hi all,

I’ve running wandb.watch() to track the parameters and gradients in my model - however I’m not really sure how to go about interpreting the resulting graphs.

1 Like
2

Have a look at this, there are also other blogposts out there (look for gradient debugging in deep learning).

Overall it is a bit of a wizardry but can really help testing your assumptions. E.g. you can see if the gradients are changing, are peaking, are too normally distributed, affect early layers and also if the parameters reflect non-gaussians etc. So it is not going to explain the model to you but can serve as a low level debugging tool in my opinion.

2 Likes