Dear wandb Team,
I am experiencing several issues when using wandb with Lightning.
- Images are not displayed (as described in this post from last year: Select runs that logged image with the key Val_dice_Ground truth_Epoch: 3_ to visualize data here. · Issue #5337 · wandb/wandb · GitHub)
- Gradients are not logged. I initialize my model, then call wandb.init, set up my checkpoint callback, profiler, logger, then I call wandb.watch(model,log=‘all’, log_freq=1), and then set up the trainer and fit it. I have also tried initializing the model after wandb.init but this did not change behavior. Other people have pointed out similar problems in different settings, e.g. Parameters and Gradient is not logged by WandB under FSDP strategy · Issue #17512 · Lightning-AI/pytorch-lightning · GitHub
Otherwise, my logger works fine. I am not sure if 2. is cause by Lightning or wandb. I am hoping there is an easy fix, e.g. that I used the wrong order of steps.
I am grateful for any help! Thanks in advance