# How can I log best values of a metric/loss in wandb summary using Pytorch-Lightning?

**URL:** <https://community.wandb.ai/t/how-can-i-log-best-values-of-a-metric-loss-in-wandb-summary-using-pytorch-lightning/941>\
**Category:** W&B Help\
**Created:** [October 13, 2021, 6:07am UTC](https://community.wandb.ai/t/how-can-i-log-best-values-of-a-metric-loss-in-wandb-summary-using-pytorch-lightning/941 "2021-10-13T06:07:10Z")\
**Posts on this page:** 6\
**Page:** 1

<div class="post-metadata">

**Author:** ![al-3002-w](https://sea2.discourse-cdn.com/flex020/user_avatar/community.wandb.ai/al-3002-w/32/246_2.png) [@al-3002-w](https://community.wandb.ai/u/al-3002-w)\
**Post date:** [October 13, 2021, 6:07am UTC](https://community.wandb.ai/t/how-can-i-log-best-values-of-a-metric-loss-in-wandb-summary-using-pytorch-lightning/941/1 "2021-10-13T06:07:10Z")

</div>

In wandb documentation there are examples like this: `wandb.run.summary["best_accuracy"] = best_accuracy`  
Not sure how to do something like that in training\_step/validation\_step using self.log, considering the values of metrics are automatically calculated on epochs under the hood of the API.

---

<div class="post-metadata">

**Author:** ![\_scott](https://sea2.discourse-cdn.com/flex020/user_avatar/community.wandb.ai/_scott/32/95_2.png) [@\_scott](https://community.wandb.ai/u/_scott)\
**Post date:** [October 15, 2021, 10:18am UTC](https://community.wandb.ai/t/how-can-i-log-best-values-of-a-metric-loss-in-wandb-summary-using-pytorch-lightning/941/2 "2021-10-15T10:18:22Z")

</div>

Hi,  
Can you give a bit more info of what you’re trying to do? Have you tried doing the above in your own lightning module? You could also try adding a Callback which has access to the logger.

> **[PyTorch Lightning](https://docs.wandb.ai/guides/integrations/lightning#how-do-i-log-media-objects)**
>
> Build scalable, structured, high-performance PyTorch models with Lightning and log them with W&B.

---

<div class="post-metadata">

**Author:** ![al-3002-w](https://sea2.discourse-cdn.com/flex020/user_avatar/community.wandb.ai/al-3002-w/32/246_2.png) [@al-3002-w](https://community.wandb.ai/u/al-3002-w)\
**Post date:** [October 15, 2021, 11:18am UTC](https://community.wandb.ai/t/how-can-i-log-best-values-of-a-metric-loss-in-wandb-summary-using-pytorch-lightning/941/3 "2021-10-15T11:18:44Z")

</div>

I have the following code:

```
def training_step(self, batch, *args, **kwargs): # type: ignore
    pred = self(batch).squeeze(-1)
    loss = self.loss(pred, batch['p'], batch['u_out']).mean()
    self.log('train_loss', loss, on_step=True, on_epoch=True, prog_bar=True, logger=True)

    score = self.metric(pred, batch['p'], batch['u_out']).mean()
    self.log(f'train_mae', score, on_step=True, on_epoch=True, prog_bar=True, logger=True)
    return loss

```

As you can see, I calculate the metric and the loss only during the training step (and the validation step) and the values at epoch level are aggregated and logged automatically by pytorch-lightning and wandb.

I suppose I could return the loss and the metric in `training_step` and then calculate epoch level values manually in `training_epoch_end` and then log the best value.

But I hope there is a better solution.

---

<div class="post-metadata">

**Author:** ![\_scott](https://sea2.discourse-cdn.com/flex020/user_avatar/community.wandb.ai/_scott/32/95_2.png) [@\_scott](https://community.wandb.ai/u/_scott)\
**Post date:** [October 15, 2021, 4:28pm UTC](https://community.wandb.ai/t/how-can-i-log-best-values-of-a-metric-loss-in-wandb-summary-using-pytorch-lightning/941/4 "2021-10-15T16:28:03Z")

</div>

One thing you can do is use `wandb.run.define_metric("train_mae", summary="max")`

> **[wandb.Run](https://docs.wandb.ai/ref/python/run#define_metric)**

This will automatically store both the latest and the max value in the summary for filtering.

---

<div class="post-metadata">

**Author:** ![al-3002-w](https://sea2.discourse-cdn.com/flex020/user_avatar/community.wandb.ai/al-3002-w/32/246_2.png) [@al-3002-w](https://community.wandb.ai/u/al-3002-w)\
**Post date:** [October 22, 2021, 5:28pm UTC](https://community.wandb.ai/t/how-can-i-log-best-values-of-a-metric-loss-in-wandb-summary-using-pytorch-lightning/941/5 "2021-10-22T17:28:08Z")

</div>

It turns out, that wandb logger doesn’t have a possibility to define the metric or summary. In the end I use a simple hacking solution:

```auto
self.mae = score if score < self.mae else self.mae
self.log('mae', self.mae, on_step=False, on_epoch=True, prog_bar=True, logger=True)

```

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/flex020/uploads/wandb/original/1X/366b649231631dbab896843020da0056074ac79d.png) [@system](https://community.wandb.ai/u/system)\
**Post date:** [April 20, 2022, 6:02pm UTC](https://community.wandb.ai/t/how-can-i-log-best-values-of-a-metric-loss-in-wandb-summary-using-pytorch-lightning/941/6 "2022-04-20T18:02:06Z")

</div>

This topic was automatically closed 60 days after the last reply. New replies are no longer allowed.