# SOLVED: Runs are not logged separately

**URL:** <https://community.wandb.ai/t/solved-runs-are-not-logged-separately/1103>\
**Category:** W&B Help\
**Tags:** wandb\
**Created:** [October 26, 2021, 5:27pm UTC](https://community.wandb.ai/t/solved-runs-are-not-logged-separately/1103 "2021-10-26T17:27:02Z")\
**Posts on this page:** 5\
**Page:** 1

<div class="post-metadata">

**Author:** ![wardanie](https://avatars.discourse-cdn.com/v4/letter/w/d2c977/32.png) [@wardanie](https://community.wandb.ai/u/wardanie)\
**Post date:** [October 26, 2021, 5:27pm UTC](https://community.wandb.ai/t/solved-runs-are-not-logged-separately/1103/1 "2021-10-26T17:27:02Z")

</div>

Hello, I’m trying to run several studies and have them log into separate runs but be grouped un der the name “Scenario Eval” as part of the project “Scenario”. I have one python script running a for-loop with multiple calls of the following lines:

```auto
for STUDY_NAME, study ...:
    wandb_logger = WandbLogger(project="Scenario", name=STUDY_NAME+'_EVAL_'+str(study.best_params['lr']),
group='Scenario Eval',
 log_model="all",
 reinit=True)
            trainer = pl.Trainer(
                stochastic_weight_avg=False,
                logger=wandb_logger,
                checkpoint_callback=False,
                log_every_n_steps=10,
                default_root_dir = cache_path,
                max_epochs=N_EPOCHS_EVAL,
                gpus=1 if torch.cuda.is_available() else None,
            )
            wandb_logger.watch(model)
            trainer.fit(model, datamodule=datamodule)
            trainer.logger.log_hyperparams(hyperparameters)

```

In Wandb everything shows up as a single run, despite multiple loggers being created in python with explicit “reinit=True”.

 ![W&B Chart 26.10.2021, 19_30_41](https://us1.discourse-cdn.com/flex020/uploads/wandb/original/1X/d1e8a5e923f3641d8ba225411a83542eece543f7.png)  
Do I need to do something else to separate these runs?

---

<div class="post-metadata">

**Author:** ![wardanie](https://avatars.discourse-cdn.com/v4/letter/w/d2c977/32.png) [@wardanie](https://community.wandb.ai/u/wardanie)\
**Post date:** [October 26, 2021, 5:42pm UTC](https://community.wandb.ai/t/solved-runs-are-not-logged-separately/1103/2 "2021-10-26T17:42:43Z")

</div>

Fix mentioned here:

> <https://github.com/PyTorchLightning/pytorch-lightning/issues/5212#issuecomment-748898571>
>
> \#### What is your question?
> 
> What is the proposed approach to log multiple run…s (e.g., models with different hyperparameters) in one script using the WandbLogger?
> 
> I use optuna for hyperparameter tuning and create a wandb logger in each run. Although I have unique names per run (\`WandbLogger(name='unique\_name', project='MNIST')\`), the runs are not logged separately. Instead, they are put in the same wandb run-file. After the first run, every logging step gives the following warning: \`wandb: WARNING Step must only increase in log calls. Step 599 \< 644; dropping {'train\_loss': 0.098867267370224, 'train\_acc': 0.96484375}.\`, obviously because the step starts again at 0 for the second run.
> 
> Edit for clarity: I expect each run to appear as separate wandb run, such that I can compare training progress etc.
> 
> \#### Code
> 
> I copied a basic MNIST example from \[here\](https://colab.research.google.com/drive/16d1uctGaw2y9KhGBlINNTsWpmlXdJwRW#scrollTo=NI1Bh8CGI-FG). The data and model definition don't matter here, just the logging part at the end.
> 
> \`\`\`python
> \# Weights & Biases
> from pytorch\_lightning.loggers import WandbLogger
> 
> \# Pytorch modules
> import torch
> from torch.nn import functional as F
> from torch.optim import Adam
> from torch.utils.data import DataLoader, random\_split
> 
> \# Pytorch-Lightning
> from pytorch\_lightning import LightningDataModule, LightningModule, Trainer
> import pytorch\_lightning as pl
> 
> \# Dataset
> from torchvision.datasets import MNIST
> from torchvision import transforms
> 
> 
> class LitMNIST(LightningModule):
> 
> def \_\_init\_\_(self, n\_classes=10, n\_layer\_1=128, n\_layer\_2=256, lr=1e-3):
> '''method used to define our model parameters'''
> super().\_\_init\_\_()
> 
> # mnist images are (1, 28, 28) (channels, width, height)
> self.layer\_1 = torch.nn.Linear(28 \* 28, n\_layer\_1)
> self.layer\_2 = torch.nn.Linear(n\_layer\_1, n\_layer\_2)
> self.layer\_3 = torch.nn.Linear(n\_layer\_2, n\_classes)
> 
> # optimizer parameters
> self.lr = lr
> 
> # metrics
> self.accuracy = pl.metrics.Accuracy()
> 
> # optional - save hyper-parameters to self.hparams
> # they will also be automatically logged as config parameters in W&B
> self.save\_hyperparameters()
> 
> def forward(self, x):
> '''method used for inference input -\> output'''
> 
> batch\_size, channels, width, height = x.size()
> 
> # (b, 1, 28, 28) -\> (b, 1\*28\*28)
> x = x.view(batch\_size, -1)
> x = self.layer\_1(x)
> x = F.relu(x)
> x = self.layer\_2(x)
> x = F.relu(x)
> x = self.layer\_3(x)
> 
> x = F.log\_softmax(x, dim=1)
> return x
> 
> def training\_step(self, batch, batch\_idx):
> '''needs to return a loss from a single batch'''
> x, y = batch
> logits = self(x)
> loss = F.nll\_loss(logits, y)
> 
> # Log training loss
> self.log('train\_loss', loss)
> 
> # Log metrics
> self.log('train\_acc', self.accuracy(logits, y))
> 
> return loss
> 
> def validation\_step(self, batch, batch\_idx):
> '''used for logging metrics'''
> x, y = batch
> logits = self(x)
> loss = F.nll\_loss(logits, y)
> 
> # Log validation loss (will be automatically averaged over an epoch)
> self.log('valid\_loss', loss)
> 
> # Log metrics
> self.log('valid\_acc', self.accuracy(logits, y))
> 
> def test\_step(self, batch, batch\_idx):
> '''used for logging metrics'''
> x, y = batch
> logits = self(x)
> loss = F.nll\_loss(logits, y)
> 
> # Log test loss
> self.log('test\_loss', loss)
> 
> # Log metrics
> self.log('test\_acc', self.accuracy(logits, y))
> 
> def configure\_optimizers(self):
> '''defines model optimizer'''
> return Adam(self.parameters(), lr=self.lr)
> 
> 
> class MNISTDataModule(LightningDataModule):
> 
> def \_\_init\_\_(self, data\_dir='./', batch\_size=256):
> super().\_\_init\_\_()
> self.data\_dir = data\_dir
> self.batch\_size = batch\_size
> self.transform = transforms.ToTensor()
> 
> def prepare\_data(self):
> '''called only once and on 1 GPU'''
> # download data
> MNIST(self.data\_dir, train=True, download=True)
> MNIST(self.data\_dir, train=False, download=True)
> 
> def setup(self, stage=None):
> '''called one ecah GPU separately - stage defines if we are at fit or test step'''
> # we set up only relevant datasets when stage is specified (automatically set by Pytorch-Lightning)
> if stage == 'fit' or stage is None:
> mnist\_train = MNIST(self.data\_dir, train=True, transform=self.transform)
> self.mnist\_train, self.mnist\_val = random\_split(mnist\_train, \[55000, 5000\])
> if stage == 'test' or stage is None:
> self.mnist\_test = MNIST(self.data\_dir, train=False, transform=self.transform)
> 
> def train\_dataloader(self):
> '''returns training dataloader'''
> mnist\_train = DataLoader(self.mnist\_train, batch\_size=self.batch\_size)
> return mnist\_train
> 
> def val\_dataloader(self):
> '''returns validation dataloader'''
> mnist\_val = DataLoader(self.mnist\_val, batch\_size=self.batch\_size)
> return mnist\_val
> 
> def test\_dataloader(self):
> '''returns test dataloader'''
> mnist\_test = DataLoader(self.mnist\_test, batch\_size=self.batch\_size)
> return mnist\_test
> 
> 
> \# setup data
> mnist = MNISTDataModule()
> 
> \# -------------- relevant part starts here --------------
> 
> \# create logger for first run.
> wandb\_logger = WandbLogger(name='run0', project='MNIST')
> 
> \# setup model - choose different hyperparameters per experiment
> model = LitMNIST(n\_layer\_1=128, n\_layer\_2=256, lr=1e-3)
> 
> trainer = Trainer(
> logger=wandb\_logger, # W&B integration
> gpus=0, # no GPU
> max\_epochs=3 # number of epochs
> )
> 
> trainer.fit(model, mnist)
> 
> \# create logger for second run.
> wandb\_logger = WandbLogger(name='run1', project='MNIST')
> 
> \# setup model - choose different hyperparameters per experiment
> model = LitMNIST(n\_layer\_1=128, n\_layer\_2=256, lr=1e-2)
> 
> trainer = Trainer(
> logger=wandb\_logger, # W&B integration
> gpus=0, # no GPU
> max\_epochs=3 # number of epochs
> )
> 
> trainer.fit(model, mnist)
> \`\`\`
> 
> \#### What's your environment?
> 
> - OS: MacOS
> - Packaging: pip
> - Version: pytorch-lightning 1.1.1, wandb 0.10.12

seems to be just weird behaviour

---

<div class="post-metadata">

**Author:** ![\_scott](https://sea2.discourse-cdn.com/flex020/user_avatar/community.wandb.ai/_scott/32/95_2.png) [@\_scott](https://community.wandb.ai/u/_scott)\
**Post date:** [October 26, 2021, 6:30pm UTC](https://community.wandb.ai/t/solved-runs-are-not-logged-separately/1103/3 "2021-10-26T18:30:46Z")

</div>

Thanks for updating this.  
Did you solve your issue? Were you able to get different runs for your different studies?

---

<div class="post-metadata">

**Author:** ![wardanie](https://avatars.discourse-cdn.com/v4/letter/w/d2c977/32.png) [@wardanie](https://community.wandb.ai/u/wardanie)\
**Post date:** [October 26, 2021, 9:41pm UTC](https://community.wandb.ai/t/solved-runs-are-not-logged-separately/1103/4 "2021-10-26T21:41:41Z")

</div>

Yup, I’m now getting new runs for every training as expected

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/flex020/uploads/wandb/original/1X/366b649231631dbab896843020da0056074ac79d.png) [@system](https://community.wandb.ai/u/system)\
**Post date:** [April 20, 2022, 6:02pm UTC](https://community.wandb.ai/t/solved-runs-are-not-logged-separately/1103/5 "2022-04-20T18:02:06Z")

</div>

Closed on April 20, 2022, 6:02pm UTC

This topic was automatically closed 60 days after the last reply. New replies are no longer allowed.