How use Metrics with pytorch?

Hello, I was used to tensorflow and keras, where the metrics were log in a very simple way, like this:
model.compile(config.optimizer, config.loss_function, metrics=[‘accuracy’, ‘recall’, ‘AUC’])
But I want to use ResNet-18, and it doesn’t have in TensorFlow, so I decided to migrate to PyTorch. I was using this tutorial as a guideline:

The problem is that I’m not finding an intuitive way to log the metrics as I used to. My training data and test data are not being logged. Do I need to specify the metrics equations and push to log?
In this tutorial, only the test metrics are being logged, and this is the function:
def test(model, test_loader):

# Run the model on some test examples
with torch.no_grad():
    correct, total = 0, 0
    for images, labels in test_loader:
        images, labels =,
        outputs = model(images)
        _, predicted = torch.max(, 1)
        total += labels.size(0)
        correct += (predicted == labels).sum().item()

    print(f"Accuracy of the model on the {total} " +
          f"test images: {correct / total:%}")

    wandb.log({"test_accuracy": correct / total})

# Save the model in the exchangeable ONNX format
torch.onnx.export(model, images, "model.onnx")"model.onnx")

Hi @flora-ufsc24, thank you for writing in! Specifying the metircs in the wandb.log will do the tric for you and shoul add it to your wandb workspace. Could you please show me how you used to log it with tf and got the desired outcome?

Hey Artsiom!
Yes, this is what Im using in tf and it works:

#Initialize wandb with your project name
run = wandb.init(project='Test-Metrics', #Were the project goes
                 config={#and include hyperparameters and metadata
                         "optimizer": "adam",
                         "architecture": "Transferlearing_ResNet50",#Not necessary but good practices
                         "dataset": "My 1000 data 70/20/10"
config = wandb.config #We'll use this to configure our experiment

#Initialize modl like you usually do
model = create_model()

#Compile model like you usually do
#Notice that we use config, so our metadata matches what gets executed
#optimizer = tf.keras.optimizers.Adam(config.learning_rate)
model.compile(config.optimizer, config.loss_function, metrics=['accuracy','recall','AUC','precision','R2Score','TruePositives','FalsePositives'])

#We train with our beloved
#Notice WandCallback is udes as a regular callback
#We again use config,y_train,

# Avaliar o modelo usando o conjunto de teste
test_loss, test_accuracy = model.evaluate(x_test_preprocessed, y_test)

And this is how Im doing with pytorch right now:

# Função para calcular acurácia
def calculate_accuracy(outputs, labels):
    _, predicted = torch.max(outputs, 1)
    correct = (predicted == labels).sum().item()
    accuracy = correct / labels.size(0)
    return accuracy

# Variáveis para controle de early stopping
early_stopping_counter = 0
best_test_loss = float('inf')

# Loop de treinamento
for epoch in range(epochs):
    train_loss = 0.0
    train_accuracy = 0.0

    for inputs, labels in train_loader:
        inputs, labels =,


        outputs = model(inputs)
        loss = criterion(outputs, labels)

        train_loss += loss.item() * inputs.size(0)
        train_accuracy += calculate_accuracy(outputs, labels) * inputs.size(0)

    # Calcular métricas de treinamento
    train_loss /= len(train_loader.dataset)
    train_accuracy /= len(train_loader.dataset)

    # Avaliação no conjunto de teste
    test_loss = 0.0
    test_accuracy = 0.0

    with torch.no_grad():
        for inputs, labels in test_loader:
            inputs, labels =,

            outputs = model(inputs)
            loss = criterion(outputs, labels)

            test_loss += loss.item() * inputs.size(0)
            test_accuracy += calculate_accuracy(outputs, labels) * inputs.size(0)

    # Calcular métricas de teste
    test_loss /= len(test_loader.dataset)
    test_accuracy /= len(test_loader.dataset)

    # Registrar métricas no wandb apenas no final de cada época
        'epoch': epoch + 1,
        'train_loss': train_loss,
        'train_accuracy': train_accuracy,
        'test_loss': test_loss,
        'test_accuracy': test_accuracy
    }, step=epoch + 1)

    # Imprimir métricas
    print(f'Epoch [{epoch + 1}/{epochs}], '
          f'Train Loss: {train_loss:.4f}, Train Accuracy: {train_accuracy:.4f}, '
          f'Test Loss: {test_loss:.4f}, Test Accuracy: {test_accuracy:.4f}')

    # Verificar se a perda de teste diminuiu
    if test_loss < best_test_loss:
        best_test_loss = test_loss
        early_stopping_counter = 0
        early_stopping_counter += 1

    # Parar o treinamento se a perda de teste não diminuir por 10 épocas
    if early_stopping_counter >= 10:
        print("Early stopping triggered.")

# Fechar a sessão do wandb ao final do treinamento

I know that I’m not using the same metrics in both codes, but it’s just an example. I still have a lot of doubts about training in PyTorch because in Keras you just need to use model.compile , and Lucas’ YouTube videos are amazing. If you know of any other materials that teach PyTorch with Weights & Biases, I would love to hear about them. Thanks for your time."

Thank you so much for sending it over, @flora-ufsc24! Taking a look :slight_smile:

Regarding some resources of using pytorch with wandb,

We have our docs here, regarding the wandb and pytorch integration:

As well as a full out integration tutorial:

That comes along with this colab notebook:

As well as this intro report here:


1 Like

Hi there, I wanted to follow up on this request. Please let us know if we can be of further assistance or if your issue has been resolved.

Hi, since we have not heard back from you, we are going to close this request. If you would like to reopen the conversation, please let us know! Unfortunately, at the moment, we do not receive notifications if a thread reopens on Discourse. So, please feel free to create a new ticket regarding your concern if you’d like to continue the conversation.

1 Like