In my project, one of the network is larger than the others and has to use a smaller batch size. To keep the comparison fair, I used a 2x smaller batch size but accumulate gradients for 2 batches, which is theoretically the same amount of gradient updates as long as I keep the number of epochs 2x larger than the normal models. However, in the plots that are being tracked, the bigger model will look like it learns twice as slow as the normal model. This is expected, but to simulate the effect training with the same batch size, I need to shrink the x axis of the big model’s learning curve.
How would I do that? There is an expression column that I can use when editing the panels, but this seems to only work for all the curves in the plot, not selectively.
Maybe somebody else will have a better solution but I believe this is only possible if you know ahead of time that you want to change the x-axis.
You need to use a method called define_metric in which you can define a custom step.
Here’s an example of setting a custom x-axis metric, instead of using the default step:
import wandb
wandb.init()
# define our custom x axis metric
wandb.define_metric("custom_step")
# define which metrics will be plotted against it
wandb.define_metric("validation_loss", step_metric="custom_step")
for i in range(10):
log_dict = {
"train_loss": 1/(i+1),
"custom_step": i**2,
"validation_loss": 1/(i+1)
}
wandb.log(log_dict)