Different history run lengths of a metric

Hi everyone,

I have been using W&B to log my outputs of experiments and in one case I have a set up with fixed code and the hyperparameters change and the same code is run on the same data for the same amount of steps. However, when I use my own function to retrieve the values logged during training, they are of different lengths and it’s causing quite a headache to plot these metrics over time because they don’t align naturally.

I use my own function to retrieve a metric from a run:

def get_wandb_history(identifier, key):
    run = api.run(identifier)
    run_history = run.history()
    key_history = run_history[key]
    key_history = np.array([x for x in key_history if float(x) > 0.0])
    return key_history

The penultimate line is because of the NaNs that store info at steps in between epochs.
I ran 6 different runs where only a single hyperparameter changed. After retrieving the results via the function above, the lengths I have for them are:


What’s going on here? Is there a better / more reliable way to check the stored outputs? In the visualisation tool online, the runs are aligned perfectly and show the effect I was hoping to find. However, when I want to use those values in my own plots, they are not temporally aligned. So, they are being stored correctly for visualisation on the web interface - just not when I want to retrieve them (?)

Hi @alxmrphi, thanks for reporting this! Could you please share with me a link to the project where these runs are stored and also what are the runs you are retrieving the results from? It would be great also if you could send me a code snippet to create these runs and try to reproduce the same behaviour you do and see what’s happening here.

Hi Luis,

Sure thing. Regarding sending the link and the code, I’d prefer to do that privately but I can’t find an email address for you. If you could let me know (or Twitter handle, social media info) then I can send via there if that’s what you prefer.

Hi @alxmrphi, thanks for your answer! Of course, could you send an email to support@wandb.com and so we can track it and have a look at the issue? It’d be great if you could notify me once you do that. Thanks!

Email has been sent :slight_smile: Thanks.

Hi @alxmrphi, thanks! I’ll have a look at this and get back to you by email.

This request was closed and merged into request #40595 “Reported issue in Forum”.

This topic was automatically closed 60 days after the last reply. New replies are no longer allowed.