Summary metric from runs not visible in sweep (bug?)

[aabywan]

Greetings WandB, many thanks for your fantastic service!

I have been running several sweeps in a project and only realised that the summary metrics for the sweeping overview got several missing values despite the runs containing and having successfully logged these. Did I log my metrics wrong and need to redo all my experiments, or is it a bug, as described below?

Please see this first image, containing the run summary of toasty-sweep-31, which logged Test_AUC_std and Test_EER_std. Let’s focus on the ‘Test_AUC_std’ since that column is also visible in the following screenshot showing the sweep summary metrics; however, the same issue appears for several other columns also logged within each run.

In the sweep summary metrics, I am showing all columns available, including Test_auc_std, but now the AUC is lowercase, and the values are missing. Same for the “Test_EER_std” etc.

Since the columns are automatically inferred from runs, how could I ‘refresh’ the columns to synchronise with the runs? I do not recall logging ‘Test_auc’, but that may be a spillover from early runs. Is there are way to ‘recompile’ the project and these summary metrics?

The sweep is available here

[artsiom]

Hi Peter,

It is a bug and I am very sorry you are running into this. Unfortunately as of yet, wandb isn’t case-sensitive, and if in one of the older runs you have switched it to lowercase and then back to uppercase, that might have messed it up, and now it’s showing only the lowercase one although both of them are saved successfully as shown in the summary.

For now, the workaround would be rerunning your sweep and making sure you don’t have the same-named variables with different capitalizations that might mess it up.

I’ll report this bug instance to our eng team.

Apologies you are running into this again,

Warmly,
Artsiom