Run.history() returns different values on almost each call

W&B Support
,
#1

I recently started using the wandb.Api() in order not to manually download all the Charts in .csv format.

The problem is that I cannot get consistent results, most of the times that I call the API in a jupyter-notebook I get different results.

I have made public one of my dashboards to tackle this issue. Here is a screenshot with a reproducible example:

2022-05-16-165542_647x517_scrot
2022-05-16-165542_647x517_scrot647Ɨ517 50.3 KB

In order to obtain the csv_val_f1 variable one just needs to download the Val F1 chart. Two things can be seen here:

  1. Multiple runs of the same code produce different results
  2. The maximum value obtained by the API differs from the maximum value obtained by manually downloading the .csv version of the Chart.

Any ideas on what Iā€™m missing?

#2

I am experiencing the same issue.
I wanted to generate matplotlib plots using the API, but run.history() shows different results at each call so that it is impossible to reproduce the wandb charts .

#3

Hi @jaeheelee and @carloshernandezp,
I believe you are seeing this because we sample the data points when you call run.history(). You can use run.scan_history() if you would like to have the entire history returned. Here is some more information on this.

Let me know if this solves the issue for you.

Thank you,
Nate

#4

Yes, this solved my problem. Thank you @nathank !

1 Like
#5

This indeed solved the issue. Thank you for answering!

1 Like