The following code until recently was working. right now it only send back “None” for every logged value.
import wandb
api = wandb.Api(timeout=109)
entity, project = "barthelemymp", "Generalization_below20out" # set to your entity and project
gen = api.run("barthelemymp/Generalization_below20out/6hz5riae")
results300 = {}
for row in gen.scan_history():
if "epochT" in row.keys():
if row["epochT"] == 300:
for k in row.keys():
if "_e" in k:
print(k)
results300[k] = 1- row[k]
Mainly here I save every logged value which name end in “_e” at epoch 300.
Eventhough the code is probably not the most efficient until recently it was loading the correct values (for the same run). Something changed apparently
I really need to be able to have access to this data. What happened ?
Do you think there is a way to get them back ? Sorry to bother but I really need to access these results before a deadline. If you think it will be hard, I ll try to rerun all my experiments with tensorboard asap.
Apologies for delay in response. What did the data look like before you started seeing all of the None’s. Once I go into wandb UI I do see that you mostly have None’s logged into the app.
Hi @artsiom , they are not really None. That is the weird thing. Few weeks ago I could retrieve the data from this exact run. Second the data appears in the workspace / charts I can see the values. So it is stored somewhere in wandb. For example Weights & Biases
you look for “TILTSLLVL_e” chart, it does appears as a function of training steps.
When I go to wandb and hover over “TILTSLLVL_e”, I see that the graph jumps around quite q bit while hovering over the graph, meaning that it wasn’t recorded for every step, and only those steps where there is a value recorded it “jumps the cursor to while hovering”, those are the recorded steps.
Pulling up runs.history using the weave tables, you could also see that most of the recorded values are none.
Hi Barthelemy, since we have not heard back from you we are going to close this request. If you would like to re-open the conversation, please let us know! If you are interested in continuing this conversation, please create a new thread and link this thread there.
Hi @artsiom Sorry for my late respond I had not seen you answered. This may come from my bad use of wandb as I log the scores as they are computed. So at every step I log different values, but also log an epoch variable.
So for every value of the epochT variable all the variables are logged. I should be able to get them. (again the code I showed before was working a few weeks ago)
Best
