I’m trying to plot some results from Ray, but due to the presence of NaNs, my X axis is scaled very weirdly. Is there any way to ignore NaNs?
My issue is very similar to this one: NaN loss causes incorrect X axis time scale
This is the way it looks now:
Hi Olaf, thanks for writing! Could you please send me a link to the chart? When trying ti reproduce it, I obtain a normal chart:
Thanks for looking into my issue!
Since you can get a normal chart, it may be a Ray/WandB integration issue.
Here’s my chart: Weights & Biases
Hi Olaf! Thanks for your answer! You have sent me a link to a report, so I can’t see what data has the graph, could you please send me a link to the chart or try to load the chart in the report again?
Hi! Do I need to share the chart in some way as to share the data too? I have not used WandB much before.
Hi Olaf! You can send me a link to your Workspace and tell me what chart is causing the issue , would it be okay?
The workspace is Weights & Biases and the chart that is currently broken is
Thanks in advance!
Hi Olaf! Thanks for the link! When I find this chart, it doesn’t seem to have any NaN values and is shown right, have you been able to solve the issue?
Nope, when the chart is set to X axis being Wall time, I still get NaNs, and incorrect scaling. This happens only for some groups of my runs, though.
Hi Olaf! Thanks for clarifying, I’ve been able to find the chart that you mention. Could you please explain me what would be the right behaviour of the chart and what would you like to represent? From my understanding, it’s working properly. I you click in More actions → Export Panel → CSV Export, in the preview you can see that all the values are NaN. Also, as the Wall Time is a number and not a Series, wouldn’t it be useful to create a bar plot with Weave?
Ideally, the behaviour would be that the NaNs are ignored in the chart scaling, so instead of the previous behaviour (I can only post one image per post), I would see this, but with the Wall Time as X axis.
In terms of all values being NaNs - at least to me it looks like there is only a couple of NaNs per run, and then the runs continue fine. So that breaks the scaling, I think.
As for the bar plot - I’m not sure what I could do there. My aim is to plot the reward over run time.
Thanks for the explanation! I can see the issue here, I will create a request for this feature! Please let me know if I can be of further assistance.
I’m glad to hear this may be a feature in the future! Thanks for all your help!
Thanks you for suggesting this feature, it is really useful! Have a great day!