I am hoping to inspect some thing in the .wandb file before I upload them (it is faster to get some quick results this way than to upload and then redownload data), but I am not sure what they are encoded in. Do you know if there are tools in the wandb library I can use to open these?
Hi @evanv , we don’t really support tools to do this. Can you give me some more context around why this is a desired feature?
I have a lot of runs I need to do because of seeding issues and they take a while to upload to wandb (I have to run everything offline). I definitely want to have them in wandb for reproducibility, logging, and analysis, but there is some analysis I can do very quickly if I am able to extract it from the logged data. Otherwise I need to wait several hours to upload to wandb and several more hours to download everything to get the data I need which is a bit counter-intuitive. Thanks so much for your help!
It sounds like local, the self hosted version of wandb might be a better option for your use case.
Yeah, that is the general sense I have gotten. Unfortunately it doesn’t seem to be an option for me because the HPC I am using does not permit docker (it seems most HPCs don’t because of security vulnerabilities). Do you know if I can setup wandb locally on my laptop and still run on seperate machines that don’t have access to that wandb local setup?
Hi @evanv , I guess you could run your experiments on your HPC, scp all the wandb data over to your laptop where you’ve set up a local instance. Then you could view the results without having to upload them to the internet.
However, I’m not sure it wouldn’t be simpler to write some kind of function that logs the high level data you want to inspect before uploading to a csv or pdf plot.
1 Like
Hey @aidanjd ,
That actually is a great idea if it works and would solve a lot of issues! Is there anything inherent to the .wandb files that specifies that they are read by your servers as opposed to a wandb local instance? The thing I am trying to wrap my brain around is how I actually view the files from my local device using the wandb local instance.
The only thing you’d have to worry about is making sure the are accessible from the docker container that is running the local instance.
The only thing you’d have to worry about is making sure the are accessible from the docker container that is running the local instance.
[Discourse post]
This topic was automatically closed 60 days after the last reply. New replies are no longer allowed.