I have been using ‘wandb sync wandb/*’ command to sync run and it was working well, But now I see this error when run wandb sync. Nothing in the env has changed.
Exception in thread Thread-1:
Traceback (most recent call last):
File “/nlsasfs/home/ai4bharat/ananyasb/anaconda3/envs/labse_finetuning2/lib/python3.9/threading.py”, line 980, in _bootstrap_inner
self.run()
File “/nlsasfs/home/ai4bharat/ananyasb/anaconda3/envs/labse_finetuning2/lib/python3.9/site-packages/wandb/sync/sync.py”, line 269, in run
ds.open_for_scan(sync_item)
File “/nlsasfs/home/ai4bharat/ananyasb/anaconda3/envs/labse_finetuning2/lib/python3.9/site-packages/wandb/sdk/internal/datastore.py”, line 104, in open_for_scan
self._read_header()
File “/nlsasfs/home/ai4bharat/ananyasb/anaconda3/envs/labse_finetuning2/lib/python3.9/site-packages/wandb/sdk/internal/datastore.py”, line 201, in _read_header
raise Exception(“Invalid header”)
Exception: Invalid header
Hey @oneraghavan, I have a few questions/troubleshooting steps to help me dig into this further:
- Sometimes this error can occur when you are using a different SDK version to sync the files than the SDK version that created the run files in the first place. What wandb SDK version are you running?
- Could you try clearing the local wandb cache and trying to sync again? You can do this with the following command:
rm -rf ~/.cache/wandb/
- Try reinitializing wandb and syncing again
Thank you for providing this info!
Hi Raghavan,
We wanted to follow up with you regarding your support request as we have not heard back from you. Please let us know if we can be of further assistance after trying any of the above steps.
Hi @oneraghavan , since we have not heard back from you we are going to close this request. If you would like to re-open the conversation, please let us know!
This topic was automatically closed 60 days after the last reply. New replies are no longer allowed.