Hi there! I encountered the same issue. For context, I was training with various GPUs and various models (24).
What I deduce from the amount of time I spent resolving this issue is that using a different API key (from a different wandb account), I was able to run 4 runs simultaneously. However, on my 5th run, it did not work. However, after changing the wandb accounts, it worked. Or maybe because I used a different GPU (a 4080super rather than a 4090). However, I still believe that it might be caused by simultaneous runs under one account, even if Wandb does not impose a hard limit on the number of runs that can be run concurrently with one API key.
I hope this helps people who have experienced the same issue. My solution for sure isn’t the best, but it worked for me, but sadly wasted a lot of time and money on my rented GPUs lol