Memory Error Cause Fail Sweep Runnings

Hi, I really liked ur amazing tool and just started to use it. I m using sweep config for hyperparameters. Sometimes my running is get broke due to some reason in the models parameters etc. and my gpu get stuck with full memory. Is there a way to kill the gpu’s current job to free the gpu ram? it would be amazing to apply it to my code with a given PID number and kill it.

Hey there, here is how you can kill the current job that’s running. Let me know if there are any questions.

Hi there, I wanted to follow up on this request. Please let us know if we can be of further assistance or if your issue has been resolved.