Wandb weave - is there a way of enforcing weave parallelism?

In the beginning all my evals ran fully parallelized as expected, then I set weave_parallelism as an environment variable, and then removed it again but now all my evals are only running sequentially, even if i set WEAVE_PARALLELISM=10 or a higher number. Is there any way of enforcing parallelism in weave?