Can't associate sweeps with project

Hi,

I’m using wandb (great product!!!) and have been able to set up projects, do runs and am now working with sweeps (FANTASTIC!). However I can’t figure out how to associate my sweeps with a project.

I have:

import wandb

sweep_config = {
  "project" : "HDBSCAN_Clustering",
  "method" : "random",
  "parameters" : {
    "min_cluster_size" :{
      "values": [*range(20,500)]
    },
    "min_sample_pct" :{
      "values": [.25, .5, .75, 1.0]
    }
  }
}

Then when I:

sweep_id = wandb.sweep(sweep_config)

I get

Sweep URL: https://wandb.ai/teamberkeley/uncategorized/sweeps/jk9c1l8q

Note: teamberkeley/uncategorized/sweeps

They are of course uncategorized in the projects interface as well.

No luck with running wandb.init beforehand either thusly:

wandb.init(project=‘HDBSCAN_Clustering’)

Same result (despite the fact that at this point if I do ‘runs’ with wandb they are attached to the correct project after this init). Please let me know what I’m doing wrong!

I’d really like to fix this. I’m sure I’m doing something wrong but after looking through the doc, following examples and googling I can’t figure it out. Please help.

Hey @drob, could you try the following?

When calling wandb.sweep, try setting the project and entity attributes there, like:

wandb.sweep(sweep_config, enitity='drob', project='HDBSCAN_Clustering')

Let me know how it goes.

Thanks very much, really appreciated.

Better kinda…

So I added ‘entity’ and ‘project’ to wandb.sweep.

First time - timed out
Second time - seemed like it worked - I saw that the sweep url was with the correct projects… So YAY! Then it errored out.

Tried a couple more times - error, then restarted the VM (Colab+)… no luck :(. Here’s the error:

wandb: Currently logged in as: drob707 (teamberkeley). Use `wandb login --relogin` to force relogin

Tracking run with wandb version 0.12.19

Run data is saved locally in `/content/drive/MyDrive/Projects/StepWiseTuning/wandb/run-20220623_231502-cp4idak5`

Syncing run **[generous-meadow-26](https://wandb.ai/teamberkeley/HDBSCAN_Clustering/runs/cp4idak5)** to [Weights & Biases](https://wandb.ai/teamberkeley/HDBSCAN_Clustering) ([docs](https://wandb.me/run))

wandb: ERROR Error while calling W&B API: permission denied (<Response [403]>)

---------------------------------------------------------------------------

HTTPError Traceback (most recent call last)

[/usr/local/lib/python3.7/dist-packages/wandb/sdk/lib/retry.py](https://localhost:8080/#) in __call__(self, *args, **kwargs) 101 try: --> 102 result = self._call_fn(*args, **kwargs) 103 # Only print resolved attempts once every minute

---
18 frames
---

HTTPError: 403 Client Error: Forbidden for url: https://api.wandb.ai/graphql

During handling of the above exception, another exception occurred:

UsageError Traceback (most recent call last)

UsageError: permission denied

During handling of the above exception, another exception occurred:

CommError Traceback (most recent call last)

[/usr/local/lib/python3.7/dist-packages/wandb/sdk/internal/internal_api.py](https://localhost:8080/#) in no_retry_4xx(e) 1890 return True 1891 body = json.loads(e.response.content) -> 1892 raise UsageError(body["errors"][0]["message"]) 1893 1894 # TODO(dag): replace this with a query for protocol versioning

CommError: permission denied
But

Any chance we could figure this out?

Hmm, that’s odd. Can you share the full traceback for me to look into? From a quick glance though, I see you have an HTTP 403 (Forbidden) - are you sure you are logging to the correct entity?

Thanks,
Ramit

Please see a complete traceback below.

are you sure you are logging to the correct entity?

I go through the authentication without a problem.

I used the params that you suggested (adjusted for spelling) -

wandb.init(project='HDBSCAN_Clustering')
sweep_id = wandb.sweep(hdbscan_config, entity='drob', project='HDBSCAN_Clustering')

I have also tried:

wandb.init(project='HDBSCAN_Clustering')
sweep_id = wandb.sweep(hdbscan_config, entity='teamberkeley', project='HDBSCAN_Clustering')

Neither works. If I remove the ‘entity’ setting from sweep() I am definately logged in because I can execute sweeps and everything works. However, the issue is that everything winds up in ‘uncategorized’.

Here is a link to my colab notebook. As you can see it is very stripped down:

Perhaps you could point me to code that works that does what I’m trying to do?

Changes to your `wandb` environment variables will be ignored because your `wandb` session has already started. For more information on how to modify your settings with `wandb.init()` arguments, please refer to the W&B docs.
Problem at: <ipython-input-13-68824eaa576c> 1 <module>
Traceback (most recent call last):
  File "/usr/local/lib/python3.7/dist-packages/wandb/sdk/wandb_init.py", line 999, in init
    run = wi.init()
  File "/usr/local/lib/python3.7/dist-packages/wandb/sdk/wandb_init.py", line 651, in init
    backend.cleanup()
  File "/usr/local/lib/python3.7/dist-packages/wandb/sdk/backend/backend.py", line 246, in cleanup
    self.interface.join()
  File "/usr/local/lib/python3.7/dist-packages/wandb/sdk/interface/interface_shared.py", line 475, in join
    super().join()
  File "/usr/local/lib/python3.7/dist-packages/wandb/sdk/interface/interface.py", line 666, in join
    _ = self._communicate_shutdown()
  File "/usr/local/lib/python3.7/dist-packages/wandb/sdk/interface/interface_shared.py", line 472, in _communicate_shutdown
    _ = self._communicate(record)
  File "/usr/local/lib/python3.7/dist-packages/wandb/sdk/interface/interface_shared.py", line 226, in _communicate
    return self._communicate_async(rec, local=local).get(timeout=timeout)
  File "/usr/local/lib/python3.7/dist-packages/wandb/sdk/interface/interface_shared.py", line 231, in _communicate_async
    raise Exception("The wandb backend process has shutdown")
Exception: The wandb backend process has shutdown
wandb: ERROR Abnormal program exit
---------------------------------------------------------------------------
Exception                                 Traceback (most recent call last)
/usr/local/lib/python3.7/dist-packages/wandb/sdk/wandb_init.py in init(job_type, dir, config, project, entity, reinit, tags, group, name, notes, magic, config_exclude_keys, config_include_keys, anonymous, mode, allow_val_change, resume, force, tensorboard, sync_tensorboard, monitor_gym, save_code, id, settings)
    998         try:
--> 999             run = wi.init()
   1000             except_exit = wi.settings._except_exit

8 frames
/usr/local/lib/python3.7/dist-packages/wandb/sdk/wandb_init.py in init(self)
    650                     # we don't need to do console cleanup at this point
--> 651                     backend.cleanup()
    652                     self.teardown()

/usr/local/lib/python3.7/dist-packages/wandb/sdk/backend/backend.py in cleanup(self)
    245         if self.interface:
--> 246             self.interface.join()
    247         if self.wandb_process:

/usr/local/lib/python3.7/dist-packages/wandb/sdk/interface/interface_shared.py in join(self)
    474     def join(self) -> None:
--> 475         super().join()
    476 

/usr/local/lib/python3.7/dist-packages/wandb/sdk/interface/interface.py in join(self)
    665             return
--> 666         _ = self._communicate_shutdown()
    667 

/usr/local/lib/python3.7/dist-packages/wandb/sdk/interface/interface_shared.py in _communicate_shutdown(self)
    471         record = self._make_record(request=request)
--> 472         _ = self._communicate(record)
    473 

/usr/local/lib/python3.7/dist-packages/wandb/sdk/interface/interface_shared.py in _communicate(self, rec, timeout, local)
    225     ) -> Optional[pb.Result]:
--> 226         return self._communicate_async(rec, local=local).get(timeout=timeout)
    227 

/usr/local/lib/python3.7/dist-packages/wandb/sdk/interface/interface_shared.py in _communicate_async(self, rec, local)
    230         if self._process_check and self._process and not self._process.is_alive():
--> 231             raise Exception("The wandb backend process has shutdown")
    232         future = self._router.send_and_receive(rec, local=local)

Exception: The wandb backend process has shutdown

The above exception was the direct cause of the following exception:

Exception                                 Traceback (most recent call last)
<ipython-input-13-68824eaa576c> in <module>()
----> 1 wandb.init(project='HDBSCAN_Clustering')
      2 sweep_id = wandb.sweep(hdbscan_config, entity='drob', project='HDBSCAN_Clustering')

/usr/local/lib/python3.7/dist-packages/wandb/sdk/wandb_init.py in init(job_type, dir, config, project, entity, reinit, tags, group, name, notes, magic, config_exclude_keys, config_include_keys, anonymous, mode, allow_val_change, resume, force, tensorboard, sync_tensorboard, monitor_gym, save_code, id, settings)
   1035             if except_exit:
   1036                 os._exit(-1)
-> 1037             raise Exception("problem") from error_seen
   1038     return run

Exception: problem

Ahhh fixed. The entity is ‘drob707’, not ‘drob’. Thanks!