Sweeps & gin files? (and parameters that are lists? )

W&B Help
1

Hi, I’m a longtime WandB user but new to sweeps.

We want to do some model architecture searches and the models are defined in .gin files. Here’s a snippet:

RATIOS = [4, 4, 4, 2, 2]
DILATIONS = [1, 3, 9]
KERNEL_SIZE = 3

models.OurAwesomeModel:
    ratios = %RATIOS
    dilations = %DILATIONS
    pre_conv = @encoder/pre_conv/nn.Conv1d
    post_conv = @encoder/post_conv/nn.Conv1d

I want to vary parameters like the all-caps ones above – including the ones that are lists!

How do we do that? I’m reading the Documentation on Sweeps and I can’t find anything that address this.
Given that gin files are Python-like, is there some kind of wandb.sweeps.gin() that I can put in there ?
The sweeps docs I’m seeing make it seem like the only control options are command line args. By the time I’m parsing command line args, those architecture bins in the .gin file have already been instantiated. :person_shrugging:

Now, obviously I could write a little intermediary script that takes the parameters as command line args and outputs a header to a (new) .gin script and then calls my main code from there – I’m not above doing that! — but is there perhaps some other gin-friendly/gin-powered way that’s already been devised?

One more question:( I checked the Docs and the FAQ but I still couldn’t find the answer to this)
How do we get the sweeps controller to vary the elements in a list? I don’t have to make a separate variable for every list element, do I?

Thanks.

3

Actually gin does have a way to override. When you read the file, you use gin.parse_config_files_and_bindings(), e.g.

gin.parse_config_files_and_bindings([gin_in], \
      [f"RATIOS = {args.RATIOS}\nDILATIONS = {args.DILATIONS}\nKERNEL_SIZE = {args.KERNEL_SIZE}"])

…So that only leaves the question of how to specify lists!

4

Hi Scott,
Thanks for adding the update for the first question.
You can vary elements in a list by defining it like I’ve done for optimizer below:

program: train.py
method: grid
parameters:
  optimizer:
    values: ["adam", "sgd"]
5

@_scott , Thanks for your reply. Let me clarify: that’s not what I mean. In your example, “adam” and “sgd” are strings, not lists.

In my case, for example RATIOS = [4, 4, 4, 2, 2] refers to a single list variable. I want the Sweep to try all sorts of variations like

RATIOS = [4, 4, 2, 2, 2]
RATIOS = [2, 4, 4, 2, 2]
RATIOS = [3, 4, 3, 2, 3]
RATIOS = [3, 3, 3, 3] # yes, different lengths would be ok
…etc.

I want the system to try all different sorts of numbers for those list elements. How do we do that?

PS- One thing I know that doesn’t work: trying to specify lists or tuples as values, whether written as bare lists/tuples or encapsulated in strings. I tried a config file where is specified a few pre-made choices (not what I want but i thought I’d try anyway):

  DILATIONS:
    values:
      - "(1, 3, 9)"
      - "(1, 2, 4, 8)"
      - "(1, 4, 16)"
    distribution: categorical

and the parser mangled these beyond comprehension:

  DILATIONS:
    distribution: categorical
    values:
      - (1
      - 3
      - 9)
      - (1
      - 2
      - 4
      - 8)
      - (1
      - 4
      - 16)
6

Hi @drscotthawley thanks for writing in! Would it be possible to convert the tuple to a list? Would the following work for you?

  RATIOS:
    values:
      - [4, 4, 2, 2, 2]
      - [2, 4, 4, 2, 2]
      - [3, 3, 3, 3]

This should be unpacked to the following:

  RATIOS:
    values:
      - - 4
        - 4
        - 2
        - 2
        - 2
      - - 2
        - 4
        - 4
        - 2
        - 2
      - - 3
        - 3
        - 3
        - 3
7

@thanos-wandb No. List form was in fact the original way I was specifying things, which were getting mangled similarly. Just try it and see.

Only tried switching to tuples after finding a GitHub Issue comment from three years ago where WandB co-founder Chris Van Pelt said that tuples would be supported in the next release. so…guess that never happened.

8

Hi @drscotthawley thanks for the additional information. May I please ask what’s your current wandb sdk/client version? and what do you get when your run wandb sweep sweep.yaml?

sweep.yaml

program: train.py
project: my-test-sweep
method: grid
parameters:
  RATIOS:
    values:
      - [4, 4, 2, 2, 2]
      - [2, 4, 4, 2, 2]
      - [3, 3, 3, 3]
9

See this Zendesk Support Ticket with Luis Beruga and Ryan McConneville: https://weightsandbiases.zendesk.com/hc/en-us/requests
Opened on stability-wandb Slack.

Unfortunately I’m being pulled off this project so I won’t be able to keep testing for now. :-/ Hope the WandB QA team figures it out!

10

Thanks for the update @drscotthawley since you have already created another ticket on this and you won’t be available for now to check further, I will go ahead and close this ticket. If you would like to reopen the conversation, please let us know and we will be happy to keep investigating!

11

This topic was automatically closed 60 days after the last reply. New replies are no longer allowed.