Sweep calls program with command line parameter in quotes

So… I found out myself during drafting this, but keep it here for reference/documentation. It’s a little thing, maybe of help to someone.

My train.py reads arguments from the command line and I want to use wandb sweeps.

One of the arguments is --config_file overwrite.args. I can pass this to train.py and process it with ArgumentParser() and HfArgumentParser() (from Hugging Face). If I use it as part of a sweep config, like so:

  - python3
  - ${program}
  - --config_file overwrite.args
  - ${args}
program: train.py

train.py is called as train.py "--config_file overwrite.args" (with quotes). This is represented differently in sys.argv and function calls like this treat it differently as well:

config_file_parser = argparse.ArgumentParser()
        "config_file", type=str, action="append"
    known_args, remaining_args = config_file_parser.parse_known_args()

Without quotes, it’s a known_args, with quotes it’s a remaining_args. Additionally, at some point, the HfArgumentParser()throws an error ValueError: Unknown configuration arguments with the quoted argument.

Therefore, my question/request: (how) can I get rid of the added quotes?

And the simple answer is: change the sweep config to put the flag/argument on two lines, like so:

  - python3
  - ${program}
  - --config_file
  - overwrite.args
  - ${args}
program: train.py

Hi Stephan!

This is very curious. Thank you for sharing this. To confirm, instead of having this as a single arg - --config_file overwrite.args
You split it into two and it starts working for you, right? - --config_file - overwrite.args


Hi Stephan,

We wanted to follow up with you regarding your support request as we have not heard back from you. Please let us know if we can be of further assistance or if your issue has been resolved.

Weights & Biases

Hi Stephan, since we have not heard back from you we are going to close this request. If you would like to re-open the conversation, please let us know!

Hi @artsiom,

I was not notified about your responses and only saw it by chance in the community summary e-mail. And yes, I split the arg into two, exactly the way my sweep config snippet shows. Btw, there IS an example in the docs showing that here.


This topic was automatically closed 60 days after the last reply. New replies are no longer allowed.