How can I create more complex logic for defining the parameter space of a sweep? For example, for each model family, I’d like to define a different set of parameters, or there a parameter that is a (variable) list of values.
I can write code that generates all possible hyperparameter combinations by myself (as a list or as generator).
Hey @shlomihod
We have this documentation on custom sweeps. It shows you how to define more fine-tuned control over what command the controller will be calling to the agent and not the functionality of the sweeps tuning directly. You can usually find a good way to combine sweeps with your training script to work out most search space workloads. To do fine tuning on the functionality of sweeps you could use the local controller but it is not intended for actual hyperparameter optimization workloads.
I’m not entirely sure what you mean by “each model family” but I’m assuming you mean some degree of conditional control over sweeps, which isn’t supported natively at the moment. Let me know if I’m misunderstanding though! You can most likely use the custom sweeps suggestion I mentioned to achieve something to this effect.
You can pass a list in YAML and that can be used with grid search strategy. However, it does not work with our other search strategies like Bayes (because that wouldn’t really make sense for lists). Do you have a specific use case you’d like to support if this isn’t what you’re looking for?
This topic was automatically closed 60 days after the last reply. New replies are no longer allowed.