Hyperparameter sweep within some subdomain of possible values

W&B Help
1

Hi there! I’m looking to run a hyerparameter sweep across a wide range of resnet configurations. I would like to explore two relevant paramters - the number of resnet blocks, and the number of filters in each resnet block. I would like the option for both of these hyperparameters to be large, but not at the same time! As this model will just take too long to train, and so my sweep time will be dominated by just one or two modules. So in the 2D domain of n_resnet_blocks and n_conv_filters, I want to explore some subdomain of this. Is there some way this could be done within the sweep configuration? An obvious approach would be just to run consecutive sweeps, capping each dimension to some sensible value while exploring the other one, but it’d just be neater if the space could be explored within a single sweep. So interested to hear any ideas!

Thanks!

2

Hi Chris,

Thank you for contacting support! To be clear are you trying to have different types of search configurations in the same sweep?

3

Hello Chris could you please clarify for me?

4

Hi @bill-morrisson - apologies for the late reply, thanks for getting back to me. I guess I’m describing some additional criterion. Imagine we have two hyperparameters, a and b, which we both want to vary between [0, 1]. I am looking to search this space, but exclude runs where both a>0.5 and b>0.5. I guess I could add a break clause within the train() function to skip such values, but wondering if this could be done within the sweep configuration itself.