I’m using sweeps with bayesian optimisation, and i have a limited number of possible combinations between the hyperparameters I’m trying to optimise (16 possible combinations) .
In a test, I tried to run 16 runs with bayesian optimisation in the same sweep expecting to have all 16 possible hyperparameter combinations, but only 12 of the possible combinations were selected.
Is this behavior normal?
In an other test a same combination appeared 4 times.
Yes, this is quite possible. A bayesian search does not provide any guarantees over not repeating combinations, it tries to learn an optimal distribution of hyperparameters and given what I am assuming in your case is a discrete search space, this combination might have been optimal according to the bayesian search.
If you are looking to perform an exhaustive search over all hyperparameter combinations, I would suggest running a grid search.
Hi Félix, since we have not heard back from you we are going to close this request. If you would like to re-open the conversation, please let us know!
This topic was automatically closed 60 days after the last reply. New replies are no longer allowed.