Hi,
I am using Bayes searching method to sweep through a group of hyper-parameters. Number of all permutations of these 2 hyper-parameters is 15. I set sweep count as 60, in order to get more cases. However when I check the sweep plots, I see this:
There are already 44 sweeps, but it seems many of cases are exactly identical. As a result, I actually only get 6 or 7 cases, even less than grid search. As you can see above, many cases are really superpositioned…
So it confuses me now that how should I understand Bayes searching method.
And there is corresponding code about sweep:
sweep_configuration = {
'method': 'bayes',
'name': f'{args.dataset_name} 3rd class: levels, hidden_size',
'metric': {'goal': 'minimize', 'name': 'valid/ber'},
'parameters':
{
'levels': {'values': [1,2,3,4,5]},
'hidden_size': {'values': [1.0,2.0,3.0]}
}
}
args.tune_count = 60
wandb.agent(sweep_id, function=Hyper_param_Tune, count=args.tune_count)
Thanks for any help!
