Pytorch sweep help

Hi . I want to do a study in pytorch with wandb sweep. I want to try activation function optimization algorithm and lr value with various combinations. but I have no idea how to do this. I did it with keras but not in pytorch. can you help me

Hi @yasarniyaz ,

We have an example Colab Notebook detailing how to run a sweep in PyTorch. You can access it here.


thank you my friend. I looked at this resource, but it doesn’t show how to change my activation functions here, and frankly, this resource seems very confusing to me. do you have any other suggestions?

I would say you would have to do some manipulation in your code to get this going. Something like:

if config.activation == 'relu':
  self.activation = nn.ReLU
elif config.activation == 'tanh':
  self.activation = nn.tanh

in your __init__ function. You can then use the self.activation object to set up your activations in your model based on the sweep config.


This topic was automatically closed 60 days after the last reply. New replies are no longer allowed.