Configuration sweep key not appearing during sweep run nor changing values

[carlosruiz]

I’m using stable baselines 3 (SB3) and in order to use one activation function or another, you must pass it to the SB3 algo as kwargs, that is:

model = A2C(
            policy=config['policy'],
            env=env,
            learning_rate=config['learning_rate'],
            gae_lambda=config['gae_lambda'],
            ent_coef=config['ent_coef'],
            tensorboard_log=LOGS_DIR,
            device=config['device'],
            verbose=config['verbose'],

            policy_kwargs=dict(
                net_arch=dict(
                    pi=config['policy_nn'],
                    vf=config['value_nn']),
                activation_fn=config['activation_fn'],
                optimizer_class=config['optimizer_class'])
        )

where:

config['activation_fn']  = th.nn.ReLU

or

config['activation_fn']  = th.nn.Tanh

If I configure the sweep dictionary key for the ‘activation_fn’ as:

'optimizer_fn':{
    'values': [th.nn.ReLU, th.nn.Tanh]
}

Afterwards when running sweeps, those values are not seen by wandb, neither in the plots appears the optimizer_fn been used nor changes to the other option.

Is it because the sweep config dictionary only accepts string for that kind of values?

[bill-morrisson]

Hey Carlos,

Thank you for your contacting us!

Is the value for th.nn.ReLU or th.nn.Tanh in this case a tuple or a single value or a list of values?

If it is more than one value for each of them I suggest trying this

‘optimizer_fn’: {

         'values': {'relu_values': [th.nn.ReLU], 'tanh_value':[tn.nn.Tanh]

Let me know if it still doesn’t show anything.

[carlosruiz]

Thanks for the reply bill-morrison, I finally sort it out doing the following:

# Activation function
        if config['activation_fn'] == 'Relu':
            activation_fn = th.nn.ReLU
        elif config['activation_fn'] == 'Tanh':
            activation_fn = tn.nn.Tanh

That way I can just pass the string to the sweep config, does the job.