Using sweeps for custom data generator in keras

[zookcx]

Am using custom data generator as part of my image augmentation. Are you able to use sweeps to try different parameters for such augmentation?

for example:

idg = CustomDataGenerator(rescale = 1 / 255.,
                             horizontal_flip = False, 
                             vertical_flip = False,
                             v_kernel_size=config.v_kernel_size,
                              h_kernel_size=config.h_kernel_size,  
                             height_shift_range = config.height_shift_range, 
                             width_shift_range = config.width_shift_range, 
                             rotation_range = config.rotation_range, 
                             shear_range = config.shear_range,
                             zoom_range = config.zoom_range,)
    return idg

[thanos-wandb]

Hi @zookcx thanks for writing in! Is in your example the CustomDataGenerator a function that you could call from the main training function? Would something like the following work for you?

import wandb
def main():
    wandb.init(project='custom-data-sweep')
    data = CustomDataGenerator(rescale = 1 / 255.,
                             horizontal_flip = False, 
                             vertical_flip = False,
                             v_kernel_size=wandb.config.v_kernel_size,
                              h_kernel_size=wandb.config.h_kernel_size)
    wandb.log({'data': data})
    wandb.finish()


def CustomDataGenerator(rescale = 1 / 255.,
                             horizontal_flip = False, 
                             vertical_flip = False,
                             v_kernel_size=0,
                             h_kernel_size=0,
                          ):
  
    # add your own custom data generator logic
    idg = v_kernel_size + h_kernel_size

    return idg

sweep_config = {
    'method': 'grid',
    'project': 'sweep-configs',
    'parameters': {
        'v_kernel_size': {
            'values': [32, 64, 96, 128, 256]
        },
        'h_kernel_size': {
            'values': [32, 64, 96, 128, 256]
        }
    }
}

sweep_id = wandb.sweep(sweep_config)
wandb.agent(sweep_id, function=main)

[thanos-wandb]

Hi @zookcx just checking in here to see if the above snippet would work for you and if you had any further questions? thanks!

[thanos-wandb]

Hi @zookcx since we haven’t heard back from you, I will close this ticket for now. Please let us know though if you had any other questions, and we will be happy to keep investigating.