Love the sweep feature, but have one major issue when I want to adjust my priors on Bayes sweeps while they are running. I will let my models run and maybe realize that my original prior distribution is bad/too big and can tell I need to narrow it down. However currently it is hard to visualize the distribution for hyperparameter sampling and have to sort of guess by the colors and density of lines. Ideally each vertical axis on the parallel coordinates plot will have some light distribution of the prior and current posterior. Then ideally I should be able to adjust the prior on the fly by potentially discretizing the distribution and just dragging the areas I don’t want to the left/right. This would be game changing as someone who spends most of their research time not working on the model but just finding the right hyperparameters, and I will notice that my original range or guess was bad. Then I kill the sweep, start a new one, and restart the process, which I have noticed has taken up a lot of my time.
Maybe something like this - the visualization aspect - already exists, but I am not able to find it in the custom chart setup. I think it would be amazing to have it be default into the hyperparam plots.
Hi @smerkoud, thank you for the feedback! So it sounds like there are two feature requests here.
- The ability to adjust the distribution of the parameters while the sweep is live
- Improved visibility on the parallel coordinates chart. Could you elaborate on this one a bit and let me know specifically what would help here? I’m not sure I’m totally following this statement:
Ideally each vertical axis on the parallel coordinates plot will have some light distribution of the prior and current posterior
Thank you,
Nate
Thanks for getting back to this. Yeah having #1 does require knowing what the distribution looks like, which what I want from #2. Here is sort of what I have in mind.
So currently the parallel coordinate plot does not explicitly show the current parameter distribution estimated by Bayes or the initial distribution selected. Ideally we should be able to see both with something like this. Blue being the current estimate (which I should be able to change live), and the red being the initial one set in the sweep configuration.
Apologies for the crude drawing.
Let me know if that makes sense.
Thanks,
David
Hi @smerkoud thank you for the great suggestion! I wanted to follow up on this feature request and let you know that it has been logged with our App team. I agree that this functionality would be very useful, as it doesn’t currently exist.
As a workaround, you can download the sweep configuration from the API and process it locally. You may then log the chart as matplotlib, plotly, bokeh, or as an image, though this won’t provide live updates. We will keep you posted on any progress.
