Cumulative max (highwater mark) for distributed training and sweep

W&B Help
, ,
1

Hello everybody!
have an algorithm that I run 10 times, and return the best run by a cumulative maximum - So for each run, I log the highest(cumulative) validation score of the entire run.

I ran 7 of these, and grouped them together aggregating with maximum. However, since each experiment validates at different timestep, the resulting graph is not a cumulative maximum of the entire 7 runs. That happens because at each validation point, not all runs are present. What I got, with what I want to achieve marked in red:

enter image description here

  1. Is this achievable?
  2. How can I set a sweep that uses the cumulative validation of the entire experiment (the red line, not a single trial)?

Thanks in advance!

2

Hi Eyal,

This is possible! You can do this by clicking on the edit icon on the top right of the plot (it looks like a pencil) From there, go to the grouping tab and turn the runs toggle on. Afterwards, you can aggregate by max and see the graph that you wanted

3

Hi Eyal,

We wanted to follow up with you regarding your support request as we have not heard back from you. Please let us know if we can be of further assistance or if your issue has been resolved.

4

Hi Eyal, since we have not heard back from you we are going to close this request. If you would like to re-open the conversation, please let us know!