I hoped it might be a feature already or someone knows the answer, but I could not see a way to automatically only display the top 1 after grouping runs.
The workflow is as follows:
A sweep was conducted where there were multiple parameters (categorical), which I used to find a good combination of filtering parameters.
I can sort by different metrics I observe, but how do I display such that only top_k (k=1 or 2) is displayed?
I can write code to flag those runs with tags or do this manually in the interface. But how to do so in wandb interface without adding tags?
Hi @aleksandar-cvejic, thank you for writing in! Unfortunately there is no direct way of doing this. What you could potentially do it sort the runs by the necessary metrics, unvisualize all of the runs, and then visualize the top_k runs you need manually by selecting the runs that are on top since you have sorted them by the necessary metric.
Thanks for your response, @artsiom.
I was able to make it work partially, but it was painful to do for many metrics/runs that I am logging.
I am just leaving a potential path for others.
For each metric of interest, you would use something like this:
wandb.define_metric(name=“something”, summary":“min,max”)
Note, if adding more summary items, no spaces and it can not be a summary metric!
Then the table aggregation will work, but we can directly aggregate for the Max, and display idfor the graph.
Downside:
This becomes harder with more runs, namely because of the following:
The table still displays all the runs, not the aggregate max
Because of the above, it becomes tedious to find the correct run between the table and plot
Leaving this is on the forum, even though it is hackier…
In reality I would want the top(k) to be displayed on the image and the table. (i.e., aggregate view of the table)