Grouping with multiple fields for "Parallel coordinates" & "Run comparer" not working

W&B Help
,
1

Hi,

I have read the following post: Group by multiple variables in charts - W&B Help - W&B Community (wandb.ai)

But this solution does not work for the “Parallel coordinates” or “Run comparer” to find optimal hyperparameters where multiple seeds are used for each experiment.

For me, I setup it up like such:

image
image712×364 36.5 KB

But for the “Parallel coordinates” it averages the groups of “step_size” & “features”:

image
image1495×509 99.1 KB

For the “Run comparer” it does the same, it averages the “step_size” and “features”.

How do I get the same separate runs as in the plots shown in the first image for the “Parallel coordinates” & “Run comparer”?

Best regards,
Caspar

2

Hi @capsar-meijer thank you for reporting this issue. I have logged this bug for both plots, and I have added you to the internal ticket so that we can share any updates with you once it’s fixed. I am wondering if you could try to group only by one field (eg. only step_size) if you would see a different behavior? also, what about removing the features from the PC plot (to check if the issue is related to the values being a list)?

3

Hi @thanos-wandb,

Thank you for picking this up.

  • Only 1 field grouping (eg. only step_size):

    image
    image1356×762 73 KB

    Result: it does do something else, because as opposed in my original post, now step_size is first in the group fields list. If I add observation_days behind it, it stays the same as this picture.

  • Removing “features” from the group fields list & the PC does not help:

    image
    image1336×685 69.7 KB

    The plot is the same as if I would only group on step_size. Even without the features in either the plot or group list it does not work.

Best,
Caspar

4

Hi @capsar-meijer thanks a lot for the additional details. I have updated the internal bug report to include this information too. We will keep you posted here once this functionality works for grouped runs.

5

Hi @thanos-wandb, thank you for raising the bug with the team.

After continuing analysis, I figured that another place where the grouping is lacking is in displaying media, for me specifically its displaying matplotlib figures.

I am logging a matplotlib figure that displays how well a model is able to forecast a time-series. Now I am trying out multiple hyperparameters and would like to see the figures that are related (in same group) also close by in the Media window. Currently it is just displaying individual runs without any indication of which group they belong too, making it difficult to analyse them compared to the 100+ other images.

Best regards,
Caspar

6

Thank you for the additional request here, @capsar-meijer . That’s correct we don’t support grouping in logged matplotlib figures, but I’ve logged this feature reques now with the team.

8

Hello @thanos-wandb,
I wanted to follow-up on this since the issue of grouping and Parallel Coord plots has been long reported to WandB (even before the herein thread). As I worked for a company that was a big corporate customer back in the days, we already got told, as an alleged workaround, to “press the top-right ungroup button” which never changed anything.
image
Several months later (we can soon start counting in years), any news (or plan for a fix), please ? Thanks for letting us know.

9

Hi @aurelien-morgan thank you for following up on this issue. I apologize for any inconvenience this may have caused. This feature request is currently in the backlog, as the engineering teams have prioritized other goals. I will add a +1 to the request and try to raising its priority. Thank you for your understanding and patience.