How to customize Advanced Legend to display a string

[roland_roms]

Hello W&B community!
For a custom project within our company, we attempt to compare various models across a list of category.
We therefore want to visualize, for each version of our model:

• for each category (a string)
• a score (float in 0,1).

Here is how we log scores in W&B (simplified for easy understanding):

    scores = [0.3, 0.5, 0.6, ...]
    categories = ['dog', 'cat', 'fish', ...]
    for k, (val, name) in enumerate(zip(scores, categories)):
        wandb.log({
                    f"score": val,
                    f"category": name,
                    f"id": k
                })

Here is the graph associated with this code:

Now as you can see in the picture: For each version (td_ver), we display the “score” (y) for each “id” (x). When we hover over the score itself, we do not want to show the “id”, but the “name”
(In the image, instead of displaying “25” highlighted in red, we want to display “fish”.

I assume this change needs to happen in the “Advanced Legend” section, but I could not find the correct spelling for that formula.

original:
 [[ ${x}: ${y} (${original})]] ${run:displayName}

new:
???

Thanks a lot for your help!

[mohammadbakir]

Hi @roland_roms , happy to help. To update your legend to read “fish” instead of the “id” (x), use the following

[[ fish: ${y} ]] ${run:displayName}

Please let me know if you have any questions.

[roland_roms]

Hello @mohammadbakir , thank you for your answer!

Sorry if my question wasn’t asked properly: I do not want the string “fish” to be displayed, but the name of the category.
As you can see in the code I shared previously, each score is associated with a category: (scores[1] is for the categories[1], etc).
With your solution, it would replace all occurrences of the ID ( ${x} ) by fish.
What I need is to have the first score with the first category (cat), the second score with the second category (dog), and so on…

[mohammadbakir]

Hi @roland_roms , appreciate the clarification. If you were to include your class categories in the run configuration, then you can reference that in your legend, see here. This would look like

[[ ${config:category}: ${y} ]] ${run:displayName}

[roland_roms]

@mohammadbakir

you were to include your class categories in the run configuration, then you can reference that in

Thank you for your answer, but it does not work.
Following your advice, I added categories to the config, and tried to call it in various ways in Advanced legend:

import wandb

scores = [0.3, 0.5, 0.6]
categories = ['dog', 'cat', 'fish']

config = {"categories": categories,
            "id_cat": {0: "dog", 1:"cat", 2:"fish"}}

wandb.init(project="dummy_test", name="dummy_demo", config=config)
for k, (val, name) in enumerate(zip(scores, categories)):
    wandb.log({
                "score": val,
                "category": name,
                "categories": name,
                "id_cat": name,
                "id": k
            })
wandb.finish()

As you can see, I tried to add it in various ways (categories or id_cat) and to call them in various ways inside the advanced legend. It never worked properly.

Regarding the documentation you shared here, two points are to be highlighted:

You can set value inside[[ ]] to display point specific values in the crosshair when hovering over a chart
And

Supported values inside [] are as follows: …
“config” is not part of these supported values. summary is also not supported. Therefore, I don’t think it is possible to display it this way.

[roland_roms]

I have found a temporary solution, that gives a similar result (but not exactly the same).

1. I run a first time, logging only categories:

import wandb

scores = [0.3, 0.5, 0.6]
categories = ['dog', 'cat', 'fish']

wandb.init(project="dummy_test", name="categories")
for k, (val, name) in enumerate(zip(scores, categories)):
    wandb.log({
                # "score": val,
                "category": name
            })
wandb.finish()

2. I can then run wandb logs without the categories, but with the scores (I run a first time with the version_1 scores, then a second time with version_2 scores, etc):

import wandb

scores = [0.3, 0.5, 0.6]
categories = ['dog', 'cat', 'fish']

wandb.init(project="dummy_test", name="version_1")
for k, (val, name) in enumerate(zip(scores, categories)):
    wandb.log({
                "score": val,
                # "category": name
            })
wandb.finish()

3. In wandb display pannel, I create a new line plot with the following Data as Y:

• “category” (Added manually via “regular expression”)
• score

4. In Advanced Legend, I set up the formula as:

[[ ${x}: ${y} ]] ${run:displayName}

It gives me the following results:
(top: first point appearing as “dog”. bottom: second point appearing as “cat”)

[mohammadbakir]

Hi @roland_roms , please see this project example that produces the following chart. I used

[[ ${x}: ${y} ]] ${run:displayName}:${config:animal} to customize the legend and include animal types. Is this what you intend to accomplish?

import wandb
import random

categories = ["cat","dog","frog"]

for animal_type in categories:
    config = {"animal": animal_type}

    run1 = wandb.init(
        project="Legend-Test",
        config=config,
    )

    epochs = 10
    offset = random.random() / 5
    for epoch in range(epochs):
        acc = 1 - 2 ** -epoch - random.random()  - offset 
        run1.log({"acc": acc})

    run1.finish()

[roland_roms]

Hi @mohammadbakir . Thank you so much for your time and answer!
Your solution would be great in a different scenario, but it’s not what I aim for.

• In your example, there is one line for each category (you have one run for “cat”, one for “dog”, etc). With multiple values for each one of them.
• What I have is one value per category. But multiple types of runs.

Let’s say that I want to compare two kinds of models (Yolov4 or Yolov5) for finding animals in images.
I train my model, then test yolov4 on 100 images of dogs, 100 images of cats, etc. Same for Yolov5.
I then get an average score for yolov4 on cats. then a score for yolov4 on dogs, etc. Same for yolov5.

Now I want to compare both models: first line are the scores for yolov4 (score for cats, then for dogs, then for frog). Second line are the same scores, but for yolov5.

Now each point of the line is for a category (cats, dogs, frogs). With the current system, I can see that “for the third point, yolov4 is better than yolov5”. But I cannot say what the third point is (cats, dogs, frogs, or something else).

[mohammadbakir]

Hi @roland_roms , thank you for the additional info and clarification. This is not doable natively via wandb and your proposed work around solution is the closest currently. As our charts use Vega, you may be able to accomplish this using custom charts. Please do let me know if you have any other questions.