Pattern in semantic mask logging above size (100, 100)

[rdorfman]

When attempting to log masks above size 100 x 100, the masks display a strange pattern. What is the cause of this and is there a way to ensure the pattern does not appear?

Here is a toy example comparing two random images of size (100,100) and (200,200), each with two masks of all 0s and all 1s. This is logged as:

These images were generated by the following:

import numpy as np
import wandb

wandb.init()
sizes = [100,200]

for size in sizes:
    name = str(size)
    image = np.random.randint(low=0, high=256, size=(size, size, 3), dtype=np.uint8)
    predicted_mask = np.ones((size, size), dtype=np.uint8)
    ground_truth_mask = np.zeros((size, size), dtype=np.uint8)
    class_labels = {
        0: "class 0",
        1: "class 1",
    }

    masked_image = wandb.Image(image, masks={
        "predictions": {
            "mask_data": predicted_mask,
            "class_labels": class_labels
        },
        "ground_truth": {
            "mask_data": ground_truth_mask,
            "class_labels": class_labels
        }
    })

    wandb.log({name : masked_image})

[mohammadbakir]

Hi @rdorfman, happy to help with your question. The output of logging the images is expected behavior due to sampling. Our backend will sample in a non-deterministically sparse approach. Please let us know if you have any additional questions.

[rdorfman]

Hi @mohammadbakir, thanks very much for your response. Could you please clarify

1. Is there any way to prevent/correct the sampling so that the masks are logged correctly

2. Why this sampling is occurring

[mohammadbakir]

Hi @rdorfman , apologies my comment caused confusion, I was addressing the incorrect thread.

To clarify, segmentation masks and bounding boxes are not randomly sampled by W&B when logged to the workspace. The predictions are user generated and the input of the user to wandb is directly being logged. From the code example you provided, you are generating prediction masks and class labels for each of the pixels in the image. Your comment on " masks display a strange pattern", are you referring to the rendering of bounding boxes being represented in the workspace? Would appreciate any additional clarifying remarks about your issue. Thanks

[rdorfman]

Hi @mohammadbakir,

No worries, thanks for you help. My comment on “masks display a strange pattern” does refer to the rendering of the masks in the workspace. To try and clarify the issue, if you look at the center two masks in the screenshot (the middle column), these are both generated by

predicted_mask = np.ones((size, size), dtype=np.uint8)

where size = 200 for the top image and size = 100 for the bottom image. However, you can see for the size = 200 mask (the top one), this is not displayed as a solid mask (as it should be since it is an array of only 1s), whereas the size = 100 mask below it is rendered correctly.

[mohammadbakir]

Hi @rdorfman , I ran some tests on my end with various array size and was unable to reproduce that rendering glitch you ran into. Please see this project where I have image/masks side by side displaying solid masks. I will keep an eye out for other mentions of this behavior. At this time my recommendation is to delete the panels, clear your workspace ( From bottom left of screen, My Workspace > Clear Workspace), re-run your code and see if this behavior persists. If it does, please send me a link to your workspace.

[rdorfman]

Hi @mohammadbakir , thanks for taking the time to test this. When I view the project you shared, all examples that do not show the rendering issue appear to have dimensions smaller than (100,100). However, the rendering issue shows up for me for the (200,200) masks. Could you please let me know if you can see this issue in your project for the (200,200) size mask?

[rdorfman]

Here is a screenshot from the project you shared: