Adding Image() to a Report programatically

Hello,
I am experimenting with programmatic report generation using Python.
I understand that reports are a beta feature.
I would like to add an Image object to a report. The image was generated in MatPlotLib and rendered using PIL. I’ve read the documentation but there is little if any mention of how to handle Image objects, other than a brief mention that they are supported.

I am getting the following error:

TypeError: url object must be of type (<class 'str'>,) (got <class 'PIL.PngImagePlugin.PngImageFile'>)

The generating code for the report looks like:

blocks  = []
blocks += [wr.MarkdownBlock(text="Markdown cell with *italics* and **bold** and $e=mc^2$")]
for result in results:
    if result:
        blocks += wr.PanelGrid(panels=[wr.Image(result['image'], caption=f"Subject={result['subject']}")])
report.blocks = blocks
report.save()

Hi @kevinashaw thanks for writing in, and glad to hear you are trying the Reports API. Currently, the Image object can’t be inside the PanelGrid. Could you please try the following code snippet and let me know if that worked for you?

blocks  = []
blocks += [wr.MarkdownBlock(text="Markdown cell with *italics* and **bold** and $e=mc^2$")]
for result in results:
    if result:
        blocks += [wr.Image(result['image'], caption=f"Subject={result['subject']}")]
report.blocks = blocks
report.save()

Please also have a look at this Colab which you may find useful to find the currently supported features. The Python SDK view of our reference docs here may help as well.

Hi thanos-wandb,
That makes perfect sense. I should have seen that.
I will implement it!
Thanks!
-K

@thanos-wandb
I’m getting the following error:

AttributeError: read

Here is the basic code:

import wandb.apis.reports as wr
from PIL import Image
buf = io.BytesIO()
plt.gcf().set_size_inches(PLOT_PICTURE_WIDTH, PLOT_PICTURE_HEIGHT)
plt.savefig(buf, format='png', dpi=PLOT_DPI)
blocks += [wr.Image(Image.open(buf))]

Hi @kevinashaw the wr.Image doesn’t support the PIL.PngImagePlugin.PngImageFile type that you pass as input. If you change this argument to a PNG image it will work.

For instance you may try the following:
blocks += [wr.Image(url="https://assets.website-files.com/5ac6b7f2924c656f2b13a88c/5ecb633d54bd2fe1e0378852_logo%20white.png", caption="W&B logo")]

I hope this helps, please let me know if you have any further questions.

Hi @thanos-wandb ,
Thank you for the help.
I will note that the above command parallels how I write to wandb.log(). I avoid file-system writes and instead write to an in-memory file buffer and then hand PIL.Image.open() to the log method, since this saves time and avoids file-system hassles. Its much faster and I have a lot of images that I need to upload.
And It would be very helpful in the different parts of WandB worked in similar ways.
It would also be very helpful if there was documentation for this particular option. The ability to add images is mentioned, but there is no example or explanation.
And yes, I understand that this is a beta feature. Its just a really important feature for us!
Thank you for your help!
Kevin

Hi @kevinashaw thanks for the additional context! That’s indeed more efficient way and there is a workaround that could help you do this. It required though to first log these images in a dummy run. Please see below a code snippet:

wandb.init(entity=ENTITY, project=PROJECT)
buf = io.BytesIO()
plt.gcf().set_size_inches(PLOT_PICTURE_WIDTH, PLOT_PICTURE_HEIGHT)
plt.savefig(buf, format='png', dpi=PLOT_DPI)
example = wandb.Image(Image.open(buf))
wandb.log({"example": example})
wandb.finish()

report = wb.Report(
    project=PROJECT,
    title="Report Adding Images from runs media files",
    blocks=[wb.PanelGrid(panels=[wb.MediaBrowser(media_keys="example")])]
)
report.save()

Please let me know if there are any issues or further questions about this.

Hi @kevinashaw just checking in to see if the above code did the trick for you? thanks!

Hi @kevinashaw since we haven’t heard back from you, I will close this ticket for now. If you still experience issues with this, please let us know to re-open the ticket and keep investigating.

Hi Thanos,
Thank you for the help.
I implemented the code as you described:

  1. First log the image as a normal PNG bitmap.
  2. Connect the already existing image to the report using the wr.MediaBrowser() operation.

Unfortunately, this does not actually display the image, but instead shows a browser panel.
Since this is a Report that needs to be exportable to a PDF document, the browser is not very useful.
Instead I need to be able to display the image itself, as one might in a typical PDF report.
See example below of the actual “images” in the report. Do you have any recommendations?

@thanos-wandb Will you be able to reply to this? Thank you.

Hi @kevinashaw apologies for the late response here. Since you logged the image, these images should be available in your Run/Files/media/image_key/ folder from the UI? If you click in one image there that would give a URL which you could use for the previous approach:
wr.Image(url="https://wandb.ai/ENTITY/PROJECT/runs/RUN-ID/files/media/IMAGE_KEY/NAME.png", caption="Example")
Can you please try this for one image and see if it works for you and displayed in report as needed? I could then provide you with a code snippet to get all the file names under the media folder.