TypeError when uploading pixel value bounding box

Hi,
I was just trying to log a 2D bounding box with pixel coordinates, but I keep on running into this error:
TypeError: Object of type int is not JSON serializable

The code I used:

box_data = []

class_labels = {
    0: "face"
}

for (x,y,w,h) in face_rects:
    frame = cv2.rectangle(frame,(x,y),(x+w,y+h),(255,0,0),2)

    midX = int(x+w/2)
    midY = int(y+h/2) 
    box = {
                "position": {
                    "middle": [midX, midY],
                    "width": w,
                    "height": h
                },
                "domain" : "pixel",
                "class_id" : 0
            }
    box_data.append(box)

predictions = {"predictions": {
        "box_data": box_data,
        "class_labels": class_labels
    }
    }

img = wandb.Image(frame, boxes=predictions)

It works when I’m using the relational notation instead of pixel values, but I’d rather keep the pixel values for simplicity in the code.

@johko-cel ,

Will be glad to look into this for you. I utilized your code example and was successful in using pixel values without issue. I’m suspecting this isn’t wandb related, but in order to troubleshoot, we will require the following please.

  • Wandb Verion you are using
  • Link to your workspace where you were attempting to log this image
  • Complete traceback when experiencing this error
  • If possible more extensive code sample in the form of a Colab so we can follow the steps you took in order to reproduce the error.

Regards,

Mohammad

Hi Mohammad,

thanks for getting back to me so quickly. I also suspect that it might not be directly wandb related, but also have no idea where else it might come from.

{
	"name": "TypeError",
	"message": "Object of type int is not JSON serializable",
	"stack": "\u001b[0;31m---------------------------------------------------------------------------\u001b[0m\n\u001b[0;31mTypeError\u001b[0m                                 Traceback (most recent call last)\n\u001b[1;32m/home/johannes/Projects/blog/haar/haar_cascade.ipynb Cell 5'\u001b[0m in \u001b[0;36m<cell line: 37>\u001b[0;34m()\u001b[0m\n\u001b[1;32m     <a href='vscode-notebook-cell:/home/johannes/Projects/blog/haar/haar_cascade.ipynb#ch0000005?line=28'>29</a>\u001b[0m     box_data\u001b[39m.\u001b[39mappend(box)\n\u001b[1;32m     <a href='vscode-notebook-cell:/home/johannes/Projects/blog/haar/haar_cascade.ipynb#ch0000005?line=30'>31</a>\u001b[0m predictions \u001b[39m=\u001b[39m {\u001b[39m\"\u001b[39m\u001b[39mpredictions\u001b[39m\u001b[39m\"\u001b[39m: {\n\u001b[1;32m     <a href='vscode-notebook-cell:/home/johannes/Projects/blog/haar/haar_cascade.ipynb#ch0000005?line=31'>32</a>\u001b[0m         \u001b[39m\"\u001b[39m\u001b[39mbox_data\u001b[39m\u001b[39m\"\u001b[39m: box_data,\n\u001b[1;32m     <a href='vscode-notebook-cell:/home/johannes/Projects/blog/haar/haar_cascade.ipynb#ch0000005?line=32'>33</a>\u001b[0m         \u001b[39m\"\u001b[39m\u001b[39mclass_labels\u001b[39m\u001b[39m\"\u001b[39m: class_labels\n\u001b[1;32m     <a href='vscode-notebook-cell:/home/johannes/Projects/blog/haar/haar_cascade.ipynb#ch0000005?line=33'>34</a>\u001b[0m     }\n\u001b[1;32m     <a href='vscode-notebook-cell:/home/johannes/Projects/blog/haar/haar_cascade.ipynb#ch0000005?line=34'>35</a>\u001b[0m     }\n\u001b[0;32m---> <a href='vscode-notebook-cell:/home/johannes/Projects/blog/haar/haar_cascade.ipynb#ch0000005?line=36'>37</a>\u001b[0m img \u001b[39m=\u001b[39m wandb\u001b[39m.\u001b[39;49mImage(frame, boxes\u001b[39m=\u001b[39;49mpredictions)\n\u001b[1;32m     <a href='vscode-notebook-cell:/home/johannes/Projects/blog/haar/haar_cascade.ipynb#ch0000005?line=38'>39</a>\u001b[0m wandb\u001b[39m.\u001b[39mlog({\u001b[39m\"\u001b[39m\u001b[39mobama_pc\u001b[39m\u001b[39m\"\u001b[39m: img})\n\nFile \u001b[0;32m~/.local/share/virtualenvs/haar-tXjf4l0j/lib/python3.9/site-packages/wandb/sdk/data_types/image.py:147\u001b[0m, in \u001b[0;36mImage.__init__\u001b[0;34m(self, data_or_path, mode, caption, grouping, classes, boxes, masks)\u001b[0m\n\u001b[1;32m    144\u001b[0m \u001b[39melse\u001b[39;00m:\n\u001b[1;32m    145\u001b[0m     \u001b[39mself\u001b[39m\u001b[39m.\u001b[39m_initialize_from_data(data_or_path, mode)\n\u001b[0;32m--> 147\u001b[0m \u001b[39mself\u001b[39;49m\u001b[39m.\u001b[39;49m_set_initialization_meta(grouping, caption, classes, boxes, masks)\n\nFile \u001b[0;32m~/.local/share/virtualenvs/haar-tXjf4l0j/lib/python3.9/site-packages/wandb/sdk/data_types/image.py:175\u001b[0m, in \u001b[0;36mImage._set_initialization_meta\u001b[0;34m(self, grouping, caption, classes, boxes, masks)\u001b[0m\n\u001b[1;32m    172\u001b[0m         boxes_final[key] \u001b[39m=\u001b[39m box_item\n\u001b[1;32m    173\u001b[0m     \u001b[39melif\u001b[39;00m \u001b[39misinstance\u001b[39m(box_item, \u001b[39mdict\u001b[39m):\n\u001b[1;32m    174\u001b[0m         \u001b[39m# TODO: Consider injecting top-level classes if user-provided is empty\u001b[39;00m\n\u001b[0;32m--> 175\u001b[0m         boxes_final[key] \u001b[39m=\u001b[39m BoundingBoxes2D(box_item, key)\n\u001b[1;32m    176\u001b[0m     total_classes\u001b[39m.\u001b[39mupdate(boxes_final[key]\u001b[39m.\u001b[39m_class_labels)\n\u001b[1;32m    177\u001b[0m \u001b[39mself\u001b[39m\u001b[39m.\u001b[39m_boxes \u001b[39m=\u001b[39m boxes_final\n\nFile \u001b[0;32m~/.local/share/virtualenvs/haar-tXjf4l0j/lib/python3.9/site-packages/wandb/sdk/data_types/helper_types/bounding_boxes_2d.py:214\u001b[0m, in \u001b[0;36mBoundingBoxes2D.__init__\u001b[0;34m(self, val, key)\u001b[0m\n\u001b[1;32m    182\u001b[0m \u001b[39mdef\u001b[39;00m \u001b[39m__init__\u001b[39m(\u001b[39mself\u001b[39m, val: \u001b[39mdict\u001b[39m, key: \u001b[39mstr\u001b[39m) \u001b[39m-\u001b[39m\u001b[39m>\u001b[39m \u001b[39mNone\u001b[39;00m:\n\u001b[1;32m    183\u001b[0m     \u001b[39m\"\"\"\u001b[39;00m\n\u001b[1;32m    184\u001b[0m \u001b[39m    Arguments:\u001b[39;00m\n\u001b[1;32m    185\u001b[0m \u001b[39m        val: (dictionary) A dictionary of the following form:\u001b[39;00m\n\u001b[0;32m   (...)\u001b[0m\n\u001b[1;32m    212\u001b[0m \u001b[39m            The readable name or id for this set of bounding boxes (e.g. predictions, ground_truth)\u001b[39;00m\n\u001b[1;32m    213\u001b[0m \u001b[39m    \"\"\"\u001b[39;00m\n\u001b[0;32m--> 214\u001b[0m     \u001b[39msuper\u001b[39;49m()\u001b[39m.\u001b[39;49m\u001b[39m__init__\u001b[39;49m(val)\n\u001b[1;32m    215\u001b[0m     \u001b[39mself\u001b[39m\u001b[39m.\u001b[39m_val \u001b[39m=\u001b[39m val[\u001b[39m\"\u001b[39m\u001b[39mbox_data\u001b[39m\u001b[39m\"\u001b[39m]\n\u001b[1;32m    216\u001b[0m     \u001b[39mself\u001b[39m\u001b[39m.\u001b[39m_key \u001b[39m=\u001b[39m key\n\nFile \u001b[0;32m~/.local/share/virtualenvs/haar-tXjf4l0j/lib/python3.9/site-packages/wandb/sdk/data_types/base_types/json_metadata.py:36\u001b[0m, in \u001b[0;36mJSONMetadata.__init__\u001b[0;34m(self, val)\u001b[0m\n\u001b[1;32m     34\u001b[0m tmp_path \u001b[39m=\u001b[39m os\u001b[39m.\u001b[39mpath\u001b[39m.\u001b[39mjoin(MEDIA_TMP\u001b[39m.\u001b[39mname, util\u001b[39m.\u001b[39mgenerate_id() \u001b[39m+\u001b[39m ext)\n\u001b[1;32m     35\u001b[0m \u001b[39mwith\u001b[39;00m codecs\u001b[39m.\u001b[39mopen(tmp_path, \u001b[39m\"\u001b[39m\u001b[39mw\u001b[39m\u001b[39m\"\u001b[39m, encoding\u001b[39m=\u001b[39m\u001b[39m\"\u001b[39m\u001b[39mutf-8\u001b[39m\u001b[39m\"\u001b[39m) \u001b[39mas\u001b[39;00m fp:\n\u001b[0;32m---> 36\u001b[0m     util\u001b[39m.\u001b[39;49mjson_dump_uncompressed(\u001b[39mself\u001b[39;49m\u001b[39m.\u001b[39;49m_val, fp)\n\u001b[1;32m     37\u001b[0m \u001b[39mself\u001b[39m\u001b[39m.\u001b[39m_set_file(tmp_path, is_tmp\u001b[39m=\u001b[39m\u001b[39mTrue\u001b[39;00m, extension\u001b[39m=\u001b[39mext)\n\nFile \u001b[0;32m~/.local/share/virtualenvs/haar-tXjf4l0j/lib/python3.9/site-packages/wandb/util.py:820\u001b[0m, in \u001b[0;36mjson_dump_uncompressed\u001b[0;34m(obj, fp, **kwargs)\u001b[0m\n\u001b[1;32m    818\u001b[0m \u001b[39mdef\u001b[39;00m \u001b[39mjson_dump_uncompressed\u001b[39m(obj: Any, fp: IO[\u001b[39mstr\u001b[39m], \u001b[39m*\u001b[39m\u001b[39m*\u001b[39mkwargs: Any) \u001b[39m-\u001b[39m\u001b[39m>\u001b[39m \u001b[39mNone\u001b[39;00m:\n\u001b[1;32m    819\u001b[0m     \u001b[39m\"\"\"Convert obj to json, with some extra encodable types.\"\"\"\u001b[39;00m\n\u001b[0;32m--> 820\u001b[0m     \u001b[39mreturn\u001b[39;00m json\u001b[39m.\u001b[39;49mdump(obj, fp, \u001b[39mcls\u001b[39;49m\u001b[39m=\u001b[39;49mJSONEncoderUncompressed, \u001b[39m*\u001b[39;49m\u001b[39m*\u001b[39;49mkwargs)\n\nFile \u001b[0;32m/usr/lib/python3.9/json/__init__.py:179\u001b[0m, in \u001b[0;36mdump\u001b[0;34m(obj, fp, skipkeys, ensure_ascii, check_circular, allow_nan, cls, indent, separators, default, sort_keys, **kw)\u001b[0m\n\u001b[1;32m    173\u001b[0m     iterable \u001b[39m=\u001b[39m \u001b[39mcls\u001b[39m(skipkeys\u001b[39m=\u001b[39mskipkeys, ensure_ascii\u001b[39m=\u001b[39mensure_ascii,\n\u001b[1;32m    174\u001b[0m         check_circular\u001b[39m=\u001b[39mcheck_circular, allow_nan\u001b[39m=\u001b[39mallow_nan, indent\u001b[39m=\u001b[39mindent,\n\u001b[1;32m    175\u001b[0m         separators\u001b[39m=\u001b[39mseparators,\n\u001b[1;32m    176\u001b[0m         default\u001b[39m=\u001b[39mdefault, sort_keys\u001b[39m=\u001b[39msort_keys, \u001b[39m*\u001b[39m\u001b[39m*\u001b[39mkw)\u001b[39m.\u001b[39miterencode(obj)\n\u001b[1;32m    177\u001b[0m \u001b[39m# could accelerate with writelines in some versions of Python, at\u001b[39;00m\n\u001b[1;32m    178\u001b[0m \u001b[39m# a debuggability cost\u001b[39;00m\n\u001b[0;32m--> 179\u001b[0m \u001b[39mfor\u001b[39;00m chunk \u001b[39min\u001b[39;00m iterable:\n\u001b[1;32m    180\u001b[0m     fp\u001b[39m.\u001b[39mwrite(chunk)\n\nFile \u001b[0;32m/usr/lib/python3.9/json/encoder.py:431\u001b[0m, in \u001b[0;36m_make_iterencode.<locals>._iterencode\u001b[0;34m(o, _current_indent_level)\u001b[0m\n\u001b[1;32m    429\u001b[0m     \u001b[39myield from\u001b[39;00m _iterencode_list(o, _current_indent_level)\n\u001b[1;32m    430\u001b[0m \u001b[39melif\u001b[39;00m \u001b[39misinstance\u001b[39m(o, \u001b[39mdict\u001b[39m):\n\u001b[0;32m--> 431\u001b[0m     \u001b[39myield from\u001b[39;00m _iterencode_dict(o, _current_indent_level)\n\u001b[1;32m    432\u001b[0m \u001b[39melse\u001b[39;00m:\n\u001b[1;32m    433\u001b[0m     \u001b[39mif\u001b[39;00m markers \u001b[39mis\u001b[39;00m \u001b[39mnot\u001b[39;00m \u001b[39mNone\u001b[39;00m:\n\nFile \u001b[0;32m/usr/lib/python3.9/json/encoder.py:405\u001b[0m, in \u001b[0;36m_make_iterencode.<locals>._iterencode_dict\u001b[0;34m(dct, _current_indent_level)\u001b[0m\n\u001b[1;32m    403\u001b[0m         \u001b[39melse\u001b[39;00m:\n\u001b[1;32m    404\u001b[0m             chunks \u001b[39m=\u001b[39m _iterencode(value, _current_indent_level)\n\u001b[0;32m--> 405\u001b[0m         \u001b[39myield from\u001b[39;00m chunks\n\u001b[1;32m    406\u001b[0m \u001b[39mif\u001b[39;00m newline_indent \u001b[39mis\u001b[39;00m \u001b[39mnot\u001b[39;00m \u001b[39mNone\u001b[39;00m:\n\u001b[1;32m    407\u001b[0m     _current_indent_level \u001b[39m-\u001b[39m\u001b[39m=\u001b[39m \u001b[39m1\u001b[39m\n\nFile \u001b[0;32m/usr/lib/python3.9/json/encoder.py:325\u001b[0m, in \u001b[0;36m_make_iterencode.<locals>._iterencode_list\u001b[0;34m(lst, _current_indent_level)\u001b[0m\n\u001b[1;32m    323\u001b[0m         \u001b[39melse\u001b[39;00m:\n\u001b[1;32m    324\u001b[0m             chunks \u001b[39m=\u001b[39m _iterencode(value, _current_indent_level)\n\u001b[0;32m--> 325\u001b[0m         \u001b[39myield from\u001b[39;00m chunks\n\u001b[1;32m    326\u001b[0m \u001b[39mif\u001b[39;00m newline_indent \u001b[39mis\u001b[39;00m \u001b[39mnot\u001b[39;00m \u001b[39mNone\u001b[39;00m:\n\u001b[1;32m    327\u001b[0m     _current_indent_level \u001b[39m-\u001b[39m\u001b[39m=\u001b[39m \u001b[39m1\u001b[39m\n\nFile \u001b[0;32m/usr/lib/python3.9/json/encoder.py:405\u001b[0m, in \u001b[0;36m_make_iterencode.<locals>._iterencode_dict\u001b[0;34m(dct, _current_indent_level)\u001b[0m\n\u001b[1;32m    403\u001b[0m         \u001b[39melse\u001b[39;00m:\n\u001b[1;32m    404\u001b[0m             chunks \u001b[39m=\u001b[39m _iterencode(value, _current_indent_level)\n\u001b[0;32m--> 405\u001b[0m         \u001b[39myield from\u001b[39;00m chunks\n\u001b[1;32m    406\u001b[0m \u001b[39mif\u001b[39;00m newline_indent \u001b[39mis\u001b[39;00m \u001b[39mnot\u001b[39;00m \u001b[39mNone\u001b[39;00m:\n\u001b[1;32m    407\u001b[0m     _current_indent_level \u001b[39m-\u001b[39m\u001b[39m=\u001b[39m \u001b[39m1\u001b[39m\n\nFile \u001b[0;32m/usr/lib/python3.9/json/encoder.py:405\u001b[0m, in \u001b[0;36m_make_iterencode.<locals>._iterencode_dict\u001b[0;34m(dct, _current_indent_level)\u001b[0m\n\u001b[1;32m    403\u001b[0m         \u001b[39melse\u001b[39;00m:\n\u001b[1;32m    404\u001b[0m             chunks \u001b[39m=\u001b[39m _iterencode(value, _current_indent_level)\n\u001b[0;32m--> 405\u001b[0m         \u001b[39myield from\u001b[39;00m chunks\n\u001b[1;32m    406\u001b[0m \u001b[39mif\u001b[39;00m newline_indent \u001b[39mis\u001b[39;00m \u001b[39mnot\u001b[39;00m \u001b[39mNone\u001b[39;00m:\n\u001b[1;32m    407\u001b[0m     _current_indent_level \u001b[39m-\u001b[39m\u001b[39m=\u001b[39m \u001b[39m1\u001b[39m\n\nFile \u001b[0;32m/usr/lib/python3.9/json/encoder.py:438\u001b[0m, in \u001b[0;36m_make_iterencode.<locals>._iterencode\u001b[0;34m(o, _current_indent_level)\u001b[0m\n\u001b[1;32m    436\u001b[0m         \u001b[39mraise\u001b[39;00m \u001b[39mValueError\u001b[39;00m(\u001b[39m\"\u001b[39m\u001b[39mCircular reference detected\u001b[39m\u001b[39m\"\u001b[39m)\n\u001b[1;32m    437\u001b[0m     markers[markerid] \u001b[39m=\u001b[39m o\n\u001b[0;32m--> 438\u001b[0m o \u001b[39m=\u001b[39m _default(o)\n\u001b[1;32m    439\u001b[0m \u001b[39myield from\u001b[39;00m _iterencode(o, _current_indent_level)\n\u001b[1;32m    440\u001b[0m \u001b[39mif\u001b[39;00m markers \u001b[39mis\u001b[39;00m \u001b[39mnot\u001b[39;00m \u001b[39mNone\u001b[39;00m:\n\nFile \u001b[0;32m~/.local/share/virtualenvs/haar-tXjf4l0j/lib/python3.9/site-packages/wandb/util.py:804\u001b[0m, in \u001b[0;36mJSONEncoderUncompressed.default\u001b[0;34m(self, obj)\u001b[0m\n\u001b[1;32m    802\u001b[0m \u001b[39melif\u001b[39;00m np \u001b[39mand\u001b[39;00m \u001b[39misinstance\u001b[39m(obj, np\u001b[39m.\u001b[39mgeneric):\n\u001b[1;32m    803\u001b[0m     obj \u001b[39m=\u001b[39m obj\u001b[39m.\u001b[39mitem()\n\u001b[0;32m--> 804\u001b[0m \u001b[39mreturn\u001b[39;00m json\u001b[39m.\u001b[39;49mJSONEncoder\u001b[39m.\u001b[39;49mdefault(\u001b[39mself\u001b[39;49m, obj)\n\nFile \u001b[0;32m/usr/lib/python3.9/json/encoder.py:179\u001b[0m, in \u001b[0;36mJSONEncoder.default\u001b[0;34m(self, o)\u001b[0m\n\u001b[1;32m    160\u001b[0m \u001b[39mdef\u001b[39;00m \u001b[39mdefault\u001b[39m(\u001b[39mself\u001b[39m, o):\n\u001b[1;32m    161\u001b[0m     \u001b[39m\"\"\"Implement this method in a subclass such that it returns\u001b[39;00m\n\u001b[1;32m    162\u001b[0m \u001b[39m    a serializable object for ``o``, or calls the base implementation\u001b[39;00m\n\u001b[1;32m    163\u001b[0m \u001b[39m    (to raise a ``TypeError``).\u001b[39;00m\n\u001b[0;32m   (...)\u001b[0m\n\u001b[1;32m    177\u001b[0m \n\u001b[1;32m    178\u001b[0m \u001b[39m    \"\"\"\u001b[39;00m\n\u001b[0;32m--> 179\u001b[0m     \u001b[39mraise\u001b[39;00m \u001b[39mTypeError\u001b[39;00m(\u001b[39mf\u001b[39m\u001b[39m'\u001b[39m\u001b[39mObject of type \u001b[39m\u001b[39m{\u001b[39;00mo\u001b[39m.\u001b[39m\u001b[39m__class__\u001b[39m\u001b[39m.\u001b[39m\u001b[39m__name__\u001b[39m\u001b[39m}\u001b[39;00m\u001b[39m \u001b[39m\u001b[39m'\u001b[39m\n\u001b[1;32m    180\u001b[0m                     \u001b[39mf\u001b[39m\u001b[39m'\u001b[39m\u001b[39mis not JSON serializable\u001b[39m\u001b[39m'\u001b[39m)\n\n\u001b[0;31mTypeError\u001b[0m: Object of type int is not JSON serializable"
}

It’s your class labels:

class_labels = {
    0: "face"
}

json does not support numerics as keys

Hi @johko-cel , apologies for delayed response, I was out of office. Upon review of your colab work, the error stems from you attempting to predict outside the dimensions of the frame set. Example: If I was to divide the width and height by 2, then you will not hit an error as the prediction box will be within the dimensions of the frame.

 box = {
                "position": {
                    "middle": [midX, midY],
                    "width": w/2,
                    "height": h/2
                },

The confusion is unfortunately due to our printed traceback message, this is not a JSON related error. I will file a report with engineering to address this.

Thank you for investigating @mohammadbakir , I also was out of office for a while, so no worries.

Actually when I turn the 0 into a string, it gives me the error TypeError: Class labels must be a dictionary of numbers to string
But I guess @mohammadbakir found the right issue in the code.

Hey Mohammad,
actually today I came back to the project and found something interesting about the bounding box.
Actually, it is not outside of the dimensions of the frame set and I can do a little trick to get the right size of the bounding box working:

 box = {
                "position": {
                    "middle": [midX, midY],
                    "width": (w/2)+(w/2),
                    "height": (h/2)+(h/2)
                },

So what is the difference here? After dividing, the coordinates are float values and not int anymore and this makes it work(can also just cast them). Sounds like a bug and I just created an issue for it: [CLI]: Logging bounding boxes only works with float dimensions, not int · Issue #3982 · wandb/wandb · GitHub