Error with logging a wandb table that contains pandas dataframe

[mohamed_ragab]

   wandb.log({'full_results': wandb.Table(dataframe=self.averages_results_df, allow_mixed_types=True)})

error:
TypeError: keys must be str, int, float, bool or None, not a tuple

[ramit_goolry]

Hi @mohamed_ragab,

Thanks for writing in! Could you send the stack trace associated with the error? It will help up figure out what is happening better.

Thanks,
Ramit
Weights and Biases Support

[mohamed_ragab]

@ramit_goolry

stack trace associated

Traceback (most recent call last):
  File "D:/AdaTime/main.py", line 38, in <module>
    trainer.train()
  File "D:\AdaTime\trainer.py", line 143, in train
    wandb.log({'full_results': wandb.Table(dataframe=self.averages_results_df, allow_mixed_types=True)})
  File "C:\ProgramData\Anaconda3\lib\site-packages\wandb\sdk\wandb_run.py", line 1229, in log
    self.history._row_add(data)
  File "C:\ProgramData\Anaconda3\lib\site-packages\wandb\sdk\wandb_history.py", line 44, in _row_add
    self._flush()
  File "C:\ProgramData\Anaconda3\lib\site-packages\wandb\sdk\wandb_history.py", line 59, in _flush
    self._callback(row=self._data, step=self._step)
  File "C:\ProgramData\Anaconda3\lib\site-packages\wandb\sdk\wandb_run.py", line 972, in _history_callback
    self._backend.interface.publish_history(
  File "C:\ProgramData\Anaconda3\lib\site-packages\wandb\sdk\interface\interface.py", line 491, in publish_history
    data = data_types.history_dict_to_json(run, data, step=step)
  File "C:\ProgramData\Anaconda3\lib\site-packages\wandb\sdk\data_types.py", line 2325, in history_dict_to_json
    payload[key] = val_to_json(run, key, val, namespace=step)
  File "C:\ProgramData\Anaconda3\lib\site-packages\wandb\sdk\data_types.py", line 2397, in val_to_json
    art.add(val, key)
  File "C:\ProgramData\Anaconda3\lib\site-packages\wandb\sdk\wandb_artifacts.py", line 522, in add
    do_write(f)
  File "C:\ProgramData\Anaconda3\lib\site-packages\wandb\sdk\wandb_artifacts.py", line 510, in do_write
    f.write(json.dumps(val, sort_keys=True))
  File "C:\ProgramData\Anaconda3\lib\json\__init__.py", line 234, in dumps
    return cls(
  File "C:\ProgramData\Anaconda3\lib\json\encoder.py", line 199, in encode
    chunks = self.iterencode(o, _one_shot=True)
  File "C:\ProgramData\Anaconda3\lib\json\encoder.py", line 257, in iterencode
    return _iterencode(o, 0)
TypeError: keys must be str, int, float, bool or None, not tuple
wandb: ERROR Problem finishing run
Traceback (most recent call last):
  File "C:\ProgramData\Anaconda3\lib\site-packages\wandb\sdk\wandb_run.py", line 1671, in _atexit_cleanup
    self._on_finish()
  File "C:\ProgramData\Anaconda3\lib\site-packages\wandb\sdk\wandb_run.py", line 1815, in _on_finish
    self.history._flush()
  File "C:\ProgramData\Anaconda3\lib\site-packages\wandb\sdk\wandb_history.py", line 59, in _flush
    self._callback(row=self._data, step=self._step)
  File "C:\ProgramData\Anaconda3\lib\site-packages\wandb\sdk\wandb_run.py", line 972, in _history_callback
    self._backend.interface.publish_history(
  File "C:\ProgramData\Anaconda3\lib\site-packages\wandb\sdk\interface\interface.py", line 491, in publish_history
    data = data_types.history_dict_to_json(run, data, step=step)
  File "C:\ProgramData\Anaconda3\lib\site-packages\wandb\sdk\data_types.py", line 2325, in history_dict_to_json
    payload[key] = val_to_json(run, key, val, namespace=step)
  File "C:\ProgramData\Anaconda3\lib\site-packages\wandb\sdk\data_types.py", line 2397, in val_to_json
    art.add(val, key)
  File "C:\ProgramData\Anaconda3\lib\site-packages\wandb\sdk\wandb_artifacts.py", line 522, in add
    do_write(f)
  File "C:\ProgramData\Anaconda3\lib\site-packages\wandb\sdk\wandb_artifacts.py", line 510, in do_write
    f.write(json.dumps(val, sort_keys=True))
  File "C:\ProgramData\Anaconda3\lib\json\__init__.py", line 234, in dumps
    return cls(
  File "C:\ProgramData\Anaconda3\lib\json\encoder.py", line 199, in encode
    chunks = self.iterencode(o, _one_shot=True)
  File "C:\ProgramData\Anaconda3\lib\json\encoder.py", line 257, in iterencode
    return _iterencode(o, 0)
TypeError: keys must be str, int, float, bool or None, not tuple

[ramit_goolry]

Hi @mohamed_ragab,

It seems like the keys in your data frame are tuples, a feature that is not supported by Wandb Tables right now. From the looks of it, your data frame is being logged like this:

As a workaround, you can pass in the string representation of the tuple, which will allow you to pass the dataframe to wandb:

This will work to upload your data to wandb.

Thanks,
Ramit