# Wandb error by usage of mlflow and hydra regarding protobuf lib

**URL:** <https://community.wandb.ai/t/wandb-error-by-usage-of-mlflow-and-hydra-regarding-protobuf-lib/3866>\
**Category:** W&B Help\
**Tags:** wandb\
**Created:** [February 12, 2023, 5:17pm UTC](https://community.wandb.ai/t/wandb-error-by-usage-of-mlflow-and-hydra-regarding-protobuf-lib/3866 "2023-02-12T17:17:38Z")\
**Posts on this page:** 5\
**Page:** 1

<div class="post-metadata">

**Author:** ![ilo](https://avatars.discourse-cdn.com/v4/letter/i/67e7ee/32.png) [@ilo](https://community.wandb.ai/u/ilo)\
**Post date:** [February 12, 2023, 5:17pm UTC](https://community.wandb.ai/t/wandb-error-by-usage-of-mlflow-and-hydra-regarding-protobuf-lib/3866/1 "2023-02-12T17:17:38Z")

</div>

Hi all,  
having an issue and need some help for Python project inside a Udacity course.  
We shall use Python 3.8, therefore I am using 3.8.16 and having created a virtual env starting from that version and activated virtual env.

Using Jupyter Lab I am trying to use a pipeline for mlflow, hydra and wandb I am getting the same error as mentioned e.g. in

> <https://github.com/espnet/espnet/issues/3708>
>
> While Looking up ESPNet using this Tutorial: https://colab.research.google.com/d…rive/1L85G7jdhsI1QKs2o0qCGEbhm5X4QV2zN
> 
> I came across this issue when the Library "Wandb" is imported.
> 
> \`\`\`
> \# python3 -m espnet2.bin.asr\_train --collect\_stats true --use\_preprocessor true --bpemodel data/token\_list/bpe\_unigram30/bpe.model --token\_type bpe --token\_list data/token\_list/bpe\_unigram30/tokens.txt --non\_linguistic\_symbols none --cleaner none --g2p none --train\_data\_path\_and\_name\_and\_type dump/raw/train\_nodev/wav.scp,speech,sound --train\_data\_path\_and\_name\_and\_type dump/raw/train\_nodev/text,text,text --valid\_data\_path\_and\_name\_and\_type dump/raw/train\_dev/wav.scp,speech,sound --valid\_data\_path\_and\_name\_and\_type dump/raw/train\_dev/text,text,text --train\_shape\_file exp/asr\_stats\_raw\_bpe30/logdir/train.32.scp --valid\_shape\_file exp/asr\_stats\_raw\_bpe30/logdir/valid.32.scp --output\_dir exp/asr\_stats\_raw\_bpe30/logdir/stats.32 --frontend\_conf fs=16k 
> \# Started at Wed Oct 27 16:36:43 EDT 2021
> \`\`\`
> 
> \`\`\`
>     from .wandb\_init import \_attach, init  # noqa: F401
>   File "/home/debayan/.local/lib/python3.8/site-packages/wandb/sdk/wandb\_init.py", line 35, in \<module\>
>     from .backend.backend import Backend
>   File "/home/debayan/.local/lib/python3.8/site-packages/wandb/sdk/backend/backend.py", line 20, in \<module\>
>     from ..interface import interface
>   File "/home/debayan/.local/lib/python3.8/site-packages/wandb/sdk/interface/interface.py", line 21, in \<module\>
>     from wandb.proto import wandb\_internal\_pb2 as pb
>   File "/home/debayan/.local/lib/python3.8/site-packages/wandb/proto/wandb\_internal\_pb2.py", line 15, in \<module\>
>     from wandb.proto import wandb\_base\_pb2 as wandb\_dot\_proto\_dot\_wandb\_\_base\_\_pb2
>   File "/home/debayan/.local/lib/python3.8/site-packages/wandb/proto/wandb\_base\_pb2.py", line 21, in \<module\>
>     create\_key=\_descriptor.\_internal\_create\_key,
> AttributeError: module 'google.protobuf.descriptor' has no attribute '\_internal\_create\_key
> \`\`\`
> 
> This was in Wandb version 0.12.6 .
> 
> However, when I reverted to Wandb version 0.12.2, it started working as expected.
> 
> Feel free to reach out if any questions!

I tried already the mentioned protobuf version 3.20.0 and 3.20.1 and different wandb versions via conda.yml, always getting the same error:

```auto
name: download_data
channels:
  - conda-forge
  - defaults
dependencies:
  - python=3.8
  - requests=2.24.0
  - pip=20.3.3
  - mlflow=2.1.1
  - hydra-core=1.3.1 #1.0.6
# - wandb=0.13.10
  - pip:
      #- wandb==0.10.21
      - protobuf==3.20.0
      - wandb==0.12.2
      - hydra-joblib-launcher==1.1.2

```

Still getting the following stacktrace for the ML model and its param setting:  
🌞 ❌ mlflow run . -P hydra\_options=“random\_forest\_pipeline.random\_forest.max\_depth=5”  
2023/02/12 17:55:15 INFO mlflow.utils.conda: === Creating conda environment mlflow-8284cfd5101c5c151da499d35f932662f514265c ===  
Collecting package metadata (repodata.json): …working… done  
Solving environment: …working… done  
Preparing transaction: …working… done  
Verifying transaction: …working… done  
Executing transaction: …working… done  
Installing pip dependencies: …working… done  
2023/02/12 18:02:28 INFO mlflow.projects.utils: === Created directory /tmp/tmpcv9ww8ry for downloading remote URIs passed to arguments of type ‘path’ ===  
2023/02/12 18:02:28 INFO mlflow.projects.backend.local: === Running command ‘source /home/ilona/miniconda3/bin/…/etc/profile.d/conda.sh && conda activate mlflow-8284cfd5101c5c151da499d35f932662f514265c 1\>&2 && python main.py $(echo random\_forest\_pipeline.random\_forest.max\_depth=5)’ in run with ID ‘a6d323399b9242c1b227bae18109b69d’ ===  
2023/02/12 18:02:39 INFO mlflow.utils.conda: Conda environment mlflow-70e71a7afdc413046f59e7b87abd03dc50d8745f already exists.  
2023/02/12 18:02:39 INFO mlflow.projects.utils: === Created directory /tmp/tmpsxd\_dtnx for downloading remote URIs passed to arguments of type ‘path’ ===  
2023/02/12 18:02:39 INFO mlflow.projects.backend.local: === Running command ‘source /home/ilona/miniconda3/bin/…/etc/profile.d/conda.sh && conda activate mlflow-70e71a7afdc413046f59e7b87abd03dc50d8745f 1\>&2 && python run.py --train\_data exercise\_6/data\_train.csv:latest   
–model\_config /home/ilona/MLOps/nd0821-c2-build-model-workflow-exercises/lesson-4-training-validation-experiment-tracking/exercises/exercise\_11/starter/outputs/2023-02-12/18-02-33/random\_forest\_config.yml’ in run with ID ‘59002ca997e54d1dac9fd30e5a418009’ ===  
Traceback (most recent call last):  
File “run.py”, line 15, in   
import wandb  
File “/home/ilona/miniconda3/envs/mlflow-70e71a7afdc413046f59e7b87abd03dc50d8745f/lib/python3.8/site-packages/wandb/ **init**.py”, line 38, in   
from wandb import sdk as wandb\_sdk  
File “/home/ilona/miniconda3/envs/mlflow-70e71a7afdc413046f59e7b87abd03dc50d8745f/lib/python3.8/site-packages/wandb/sdk/ **init**.py”, line 12, in   
from .wandb\_init import init # noqa: F401  
File “/home/ilona/miniconda3/envs/mlflow-70e71a7afdc413046f59e7b87abd03dc50d8745f/lib/python3.8/site-packages/wandb/sdk/wandb\_init.py”, line 29, in   
from .backend.backend import Backend  
File “/home/ilona/miniconda3/envs/mlflow-70e71a7afdc413046f59e7b87abd03dc50d8745f/lib/python3.8/site-packages/wandb/sdk/backend/backend.py”, line 17, in   
from …interface import interface  
File “/home/ilona/miniconda3/envs/mlflow-70e71a7afdc413046f59e7b87abd03dc50d8745f/lib/python3.8/site-packages/wandb/sdk/interface/interface.py”, line 18, in   
from wandb.proto import wandb\_internal\_pb2 as pb  
File “/home/ilona/miniconda3/envs/mlflow-70e71a7afdc413046f59e7b87abd03dc50d8745f/lib/python3.8/site-packages/wandb/proto/wandb\_internal\_pb2.py”, line 15, in   
from wandb.proto import wandb\_telemetry\_pb2 as wandb\_dot\_proto\_dot\_wandb\_\_telemetry\_\_pb2  
File “/home/ilona/miniconda3/envs/mlflow-70e71a7afdc413046f59e7b87abd03dc50d8745f/lib/python3.8/site-packages/wandb/proto/wandb\_telemetry\_pb2.py”, line 34, in   
\_descriptor.FieldDescriptor(  
File “/home/ilona/miniconda3/envs/mlflow-70e71a7afdc413046f59e7b87abd03dc50d8745f/lib/python3.8/site-packages/google/protobuf/descriptor.py”, line 560, in **new**  
\_message.Message.\_CheckCalledFromGeneratedFile()  
TypeError: Descriptors cannot not be created directly.  
If this call came from a \_pb2.py file, your generated code is out of date and must be regenerated with protoc \>= 3.19.0.  
If you cannot immediately regenerate your protos, some other possible workarounds are:

1. Downgrade the protobuf package to 3.20.x or lower.
2. Set PROTOCOL\_BUFFERS\_PYTHON\_IMPLEMENTATION=python (but this will use pure-Python parsing and will be much slower).

More information: [Changes made on May 6, 2022 | Protocol Buffers Documentation](https://developers.google.com/protocol-buffers/docs/news/2022-05-06#python-updates)  
…

In advance, thank you very much for help and solution proposals,  
Ilona

---

<div class="post-metadata">

**Author:** ![bill-morrisson](https://avatars.discourse-cdn.com/v4/letter/b/90ced4/32.png) [@bill-morrisson](https://community.wandb.ai/u/bill-morrisson)\
**Post date:** [February 16, 2023, 3:23pm UTC](https://community.wandb.ai/t/wandb-error-by-usage-of-mlflow-and-hydra-regarding-protobuf-lib/3866/2 "2023-02-16T15:23:23Z")

</div>

Hey Ilona,

Thank you for contacting us and bringing this up.  
I have tried reproducing it but don’t get the same error.

To clear up any misunderstandings I wish to know if you are using virtualenv package in python and/or conda envs? Mixing up both may cause import issues in general.  
Could you check if you have a virtualenv in conda env or vice versa? Let me know if that is not the case.

Also the error message from the issue sent isn’t exactly the same though it is the same cause.  
Does the error happen just when you are trying to import wandb or without?

---

<div class="post-metadata">

**Author:** ![bill-morrisson](https://avatars.discourse-cdn.com/v4/letter/b/90ced4/32.png) [@bill-morrisson](https://community.wandb.ai/u/bill-morrisson)\
**Post date:** [February 22, 2023, 5:26pm UTC](https://community.wandb.ai/t/wandb-error-by-usage-of-mlflow-and-hydra-regarding-protobuf-lib/3866/3 "2023-02-22T17:26:04Z")

</div>

Hello Ilona,

Following up to know if you have tried what I mentioned?

---

<div class="post-metadata">

**Author:** ![bill-morrisson](https://avatars.discourse-cdn.com/v4/letter/b/90ced4/32.png) [@bill-morrisson](https://community.wandb.ai/u/bill-morrisson)\
**Post date:** [February 28, 2023, 10:38pm UTC](https://community.wandb.ai/t/wandb-error-by-usage-of-mlflow-and-hydra-regarding-protobuf-lib/3866/4 "2023-02-28T22:38:49Z")

</div>

Hello Ilona,

I will be closing this ticket for lack of response. If you encounter the same issue or another one feel free to contact us.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/flex020/uploads/wandb/original/1X/366b649231631dbab896843020da0056074ac79d.png) [@system](https://community.wandb.ai/u/system)\
**Post date:** [April 13, 2023, 5:18pm UTC](https://community.wandb.ai/t/wandb-error-by-usage-of-mlflow-and-hydra-regarding-protobuf-lib/3866/5 "2023-04-13T17:18:32Z")

</div>

This topic was automatically closed 60 days after the last reply. New replies are no longer allowed.