How to install wandb on a docker image for arm?

My docker building failed at the RUN

with:

(meta_learning) brandomiranda~ ❯ docker build -f ~/iit-term-synthesis/Dockerfile_arm -t brandojazz/iit-term-synthesis:test_arm ~/iit-term-synthesis/

[+] Building 184.7s (20/28)
 => [internal] load build definition from Dockerfile_arm                                                                                           0.0s
 => => transferring dockerfile: 41B                                                                                                                0.0s
 => [internal] load .dockerignore                                                                                                                  0.0s
 => => transferring context: 2B                                                                                                                    0.0s
 => [internal] load metadata for docker.io/continuumio/miniconda3:latest                                                                           0.0s
 => [ 1/24] FROM docker.io/continuumio/miniconda3                                                                                                  0.0s
 => https://api.github.com/repos/IBM/pycoq/git/refs/heads/main                                                                                     0.3s
 => CACHED [ 2/24] RUN apt-get update   && apt-get install -y --no-install-recommends     ssh     git     m4     libgmp-dev     opam     wget      0.0s
 => CACHED [ 3/24] RUN useradd -m bot                                                                                                              0.0s
 => CACHED [ 4/24] WORKDIR /home/bot                                                                                                               0.0s
 => CACHED [ 5/24] ADD https://api.github.com/repos/IBM/pycoq/git/refs/heads/main version.json                                                     0.0s
 => CACHED [ 6/24] RUN opam init --disable-sandboxing                                                                                              0.0s
 => CACHED [ 7/24] RUN opam switch create ocaml-variants.4.07.1+flambda_coq-serapi.8.11.0+0.11.1 ocaml-variants.4.07.1+flambda                     0.0s
 => CACHED [ 8/24] RUN opam switch ocaml-variants.4.07.1+flambda_coq-serapi.8.11.0+0.11.1                                                          0.0s
 => CACHED [ 9/24] RUN eval $(opam env)                                                                                                            0.0s
 => CACHED [10/24] RUN opam repo add coq-released https://coq.inria.fr/opam/released                                                               0.0s
 => CACHED [11/24] RUN opam repo --all-switches add --set-default coq-released https://coq.inria.fr/opam/released                                  0.0s
 => CACHED [12/24] RUN opam update --all                                                                                                           0.0s
 => CACHED [13/24] RUN opam pin add -y coq 8.11.0                                                                                                  0.0s
 => [14/24] RUN opam install -y coq-serapi                                                                                                       176.3s
 => [15/24] RUN eval $(opam env)                                                                                                                   0.2s
 => ERROR [16/24] RUN pip install wandb --upgrade                                                                                                  8.0s
------
 > [16/24] RUN pip install wandb --upgrade:
#20 0.351 Defaulting to user installation because normal site-packages is not writeable
#20 0.637 Collecting wandb
#20 0.986   Downloading wandb-0.13.2-py2.py3-none-any.whl (1.8 MB)
#20 1.365 Requirement already satisfied: setuptools in /opt/conda/lib/python3.9/site-packages (from wandb) (61.2.0)
#20 1.366 Requirement already satisfied: six>=1.13.0 in /opt/conda/lib/python3.9/site-packages (from wandb) (1.16.0)
#20 1.409 Collecting promise<3,>=2.0
#20 1.472   Downloading promise-2.3.tar.gz (19 kB)
#20 2.087 Collecting PyYAML
#20 2.154   Downloading PyYAML-6.0-cp39-cp39-manylinux_2_17_aarch64.manylinux2014_aarch64.whl (731 kB)
#20 2.431 Collecting protobuf<4.0dev,>=3.12.0
#20 2.492   Downloading protobuf-3.20.1-cp39-cp39-manylinux2014_aarch64.whl (917 kB)
#20 2.648 Collecting setproctitle
#20 2.706   Downloading setproctitle-1.3.2-cp39-cp39-manylinux_2_17_aarch64.manylinux2014_aarch64.whl (30 kB)
#20 2.763 Collecting Click!=8.0.0,>=7.0
#20 2.818   Downloading click-8.1.3-py3-none-any.whl (96 kB)
#20 2.902 Collecting sentry-sdk>=1.0.0
#20 2.962   Downloading sentry_sdk-1.9.8-py2.py3-none-any.whl (158 kB)
#20 3.112 Collecting psutil>=5.0.0
#20 3.172   Downloading psutil-5.9.2.tar.gz (479 kB)
#20 3.871 Collecting pathtools
#20 3.937   Downloading pathtools-0.1.2.tar.gz (11 kB)
#20 4.431 Collecting shortuuid>=0.5.0
#20 4.509   Downloading shortuuid-1.0.9-py3-none-any.whl (9.4 kB)
#20 4.512 Requirement already satisfied: requests<3,>=2.0.0 in /opt/conda/lib/python3.9/site-packages (from wandb) (2.27.1)
#20 4.568 Collecting docker-pycreds>=0.4.0
#20 4.636   Downloading docker_pycreds-0.4.0-py2.py3-none-any.whl (9.0 kB)
#20 4.695 Collecting GitPython>=1.0.0
#20 4.781   Downloading GitPython-3.1.27-py3-none-any.whl (181 kB)
#20 4.834 Collecting gitdb<5,>=4.0.1
#20 4.892   Downloading gitdb-4.0.9-py3-none-any.whl (63 kB)
#20 4.934 Collecting smmap<6,>=3.0.1
#20 4.992   Downloading smmap-5.0.0-py3-none-any.whl (24 kB)
#20 5.005 Requirement already satisfied: urllib3<1.27,>=1.21.1 in /opt/conda/lib/python3.9/site-packages (from requests<3,>=2.0.0->wandb) (1.26.8)
#20 5.005 Requirement already satisfied: certifi>=2017.4.17 in /opt/conda/lib/python3.9/site-packages (from requests<3,>=2.0.0->wandb) (2021.10.8)
#20 5.006 Requirement already satisfied: idna<4,>=2.5 in /opt/conda/lib/python3.9/site-packages (from requests<3,>=2.0.0->wandb) (3.3)
#20 5.006 Requirement already satisfied: charset-normalizer~=2.0.0 in /opt/conda/lib/python3.9/site-packages (from requests<3,>=2.0.0->wandb) (2.0.4)
#20 5.075 Collecting urllib3<1.27,>=1.21.1
#20 5.135   Downloading urllib3-1.26.12-py2.py3-none-any.whl (140 kB)
#20 5.172 Building wheels for collected packages: promise, psutil, pathtools
#20 5.172   Building wheel for promise (setup.py): started
#20 5.851   Building wheel for promise (setup.py): finished with status 'done'
#20 5.852   Created wheel for promise: filename=promise-2.3-py3-none-any.whl size=21503 sha256=6de0373376d2a8e995959e6173507e13cba502c79b648b5884b1eac45d1ec9ae
#20 5.852   Stored in directory: /home/bot/.cache/pip/wheels/e1/e8/83/ddea66100678d139b14bc87692ece57c6a2a937956d2532608
#20 5.854   Building wheel for psutil (setup.py): started
#20 6.226   Building wheel for psutil (setup.py): finished with status 'error'
#20 6.226   ERROR: Command errored out with exit status 1:
#20 6.226    command: /opt/conda/bin/python -u -c 'import io, os, sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-install-vgietl2j/psutil_c905945489d349018aaad0a17600df0b/setup.py'"'"'; __file__='"'"'/tmp/pip-install-vgietl2j/psutil_c905945489d349018aaad0a17600df0b/setup.py'"'"';f = getattr(tokenize, '"'"'open'"'"', open)(__file__) if os.path.exists(__file__) else io.StringIO('"'"'from setuptools import setup; setup()'"'"');code = f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' bdist_wheel -d /tmp/pip-wheel-4y62c4eb
#20 6.226        cwd: /tmp/pip-install-vgietl2j/psutil_c905945489d349018aaad0a17600df0b/
#20 6.226   Complete output (45 lines):
#20 6.226   running bdist_wheel
#20 6.226   running build
#20 6.226   running build_py
#20 6.226   creating build
#20 6.226   creating build/lib.linux-aarch64-3.9
#20 6.226   creating build/lib.linux-aarch64-3.9/psutil
#20 6.226   copying psutil/_psosx.py -> build/lib.linux-aarch64-3.9/psutil
#20 6.226   copying psutil/_psbsd.py -> build/lib.linux-aarch64-3.9/psutil
#20 6.226   copying psutil/_common.py -> build/lib.linux-aarch64-3.9/psutil
#20 6.226   copying psutil/_pswindows.py -> build/lib.linux-aarch64-3.9/psutil
#20 6.226   copying psutil/_psposix.py -> build/lib.linux-aarch64-3.9/psutil
#20 6.226   copying psutil/__init__.py -> build/lib.linux-aarch64-3.9/psutil
#20 6.226   copying psutil/_compat.py -> build/lib.linux-aarch64-3.9/psutil
#20 6.226   copying psutil/_pslinux.py -> build/lib.linux-aarch64-3.9/psutil
#20 6.226   copying psutil/_pssunos.py -> build/lib.linux-aarch64-3.9/psutil
#20 6.226   copying psutil/_psaix.py -> build/lib.linux-aarch64-3.9/psutil
#20 6.226   creating build/lib.linux-aarch64-3.9/psutil/tests
#20 6.226   copying psutil/tests/__main__.py -> build/lib.linux-aarch64-3.9/psutil/tests
#20 6.226   copying psutil/tests/test_process.py -> build/lib.linux-aarch64-3.9/psutil/tests
#20 6.226   copying psutil/tests/test_aix.py -> build/lib.linux-aarch64-3.9/psutil/tests
#20 6.226   copying psutil/tests/test_misc.py -> build/lib.linux-aarch64-3.9/psutil/tests
#20 6.226   copying psutil/tests/test_bsd.py -> build/lib.linux-aarch64-3.9/psutil/tests
#20 6.226   copying psutil/tests/test_linux.py -> build/lib.linux-aarch64-3.9/psutil/tests
#20 6.226   copying psutil/tests/runner.py -> build/lib.linux-aarch64-3.9/psutil/tests
#20 6.226   copying psutil/tests/__init__.py -> build/lib.linux-aarch64-3.9/psutil/tests
#20 6.226   copying psutil/tests/test_connections.py -> build/lib.linux-aarch64-3.9/psutil/tests
#20 6.226   copying psutil/tests/test_unicode.py -> build/lib.linux-aarch64-3.9/psutil/tests
#20 6.226   copying psutil/tests/test_windows.py -> build/lib.linux-aarch64-3.9/psutil/tests
#20 6.226   copying psutil/tests/test_contracts.py -> build/lib.linux-aarch64-3.9/psutil/tests
#20 6.226   copying psutil/tests/test_sunos.py -> build/lib.linux-aarch64-3.9/psutil/tests
#20 6.226   copying psutil/tests/test_testutils.py -> build/lib.linux-aarch64-3.9/psutil/tests
#20 6.226   copying psutil/tests/test_osx.py -> build/lib.linux-aarch64-3.9/psutil/tests
#20 6.226   copying psutil/tests/test_memleaks.py -> build/lib.linux-aarch64-3.9/psutil/tests
#20 6.226   copying psutil/tests/test_posix.py -> build/lib.linux-aarch64-3.9/psutil/tests
#20 6.226   copying psutil/tests/test_system.py -> build/lib.linux-aarch64-3.9/psutil/tests
#20 6.226   running build_ext
#20 6.226   building 'psutil._psutil_linux' extension
#20 6.226   creating build/temp.linux-aarch64-3.9
#20 6.226   creating build/temp.linux-aarch64-3.9/psutil
#20 6.226   gcc -pthread -B /opt/conda/compiler_compat -Wno-unused-result -Wsign-compare -DNDEBUG -O2 -Wall -fPIC -O2 -n1 .2-a+fp16+rcpc+dotprod+crypto -isystem /opt/conda/include -I/opt/conda/include -fPIC -O2 -n1 .2-a+fp16+rcpc+dotprod+crypto -isystem /opt/conda/include -fPIC -DPSUTIL_POSIX=1 -DPSUTIL_SIZEOF_PID_T=4 -DPSUTIL_VERSION=592 -DPSUTIL_LINUX=1 -I/opt/conda/include/python3.9 -c psutil/_psutil_common.c -o build/temp.linux-aarch64-3.9/psutil/_psutil_common.o
#20 6.226   gcc: error: .2-a+fp16+rcpc+dotprod+crypto: No such file or directory
#20 6.226   gcc: error: .2-a+fp16+rcpc+dotprod+crypto: No such file or directory
#20 6.226   gcc: error: unrecognized command-line option ‘-n1’; did you mean ‘-n’?
#20 6.226   gcc: error: unrecognized command-line option ‘-n1’; did you mean ‘-n’?
#20 6.226   error: command '/usr/bin/gcc' failed with exit code 1
#20 6.226   ----------------------------------------
#20 6.226   ERROR: Failed building wheel for psutil
#20 6.226   Running setup.py clean for psutil
#20 6.550   Building wheel for pathtools (setup.py): started
#20 7.135   Building wheel for pathtools (setup.py): finished with status 'done'
#20 7.135   Created wheel for pathtools: filename=pathtools-0.1.2-py3-none-any.whl size=8806 sha256=8e205a0f68c9c7a3c0107d1cc40d94f1d2843c78270217378dcbe98212958b82
#20 7.135   Stored in directory: /home/bot/.cache/pip/wheels/b7/0a/67/ada2a22079218c75a88361c0782855cc72aebc4d18d0289d05
#20 7.136 Successfully built promise pathtools
#20 7.136 Failed to build psutil
#20 7.195 Installing collected packages: smmap, urllib3, gitdb, shortuuid, setproctitle, sentry-sdk, PyYAML, psutil, protobuf, promise, pathtools, GitPython, docker-pycreds, Click, wandb
#20 7.262   WARNING: The script shortuuid is installed in '/home/bot/.local/bin' which is not on PATH.
#20 7.262   Consider adding this directory to PATH or, if you prefer to suppress this warning, use --no-warn-script-location.
#20 7.345     Running setup.py install for psutil: started
#20 7.727     Running setup.py install for psutil: finished with status 'error'
#20 7.727     ERROR: Command errored out with exit status 1:
#20 7.727      command: /opt/conda/bin/python -u -c 'import io, os, sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-install-vgietl2j/psutil_c905945489d349018aaad0a17600df0b/setup.py'"'"'; __file__='"'"'/tmp/pip-install-vgietl2j/psutil_c905945489d349018aaad0a17600df0b/setup.py'"'"';f = getattr(tokenize, '"'"'open'"'"', open)(__file__) if os.path.exists(__file__) else io.StringIO('"'"'from setuptools import setup; setup()'"'"');code = f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' install --record /tmp/pip-record-gb2y421d/install-record.txt --single-version-externally-managed --user --prefix= --compile --install-headers /home/bot/.local/include/python3.9/psutil
#20 7.727          cwd: /tmp/pip-install-vgietl2j/psutil_c905945489d349018aaad0a17600df0b/
#20 7.727     Complete output (47 lines):
#20 7.727     running install
#20 7.727     /opt/conda/lib/python3.9/site-packages/setuptools/command/install.py:34: SetuptoolsDeprecationWarning: setup.py install is deprecated. Use build and pip and other standards-based tools.
#20 7.727       warnings.warn(
#20 7.727     running build
#20 7.727     running build_py
#20 7.727     creating build
#20 7.727     creating build/lib.linux-aarch64-3.9
#20 7.727     creating build/lib.linux-aarch64-3.9/psutil
#20 7.727     copying psutil/_psosx.py -> build/lib.linux-aarch64-3.9/psutil
#20 7.727     copying psutil/_psbsd.py -> build/lib.linux-aarch64-3.9/psutil
#20 7.727     copying psutil/_common.py -> build/lib.linux-aarch64-3.9/psutil
#20 7.727     copying psutil/_pswindows.py -> build/lib.linux-aarch64-3.9/psutil
#20 7.727     copying psutil/_psposix.py -> build/lib.linux-aarch64-3.9/psutil
#20 7.727     copying psutil/__init__.py -> build/lib.linux-aarch64-3.9/psutil
#20 7.727     copying psutil/_compat.py -> build/lib.linux-aarch64-3.9/psutil
#20 7.727     copying psutil/_pslinux.py -> build/lib.linux-aarch64-3.9/psutil
#20 7.727     copying psutil/_pssunos.py -> build/lib.linux-aarch64-3.9/psutil
#20 7.727     copying psutil/_psaix.py -> build/lib.linux-aarch64-3.9/psutil
#20 7.727     creating build/lib.linux-aarch64-3.9/psutil/tests
#20 7.727     copying psutil/tests/__main__.py -> build/lib.linux-aarch64-3.9/psutil/tests
#20 7.727     copying psutil/tests/test_process.py -> build/lib.linux-aarch64-3.9/psutil/tests
#20 7.727     copying psutil/tests/test_aix.py -> build/lib.linux-aarch64-3.9/psutil/tests
#20 7.727     copying psutil/tests/test_misc.py -> build/lib.linux-aarch64-3.9/psutil/tests
#20 7.727     copying psutil/tests/test_bsd.py -> build/lib.linux-aarch64-3.9/psutil/tests
#20 7.727     copying psutil/tests/test_linux.py -> build/lib.linux-aarch64-3.9/psutil/tests
#20 7.727     copying psutil/tests/runner.py -> build/lib.linux-aarch64-3.9/psutil/tests
#20 7.727     copying psutil/tests/__init__.py -> build/lib.linux-aarch64-3.9/psutil/tests
#20 7.727     copying psutil/tests/test_connections.py -> build/lib.linux-aarch64-3.9/psutil/tests
#20 7.727     copying psutil/tests/test_unicode.py -> build/lib.linux-aarch64-3.9/psutil/tests
#20 7.727     copying psutil/tests/test_windows.py -> build/lib.linux-aarch64-3.9/psutil/tests
#20 7.727     copying psutil/tests/test_contracts.py -> build/lib.linux-aarch64-3.9/psutil/tests
#20 7.727     copying psutil/tests/test_sunos.py -> build/lib.linux-aarch64-3.9/psutil/tests
#20 7.727     copying psutil/tests/test_testutils.py -> build/lib.linux-aarch64-3.9/psutil/tests
#20 7.727     copying psutil/tests/test_osx.py -> build/lib.linux-aarch64-3.9/psutil/tests
#20 7.727     copying psutil/tests/test_memleaks.py -> build/lib.linux-aarch64-3.9/psutil/tests
#20 7.727     copying psutil/tests/test_posix.py -> build/lib.linux-aarch64-3.9/psutil/tests
#20 7.727     copying psutil/tests/test_system.py -> build/lib.linux-aarch64-3.9/psutil/tests
#20 7.727     running build_ext
#20 7.727     building 'psutil._psutil_linux' extension
#20 7.727     creating build/temp.linux-aarch64-3.9
#20 7.727     creating build/temp.linux-aarch64-3.9/psutil
#20 7.727     gcc -pthread -B /opt/conda/compiler_compat -Wno-unused-result -Wsign-compare -DNDEBUG -O2 -Wall -fPIC -O2 -n1 .2-a+fp16+rcpc+dotprod+crypto -isystem /opt/conda/include -I/opt/conda/include -fPIC -O2 -n1 .2-a+fp16+rcpc+dotprod+crypto -isystem /opt/conda/include -fPIC -DPSUTIL_POSIX=1 -DPSUTIL_SIZEOF_PID_T=4 -DPSUTIL_VERSION=592 -DPSUTIL_LINUX=1 -I/opt/conda/include/python3.9 -c psutil/_psutil_common.c -o build/temp.linux-aarch64-3.9/psutil/_psutil_common.o
#20 7.727     gcc: error: .2-a+fp16+rcpc+dotprod+crypto: No such file or directory
#20 7.727     gcc: error: .2-a+fp16+rcpc+dotprod+crypto: No such file or directory
#20 7.727     gcc: error: unrecognized command-line option ‘-n1’; did you mean ‘-n’?
#20 7.727     gcc: error: unrecognized command-line option ‘-n1’; did you mean ‘-n’?
#20 7.727     error: command '/usr/bin/gcc' failed with exit code 1
#20 7.727     ----------------------------------------
#20 7.728 ERROR: Command errored out with exit status 1: /opt/conda/bin/python -u -c 'import io, os, sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-install-vgietl2j/psutil_c905945489d349018aaad0a17600df0b/setup.py'"'"'; __file__='"'"'/tmp/pip-install-vgietl2j/psutil_c905945489d349018aaad0a17600df0b/setup.py'"'"';f = getattr(tokenize, '"'"'open'"'"', open)(__file__) if os.path.exists(__file__) else io.StringIO('"'"'from setuptools import setup; setup()'"'"');code = f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' install --record /tmp/pip-record-gb2y421d/install-record.txt --single-version-externally-managed --user --prefix= --compile --install-headers /home/bot/.local/include/python3.9/psutil Check the logs for full command output.
------
executor failed running [/bin/sh -c pip install wandb --upgrade]: exit code: 1

why?

Docker file so far:

FROM continuumio/miniconda3

RUN apt-get update \
  && apt-get install -y --no-install-recommends \
    ssh \
    git \
    m4 \
    libgmp-dev \
    opam \
    wget \
    ca-certificates \
    rsync \
    strace

RUN useradd -m bot
WORKDIR /home/bot
USER bot

## https://stackoverflow.com/questions/73642349/how-to-have-miniconda-work-properly-with-docker-especially-naming-my-conda-en
#RUN wget https://repo.anaconda.com/miniconda/Miniconda3-latest-Linux-x86_64.sh  \
#    && bash Miniconda3-latest-Linux-x86_64.sh -b -f
#ENV PATH="/home/bot/miniconda3/bin:${PATH}"
#RUN conda create -n pycoq python=3.9 -y
## somehow this "works" but conda isn't fully aware of this. Fix later?
#ENV PATH="/home/bot/miniconda3/envs/pycoq/bin:${PATH}"

ADD https://api.github.com/repos/IBM/pycoq/git/refs/heads/main version.json

# -- setup opam like VP's PyCoq
RUN opam init --disable-sandboxing
# compiler + '_' + coq_serapi + '.' + coq_serapi_pin
RUN opam switch create ocaml-variants.4.07.1+flambda_coq-serapi.8.11.0+0.11.1 ocaml-variants.4.07.1+flambda
RUN opam switch ocaml-variants.4.07.1+flambda_coq-serapi.8.11.0+0.11.1
RUN eval $(opam env)

RUN opam repo add coq-released https://coq.inria.fr/opam/released
# RUN opam pin add -y coq 8.11.0
# ['opam', 'repo', '--all-switches', 'add', '--set-default', 'coq-released', 'https://coq.inria.fr/opam/released']
RUN opam repo --all-switches add --set-default coq-released https://coq.inria.fr/opam/released
RUN opam update --all
RUN opam pin add -y coq 8.11.0

#RUN opam install -y --switch ocaml-variants.4.07.1+flambda_coq-serapi_coq-serapi_8.11.0+0.11.1 coq-serapi 8.11.0+0.11.1
RUN opam install -y coq-serapi

RUN eval $(opam env)

# makes sure depedencies for pycoq are installed once already in the docker image
ENV WANDB_API_KEY="SECRET"
RUN pip install wandb --upgrade

Hi @brando , thank-you for writing in. There are several errors that appear, but primarily the exit code 1 could be attributed to you building the container using the base image cached layers. As a preliminary step, can you please try to build the container again and use the command --no-cache when doing so. Please let me know what results from this.

Hi @brando , since we have not heard back from you we are going to close this request. If you would like to re-open the conversation, please let us know!