Failed to verify certificate: x509: certificate signed by unknown authority

W&B Help
1

I am running a SLURM job through my school’s cluster.
I’m able to log in to my account using wandb.login(), but, whenever I try to post to https://api.wandb.ai/graphql, there is a certification error (i.e., when using .init() or defining a logger WandbLogger() from lightning).

wandb: Using wandb-core as the SDK backend. Please refer to https://wandb.me/wandb-core for more information.
wandb: Currently logged in as: hsp287. Use `wandb login --relogin` to force relogin
wandb: Appending key for api.wandb.ai to your netrc file: /home/hsp287/.netrc
wandb: ERROR failed to upsert bucket: api: failed sending: POST https://api.wandb.ai/graphql giving up after 1 attempt(s): Post "https://api.wandb.ai/graphql": tls: failed to verify certificate: x509: certificate signed by unknown authority
2

Ok, the issue is resolved.
The issue came from using a singularity container to run the script. I think there may be some additional steps when using singularity for certification.

To solve this, I switched to using a venv and installing the needed packages directly in my project directory.

1 Like
3

Hi @hsp287, great to know you managed to fix this and thank you for sharing the fix :raised_hands: . I will mark this as resolved, please don’t hesitate to reach out in the future for any further help.

1 Like
4

Thanks. I also just wanted to follow-up if anyone wanted to use a singularity container.
The solution that worked for me was the following:

  1. Download cacert.pem from the web (https://curl.se/ca/cacert.pem) using curl or wget.
  2. In the sbatch script, export the path to the cacert.pem file:
    export SSL_CERT_FILE=/path/to/cacert.pem