Permission denied when trying to access artifact versions in pytorch lightning callback

W&B Help
, ,
1

I’m trying to access artifact versions in pytorch lightning callback and I’m still getting wandb.errors.CommError: Permission denied to access (...) exception. However, when I run the same code from standalone script or even in the same codebase but going step by step using debugger it works fine. I suppose it’s somehow caused by the fact it’s ran from the callback. How can I fix it?
The minimum callback code below:

import pytorch_lightning as pl
import wandb


class MyCallback(pl.callbacks.Callback):
    def __init__(
        self,
    ) -> None:
        self._artifact_type = "my_type"
        self._artifact_name = "my_name"
        self._api = wandb.Api(
            overrides={"entity": wandb.run.entity, "project": wandb.run.project}
        )

    def on_validation_end(
        self, trainer: pl.Trainer, pl_module: pl.LightningModule
    ) -> None:
        self._log_artifact(pl_module)
        self._remove_old_versions()

    def _log_artifact(self, pl_module: pl.LightningModule) -> None:
        ...

    def _remove_old_versions(self) -> None:
        for v in self._api.artifact_versions(self._artifact_type, self._artifact_name):
            if "latest" not in v.aliases:
                v.delete()
2

Hi @develop, the fact that this does work in debugging mode leads me to think this may be an issue with the lazy loading the API uses. Could you try to reinstantiate the wandb api in your remove_old_versions(self)function? For example:
self._api = wandb.Api(overrides={"entity": wandb.run.entity, "project": wandb.run.project}

Before you iterate through the Artifact versions that exist?

1 Like
3

Hi @develop, I wanted to follow up and see if you had a chance to try this out?

4

This topic was automatically closed 60 days after the last reply. New replies are no longer allowed.