Feature Request - Bulk Download Logs

W&B Help
1

It would be nice if you could multiselect runs then bulk download logs into a zip file. This way you can filter by failed runs, grab all of those logs, then process them to see what the common errors are, how they relate to your configuration etc.

2

Hi @mjvolk3 - while this is currently not possible via the UI, you can download the log files for all the failed Runs in a project via the API, with something like the following script:


import wandb
import os

entity = ""
project = ""
dst_folder = f"{project}-failed_runs_logs"

#Initialize a W&B API client
api = wandb.Api()

#Get the failed runs from the project
failed_runs = api.runs(path=f"{entity}/{project}", filters={"state": "failed"})

#Create a directory for the logs
os.makedirs(dst_folder, exist_ok=True)

#Download the logs for each failed run
for run in failed_runs:
    log_files = run.files()
    for file in log_files:
        if file.name.endswith('.log'):
            file_name=file.name
            file.download(root=f"{dst_folder}/wandb-{run.id}", replace=True)

It’s also worth considering that only the output.log file (with all stdout and stderr from the training) is uploaded to the Files sections, while the debug.log and debug-internal.log are not uploaded but should be available in the ./wandb/run-<date_time>-<runid>/logs folder on the device running the training.

Let me know if you have any further questions.

Finally, I have also raised your suggestion as a feature request for our product team to review.