Artifacts download/change table without downloading whole artifact

mirokery · June 30, 2023, 3:05pm

Hi all,

I am working on Effective MLOps: Model Development. I got my own dataset of images around 8 GB.
I created artifact and table.I am working on spliting dataset into test,train,val.

github.com

wandb/edu/blob/main/mlops-001/lesson1/02_Split.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<a href=\"https://colab.research.google.com/github/wandb/edu/blob/main/mlops-001/lesson1/02_Split.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>\n",
    "<!--- @wandbcode{course-lesson1} -->"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "dee477f2",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Install dependencies (run once)\n",
    "!wget https://raw.githubusercontent.com/wandb/edu/main/mlops-001/lesson1/requirements.txt\n",
    "!wget https://raw.githubusercontent.com/wandb/edu/main/mlops-001/lesson1/params.py\n",

This file has been truncated. show original

Is there any possibility to join tables without downloading whole artifact ? or should I add there another table by setting Artifact(increment=True) and somehow merge them in application?

Thanks Miro

luis_bergua · July 5, 2023, 11:25am

Hi @mirokery, thanks for writing in! On the UI, you can join tables by using a Weave expression like [runs.summary["table2"], runs.summary["table1"]].concat. Please let me know if this would be useful for you and feel free to share your Workspace if you have any issues and I will help you!

luis_bergua · July 7, 2023, 8:23am

Hi Miro,

We wanted to follow up with you regarding your support request as we have not heard back from you. Please let us know if we can be of further assistance or if your issue has been resolved.

Best,
Weights & Biases

luis_bergua · July 10, 2023, 10:44am

Hi Miro,

Since we have not heard back from you we are going to close this request. If you would like to re-open the conversation, please let us know!

Best,
Luis

system · September 3, 2023, 11:25am

This topic was automatically closed 60 days after the last reply. New replies are no longer allowed.

Topic		Replies	Views
Table with images much larger than originals W&B Help artifacts , tables	9	890	May 16, 2023
How to create wandb.Table with image previews for a big dataset with most efficiency? W&B Help artifacts , tables , questions , wandb	7	921	June 17, 2023
Add file to artifact without downloading it W&B Help artifacts	6	1341	May 8, 2023
Memory limit when uploading a image dataset as table W&B Help artifacts	6	123	May 7, 2024
Best Practices for WandB Artifacts W&B Help artifacts	4	712	February 10, 2023

Artifacts download/change table without downloading whole artifact

Related topics