Two questions: one about W&B Launch, and one about W&B Artifacts

[wirrywoo]

Hey everyone! Beginner here using W&B and I’m really excited to showcase my work through this platform. For my side project, I want to run Python code on a weekly basis to update an external MongoDB database containing my dataset for my ML project. I have two W&B related questions:

• What is the best way to schedule a job that executes Python code on a scheduled basis in W&B? Can this be accomplished through W&B Launch using a Docker container that stores my Python script, then following the steps here? Launch with Docker | Weights & Biases Documentation

• Does W&B Artifacts support JSON-like data structures? Would I have to download all of the responses as .json files from an API response and then upload that to W&B Artifacts? Is there a way to link a MongoDB database to W&B and use that directly?

The main reason for these questions is to see if there is a way for me to centralize all of my processes to services in W&B. For (1), my initial plan was to run a scheduled job through Heroku, and for (2) I have an existing free-tier MongoDB database that contains all of my data from the API extract contained in the Python code mentioned in (1).

Thanks!

[artsiom]

Hi Wilson,

Thank you so much for choosing W&B!

1. Unfortunately, Launch does not have a direct function to schedule jobs time-wise. Usually it starts a job off of a trigger such as an artifact being updated/ uploaded. I think in your cause a simple chron job would do the job. If you are really interested in using Launch for this, you could have a Chron job that updates a dummy artifact which then triggers a launch job for you. But other than that, unfortunately, there is no way of doing this.

2. Regarding the json format, you can definitely upload the json files to artifacts, but they will act more as a regular file you just uploaded than a json-formatted entity you can call by the API. Since W&B Artifacts is more of a file store than a database, you can store any kind of files inside of it.

Cheers!

Artsiom

[mohammadbakir]

Hi Wilson, since we have not heard back from you we are going to close this request. If you would like to re-open the conversation, please let us know!