# Wandb for Huggingface Trainer saves only first model

**URL:** <https://community.wandb.ai/t/wandb-for-huggingface-trainer-saves-only-first-model/2270>\
**Category:** W&B Help\
**Tags:** dashboard, wandb\
**Created:** [April 20, 2022, 7:18am UTC](https://community.wandb.ai/t/wandb-for-huggingface-trainer-saves-only-first-model/2270 "2022-04-20T07:18:31Z")\
**Posts on this page:** 4\
**Page:** 1

<div class="post-metadata">

**Author:** ![kgarg8](https://avatars.discourse-cdn.com/v4/letter/k/f4b2a3/32.png) [@kgarg8](https://community.wandb.ai/u/kgarg8)\
**Post date:** [April 20, 2022, 7:18am UTC](https://community.wandb.ai/t/wandb-for-huggingface-trainer-saves-only-first-model/2270/1 "2022-04-20T07:18:31Z")

</div>

I am finetuning multiple models using for loop as follows.

```auto
for file in os.listdir(args.data_dir):
    finetune(args, file)

```

BUT `wandb` shows logs only for the first file in `data_dir` although it is training and saving models for other files. It feels very strange behavior.

```auto
wandb: Synced bertweet-base-finetuned-file1: https://wandb.ai/ ***/huggingface/runs/***

```

This is a small snippet of **finetuning** code with Huggingface:

```auto
def finetune(args, file):
    training_args = TrainingArguments(
        output_dir=f'{model_name}-finetuned-{file}',
        overwrite_output_dir=True,
        evaluation_strategy='no',
        num_train_epochs=args.epochs,
        learning_rate=args.lr,
        weight_decay=args.decay,
        per_device_train_batch_size=args.batch_size,
        per_device_eval_batch_size=args.batch_size,
        fp16=True, # mixed-precision training to boost speed
        save_strategy='no',
        seed=args.seed,
        dataloader_num_workers=4,
    )

    trainer = Trainer(
        model=model,
        args=training_args,
        train_dataset=tokenized_dataset['train'],
        eval_dataset=None,
        data_collator=data_collator,
    )
    trainer.train()
    trainer.save_model()

```

---

<div class="post-metadata">

**Author:** ![anmolmann](https://avatars.discourse-cdn.com/v4/letter/a/c37758/32.png) [@anmolmann](https://community.wandb.ai/u/anmolmann)\
**Post date:** [April 20, 2022, 7:56pm UTC](https://community.wandb.ai/t/wandb-for-huggingface-trainer-saves-only-first-model/2270/2 "2022-04-20T19:56:12Z")

</div>

@kgarg8 , you’ve set `save_strategy` to NO in your code to avoid saving anything. This would only save the final model once training is done with `trainer.save_model()` . You can update it to `save_strategy="epoch"` and it will save the model with every epoch.

Or, in order [to log models](https://docs.wandb.ai/guides/integrations/huggingface#turn-on-model-versioning), you could also set the env var `WANDB_LOG_MODEL` as [specified in our docs here](https://docs.wandb.ai/guides/integrations/huggingface#additional-w-and-b-settings). Once you set this env var, any Trainer you initialize from now on will upload models to your W&B project. Note that your model will be saved to W&B Artifacts as `run-{run_name}` .

---

<div class="post-metadata">

**Author:** ![kgarg8](https://avatars.discourse-cdn.com/v4/letter/k/f4b2a3/32.png) [@kgarg8](https://community.wandb.ai/u/kgarg8)\
**Post date:** [April 21, 2022, 2:50pm UTC](https://community.wandb.ai/t/wandb-for-huggingface-trainer-saves-only-first-model/2270/3 "2022-04-21T14:50:51Z")

</div>

`wandb.init(reinit=True)` and `run.finish()` helped me to log the models **separately** on wandb website.

The working code looks like below:

```auto

for file in os.listdir(args.data_dir):
    finetune(args, file)

import wandb
def finetune(args, file):
    run = wandb.init(reinit=True)
    ...
    run.finish()

```

Reference: [Launch Experiments with wandb.init - Documentation](https://docs.wandb.ai/guides/track/launch#how-do-i-launch-multiple-runs-from-one-script)

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/flex020/uploads/wandb/original/1X/366b649231631dbab896843020da0056074ac79d.png) [@system](https://community.wandb.ai/u/system)\
**Post date:** [June 20, 2022, 2:51pm UTC](https://community.wandb.ai/t/wandb-for-huggingface-trainer-saves-only-first-model/2270/4 "2022-06-20T14:51:40Z")

</div>

This topic was automatically closed 60 days after the last reply. New replies are no longer allowed.