# \[fixed\] Cannot use Sweeps - metric value \[null\]

**URL:** <https://community.wandb.ai/t/fixed-cannot-use-sweeps-metric-value-null/2576>\
**Category:** W&B Help\
**Tags:** sweeps\
**Created:** [June 8, 2022, 5:41pm UTC](https://community.wandb.ai/t/fixed-cannot-use-sweeps-metric-value-null/2576 "2022-06-08T17:41:48Z")\
**Posts on this page:** 3\
**Page:** 1

<div class="post-metadata">

**Author:** ![andrada](https://sea2.discourse-cdn.com/flex020/user_avatar/community.wandb.ai/andrada/32/226_2.png) [@andrada](https://community.wandb.ai/u/andrada)\
**Post date:** [June 8, 2022, 5:41pm UTC](https://community.wandb.ai/t/fixed-cannot-use-sweeps-metric-value-null/2576/1 "2022-06-08T17:41:48Z")

</div>

Hi! 🙂

I am using [this notebook](https://colab.research.google.com/github/wandb/examples/blob/master/colabs/boosting/Using_W%26B_Sweeps_with_XGBoost.ipynb#scrollTo=VCRlDRL6_5aA) as a tutorial to log sweeps into [my Dashboard](https://wandb.ai/andrada/AI4Code?workspace=user-andrada).

Everything works very well besides the Metric, which loggs with the value Null ([see this run for details](https://wandb.ai/andrada/AI4Code/sweeps/6831zeoz?workspace=user-andrada))

 ![image](https://us1.discourse-cdn.com/flex020/uploads/wandb/original/1X/3ed36db8c7cb03fbacf376fc1167163c09034ba2.png)

My process is as follows:

**Create a train function**

```auto
def train_XGBRanker(config_defaults):
    
    # 🐝 W&B Experiment
    config_defaults.update(CONFIG)
    run = wandb.init(project='AI4Code', name='xgbRanker', config=config_defaults)
    config = wandb.config
    
    # Initiate the model
    model = XGBRanker(tree_method = config.tree_method,
                      booster=config.booster,
                      objective=config.objective,
                      random_state=config.random_state, 
                      learning_rate=config.learning_rate,
                      colsample_bytree=config.colsample_bytree, 
                      eta=config.eta, 
                      max_depth=config.max_depth, 
                      n_estimators=config.n_estimators, 
                      subsample=config.subsample,
                      min_child_weight=config.min_child_weight)

    # Train the model
    model.fit(X_train, y_train, group=groups, verbose=True)

    # Create df containing the cell_id and the prediction
    predict = pd.DataFrame({"cell_id" : df_valid["cell_id"],
                            "pred" : model.predict(X_valid)}, index = df_valid.index)

    # Sort (using the predicted rank) and then group
    predict = predict.sort_values(by = ['id', 'pred'], ascending = [False, True])\
                        .groupby('id')['cell_id'].apply(list)

    # Create the same but for actual data
    actual = df_valid.sort_values(by = ['id', 'rank'], ascending = [False, True])\
                            .groupby('id')['cell_id'].apply(list)

    # Kendall Metric
    metric = kendall_tau(actual, predict)
    print(clr.S+"Kendall Tau"+clr.E, metric)
    wandb.log({"kendall_tau": np.float(metric)})

```

**try a first baseline experiment**

```auto
config_defaults = {"tree_method":'hist',
                   "booster":'gbtree',
                   "objective":'rank:pairwise',
                   "random_state":24, 
                   "learning_rate":0.1,
                   "colsample_bytree":0.9, 
                   "eta":0.05, 
                   "max_depth":6, 
                   "n_estimators":110, 
                   "subsample":0.75,
                   "min_child_weight":10}

train_XGBRanker(config_defaults)

```

which returns a `kendall_tau` of 0.5479588742699661 (so the metric isn’t null).

**and then I am running the Sweeps as follows:**

```auto
# Sweep Config
sweep_config = {
    "method": "random", # grid for all
    "metric": {
      "name": "kendall_tau",
      "goal": "maximize"   
    },
    "parameters": {
        "booster": {
            "values": ["gbtree","gblinear"]
        },
        "max_depth": {
            "values": [3, 6, 9, 12]
        },
        "learning_rate": {
            "values": [0.1, 0.05, 0.2]
        },
        "subsample": {
            "values": [1, 0.5, 0.3]
        }
    }
}

# Sweep ID
sweep_id = wandb.sweep(sweep_config, project="AI4Code")

# 🐝 RUN SWEEPS
config_defaults = {"tree_method":'hist',
                   "booster":'gbtree',
                   "objective":'rank:pairwise',
                   "random_state":24, 
                   "learning_rate":0.1,
                   "colsample_bytree":0.9, 
                   "eta":0.05, 
                   "max_depth":6, 
                   "n_estimators":110, 
                   "subsample":0.75,
                   "min_child_weight":10}

# count = the number of trials to run
wandb.agent(sweep_id, train_XGBRanker(config_defaults), count=8)

```

Could you please advise? I don’t know if this is related, but I also can’t run the sweeps for more than a `count=5` as I get the following error:

 ![image](https://us1.discourse-cdn.com/flex020/uploads/wandb/original/1X/88e725be8086de062b815ebe6c25710b0ba23b89.png)

Thank you lots!  
Andrada

UPDATE:

The issue was from the fact that I was having arguments within the `train_XGBRanker()` - moving `config_defaults` from outside the function to in the function an then passing it to `wandb.agent(sweep_id, train_XGBRanker, count=20)` did the job.

---

<div class="post-metadata">

**Author:** ![mohammadbakir](https://avatars.discourse-cdn.com/v4/letter/m/a9a28c/32.png) [@mohammadbakir](https://community.wandb.ai/u/mohammadbakir)\
**Post date:** [June 9, 2022, 7:31pm UTC](https://community.wandb.ai/t/fixed-cannot-use-sweeps-metric-value-null/2576/2 "2022-06-09T19:31:59Z")

</div>

Hi @andrada ,

Thank you for writing in with your question and providing an update. I’m glad you were able to successfully resolve this issue. Please do reach back out again if you run into any other issues.

Regards,

Mohammad

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/flex020/uploads/wandb/original/1X/366b649231631dbab896843020da0056074ac79d.png) [@system](https://community.wandb.ai/u/system)\
**Post date:** [August 8, 2022, 7:32pm UTC](https://community.wandb.ai/t/fixed-cannot-use-sweeps-metric-value-null/2576/3 "2022-08-08T19:32:48Z")

</div>

Closed on August 8, 2022, 7:32pm UTC

This topic was automatically closed 60 days after the last reply. New replies are no longer allowed.