# Sweep.best\_run() method does not return the best run **URL:** \ **Category:** W&B Help\ **Tags:** sweeps, wandb\ **Created:** [March 20, 2024, 6:09am UTC](https://community.wandb.ai/t/sweep-best-run-method-does-not-return-the-best-run/6141 "2024-03-20T06:09:26Z")\ **Posts on this page:** 4\ **Page:** 1
**Author:** ![shraddha-a](https://avatars.discourse-cdn.com/v4/letter/s/b4bc9f/32.png) [@shraddha-a](https://community.wandb.ai/u/shraddha-a)\ **Post date:** [March 20, 2024, 6:09am UTC](https://community.wandb.ai/t/sweep-best-run-method-does-not-return-the-best-run/6141/1 "2024-03-20T06:09:26Z")
Hi, I’m using Weights & Biases Sweeps to run and track hyperparameter optimization for an XGBoost model. I’ve defined the sweep configuration like so, ```auto sweep_configs = { "method": "random", "metric": {"name": "Mean_RMSE_score", "goal": "minimize"}, "parameters": { 'max_depth': {"values": [4, 9, 14, 19, 29, 34, 44, 49, 54, 59]}, 'learning_rate': {"values": [0.05, 0.0697, 0.0973, 0.1357, 0.1893, 0.2641, 0.3684, 0.5139, 0.7169, 1.0]}, 'objective': {"values": ['reg:squarederror']}, 'reg_alpha': {"values": [0.001, 0.0027, 0.0046,0.0077]}, 'reg_lambda': {"values": [0.0010, 0.0046, 0.0129, 0.0215, 0.0359, 0.0599, 0.1]}, 'min_child_weight': {"values": [0.1802, 0.3246, 0.5848, 1.0536, 3.42, 6.1616, 11.1009, 20.0]}, 'random_state': {"values": [46]}, } } ``` For each iteration in the random search process, I perform K-fold cross-validation and log the mean RMSE score to the run using `wandb.log({"Mean_RMSE_score": score})` I then use `sweep.best_run()` to fetch the run (& hyperparam config) with the lowest mean RMSE score. However, the `best_run()` method does not return the run with the lowest RMSE score. In fact, it is returning a run, that has the 5th lowest RMSE score. Below is a screenshot of some runs in one of my sweeps. ![Screenshot 2024-03-20 at 11.28.39](https://us1.discourse-cdn.com/flex020/uploads/wandb/original/2X/3/356e61a13cc502a91ef4bdeeb0b389ff32d4a4fa.png) As a result, I’m not getting the best set of hyperparameters to build the optimum model. Here is the code where I access the sweep by its sweep ID and use `best_run()` ```auto def get_best_run(sweep_id): api = wandb.Api() sweep = api.sweep(f"{SETTINGS['WANDB_ENTITY']}/{SETTINGS['WANDB_PROJECT']}/{sweep_id}") best_run = sweep.best_run() run_id = best_run.id with init_wandb_run(name = "best_hpo_run", group = "train", job_type = "hpo", run_id = run_id, resume = "must" ) as run: best_config = dict(run.config) return best_config ``` What could be going wrong here? ---
**Author:** ![luis\_bergua](https://avatars.discourse-cdn.com/v4/letter/l/dbc845/32.png) [@luis\_bergua](https://community.wandb.ai/u/luis_bergua)\ **Post date:** [March 22, 2024, 11:59am UTC](https://community.wandb.ai/t/sweep-best-run-method-does-not-return-the-best-run/6141/2 "2024-03-22T11:59:04Z")
Hi @shraddha-a, I just checked with the following code and the run is being returned properly. Would you mind executing it and seeing if it works for you? Also, what is your `wandb --version`? ```auto import wandb import numpy as np import random from sklearn.model_selection import train_test_split from sklearn.metrics import mean_squared_error import xgboost as xgb # Define sweep config sweep_configuration = { "method": "random", "metric": {"name": "Mean_RMSE_score", "goal": "minimize"}, "parameters": { 'max_depth': {"values": [4, 9, 14, 19, 29, 34, 44, 49, 54, 59]}, 'learning_rate': {"values": [0.05, 0.0697, 0.0973, 0.1357, 0.1893, 0.2641, 0.3684, 0.5139, 0.7169, 1.0]}, 'objective': {"values": ['reg:squarederror']}, 'reg_alpha': {"values": [0.001, 0.0027, 0.0046,0.0077]}, 'reg_lambda': {"values": [0.0010, 0.0046, 0.0129, 0.0215, 0.0359, 0.0599, 0.1]}, 'min_child_weight': {"values": [0.1802, 0.3246, 0.5848, 1.0536, 3.42, 6.1616, 11.1009, 20.0]}, 'random_state': {"values": [46]}, } } sweep_id = wandb.sweep(sweep=sweep_configuration, project="sweep_best_run_test") def main(): run = wandb.init() config = run.config # Generate random data X = np.random.rand(1000, 10) y = np.random.rand(1000) # Split the data X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=config.random_state) # Define the model model = xgb.XGBRegressor(max_depth=config.max_depth, learning_rate=config.learning_rate, objective=config.objective, reg_alpha=config.reg_alpha, reg_lambda=config.reg_lambda, min_child_weight=config.min_child_weight, random_state=config.random_state) # Train the model model.fit(X_train, y_train) # Make predictions y_pred = model.predict(X_test) # Calculate RMSE rmse = np.sqrt(mean_squared_error(y_test, y_pred)) # Log the RMSE wandb.log({"Mean_RMSE_score": rmse}) wandb.agent(sweep_id, function=main, count=20) ``` ![image](https://us1.discourse-cdn.com/flex020/uploads/wandb/original/2X/4/42277ea2f45b617e7f47e976d665dbbd3e2e96dd.png) ![image](https://us1.discourse-cdn.com/flex020/uploads/wandb/original/2X/2/25fd131a9239b13beb0db1029885164cc2acf066.png) ---
**Author:** ![luis\_bergua](https://avatars.discourse-cdn.com/v4/letter/l/dbc845/32.png) [@luis\_bergua](https://community.wandb.ai/u/luis_bergua)\ **Post date:** [March 27, 2024, 4:03pm UTC](https://community.wandb.ai/t/sweep-best-run-method-does-not-return-the-best-run/6141/3 "2024-03-27T16:03:49Z")
Hi there, I wanted to follow up on this request. Please let us know if we can be of further assistance or if your issue has been resolved. ---
**Author:** ![shraddha-a](https://avatars.discourse-cdn.com/v4/letter/s/b4bc9f/32.png) [@shraddha-a](https://community.wandb.ai/u/shraddha-a)\ **Post date:** [March 28, 2024, 5:28am UTC](https://community.wandb.ai/t/sweep-best-run-method-does-not-return-the-best-run/6141/4 "2024-03-28T05:28:01Z")
Hi @luis_bergua1 , Thanks for your response. I wrote my own function which was manually fetching all runs in a sweep and picking the best one with the lowest Mean\_RMSE\_score. I’ve not had a chance to look at your solution, but I might try it later. You can close this support ticket. Thanks once again!