# Rich reductions when aggregating metrics

**URL:** <https://community.wandb.ai/t/rich-reductions-when-aggregating-metrics/7351>\
**Category:** W&B Help\
**Tags:** dashboard\
**Created:** [July 27, 2024, 10:47am UTC](https://community.wandb.ai/t/rich-reductions-when-aggregating-metrics/7351 "2024-07-27T10:47:15Z")\
**Posts on this page:** 4\
**Page:** 1

<div class="post-metadata">

**Author:** ![tristanengst](https://avatars.discourse-cdn.com/v4/letter/t/b9bd4f/32.png) [@tristanengst](https://community.wandb.ai/u/tristanengst)\
**Post date:** [July 27, 2024, 10:47am UTC](https://community.wandb.ai/t/rich-reductions-when-aggregating-metrics/7351/1 "2024-07-27T10:47:15Z")

</div>

I need to aggregate logged data in a special way, involving sequential means and maxes over different config values.

In my case, I have one WandB pretraining run for each value from a grid of hyperparameters and seeds. Each of these runs generates multiple checkpoints. For each of these checkpoints, I use multiple evaluation WandB runs to evaluate it with a different set of hyperparameters. I have config values for the pretraining checkpoint and pretraining hyperparameter grid logged as part of the config for each evaluation run.

Thus for each element of the pretraining hyperparameter grid, I would like to look at the average over seeds, max over pretraining checkpoints, max over evaluation hyperparameters run. I can do the necessary math in my own code—Pandas is mostly up to the task—but I’d love to be able to do this in the WandB dashboard.

Is there a way I can do this?

Thanks!

---

<div class="post-metadata">

**Author:** ![jason-arkens17](https://avatars.discourse-cdn.com/v4/letter/j/f07891/32.png) [@jason-arkens17](https://community.wandb.ai/u/jason-arkens17)\
**Post date:** [July 31, 2024, 2:45am UTC](https://community.wandb.ai/t/rich-reductions-when-aggregating-metrics/7351/2 "2024-07-31T02:45:48Z")

</div>

Thank you for your question about complex metric aggregation in W&B sweeps. While W&B offers powerful hyperparameter optimization and visualization tools, the specific multi-step aggregation you described (sequential means and maxes over different configuration values) isn’t directly supported in the W&B dashboard or sweeps functionality. However, I can suggest some approaches that may help you achieve your goal.

First, let’s clarify what W&B can do out-of-the-box:

- Log metrics and hyperparameters from your runs
- Visualize and compare runs with grouping, filtering, and basic aggregation
- Optimize sweeps for specific metrics

For more complex analyses like you’ve described, you’ll need to combine W&B’s data logging capabilities with custom analysis. Here’s an approach you could take:

1. Log all relevant metrics and configuration values during your runs using `wandb.log()`.

2. Use the W&B API to retrieve your logged data.

3. Perform your custom aggregations using a library like Pandas.  
Here’s an expanded example of how you might do this:

This script demonstrates the full pipeline of fetching data from W&B, performing your desired aggregations, and even logging the result back to W&B if you wish.

While this requires some additional code, it allows you to leverage both W&B’s robust experiment tracking and the full flexibility of custom analysis.

Some additional suggestions:

1. Consider using W&B Tables to log structured data, which can make retrieval and analysis easier.
2. Explore W&B’s built-in visualization tools like parallel coordinates plots or custom charts for insights that don’t require complex aggregation.
3. For frequently used analyses, you could create a custom script or Jupyter notebook that pulls data from W&B and generates your desired aggregations and visualizations.

I hope this helps provide a path forward! Let me know if you have any questions about implementing this approach or if you’d like to explore other ways to analyze your data within W&B.

---

<div class="post-metadata">

**Author:** ![jason-arkens17](https://avatars.discourse-cdn.com/v4/letter/j/f07891/32.png) [@jason-arkens17](https://community.wandb.ai/u/jason-arkens17)\
**Post date:** [August 1, 2024, 5:05pm UTC](https://community.wandb.ai/t/rich-reductions-when-aggregating-metrics/7351/3 "2024-08-01T17:05:47Z")

</div>

Hi there, I wanted to follow up on this request. Please let us know if we can be of further assistance or if your issue has been resolved.

---

<div class="post-metadata">

**Author:** ![tristanengst](https://avatars.discourse-cdn.com/v4/letter/t/b9bd4f/32.png) [@tristanengst](https://community.wandb.ai/u/tristanengst)\
**Post date:** [August 2, 2024, 4:00am UTC](https://community.wandb.ai/t/rich-reductions-when-aggregating-metrics/7351/4 "2024-08-02T04:00:37Z")

</div>

Thanks! I will look into my own data post-processing.