Hi, I have weepped hyperparameters over 5k runs and trying to visualize the result using custom chart. I think I set the vega specification correctly, but the results shows only part of runs. It looks the maximum limit of runSets are fixed as 1000, even though I can set it arbitrary over 1000. In the below graph, you can see the line starts with safeguard_gs_prune_topk=3, but I have runs where that hyperparameter of 1 and 2.
And the vega specification I used is:
{
“$schema”: “https://vega.github.io/schema/vega-lite/v5.json”,
“description”: “Number of Gaussian to a metric”,
“data”: {
“name”: “wandb”
},
“params”: [
{“name”: “xrange”, “value”: [33000, 100000]},
{“name”: “yrange”, “value”: [32.0, 33.7]}
],
“title”: “${string:title}”,
“layer”: [
{
“transform”: [
{“filter”: {“field”: “${field:color}”, “equal”: “safeguard_gs”}},
{
“aggregate”: [
{“op”: “average”, “field”: “${field:x-axis}”, “as”: “avg-gauss0”},
{“op”: “average”, “field”: “${field:y-axis}”, “as”: “avg-psnr0”}
],
“groupby”: [“safeguard_gs_purne_topk”, “safeguard_gs_score_function”]
}
],
“mark”: {“type”: “line”, “tooltip”: {“content”: “data”}, “point”: {“size”: 10}},
“encoding”: {
“x”: {
“title”: “${string:xname}”,
“field”: “avg-gauss0”,
“type”: “quantitative”
},
“y”: {
“title”: “${string:yname}”,
“field”: “avg-psnr0”,
“scale”: {
“domain”: {“expr”: “yrange”}
},
“type”: “quantitative”
},
“color”: {
“scale” : {“scheme” : “plasma”},
“title” : “Pruning”,
“field”: “safeguard_gs_score_function”,
“type”: “ordinal”
}
}
}
]
}
Could you let me know the phenomenon is expected or needed to be debugged? I found related post but it doesn’t help to me (Parameter importance chart only limited to 1000 runs).