[tony00c]

Hi organizers — thank you for the dataset. A few grouped clarifications:

Q1. Is the “optimal monotonic transformation” isotonic regression or a parametric monotone fit? And is it non-decreasing only, or either monotone direction (is the sign of our indicator irrelevant)?

Q2. Is the rescale-then-MSE done per experiment/trajectory (each rescaled independently, residuals then pooled/averaged) or globally (one transform across all experiments)? Is each experiment’s ground truth normalized (e.g., to [0,1]) before MSE, and is the final score the mean of per-experiment MSE or pooled?

Q3. What is the submission schema, do we have any required columns/header format? Do we have to submit the health indicator per every run?

Thank you!

[thejbills]

Q1. There is only one metric which quantifies the damage done. It only increases (damage isn’t be removed).

Q2. The scale is not a 0,1. It’s a metric that is consistent across all samples.

Q3. We forgot to link the sample submission to the web page. We’ll provide it in the download repository.

[Author]

Posts