- This topic has 1 reply, 2 voices, and was last updated 1 week, 6 days ago by
.
-
Posts
-
Hello,
I have a question regarding the evaluation procedure used to compute the leaderboard scores on the test dataset.
The competition announcement states that an optimal monotonic transformation is applied before evaluation, implying that participants are not required to predict the exact scale of the target values.
However, based on the leaderboard scores reported for the test dataset, it appears that prediction scale may still have a substantial impact on the reported MSE score, which seems difficult to reconcile with the evaluation description.
Could you please clarify whether the announced optimal monotonic transformation is currently being applied when computing the leaderboard scores on the test dataset? In addition, could you confirm whether this transformation is an isotonic regression (or another form of monotonic calibration), and provide any details that may help participants better understand how the final MSE score is computed?
Thank you.
You will need to tested against our metrics. I.e. you should strive to hit those target values. The comment about monotonic transformation is what has been applied to data set as target vector. Your goal is to emulate it. The metric is grounded in the features derived from the degradation state as found on the images. It is not an equation/formula. The metric is intended to be a straight forward quantitative description of the degradation.
- You must be logged in to reply to this topic.