Evaluation Metric?

[ding05]

Hi W4C 2023 Organizer,

We couldn’t find the specific evaluation metric(s) for W4C 2023. Given that this year’s task is centered on regression, I was wondering if the standard MSE will be the metric, or if there are other regression metrics in place. Could you please clarify how models will be evaluated this year? Thank you, and I look forward to your reply.

Kind regards,
Ding.

[Aleksandra Gruca (network admin)]

Hi Ding!

We are just in the process of creating the leaderboard.

Please be patient; the information about the metric will be available in the next few days and we will post it on the forum.

Aleksandra

[sunrui]

Hi W4C 2023 Organizer,

Is the current evaluation metric(s) still MSE, or has it been replaced by other metric(s)?

Sun

[Aleksandra Gruca (network admin)]

Hi Sunrui!

It has been replaced with the mean Critical Success Index, and all previous leaderboard submissions have been reassessed accordingly.

Please refer to the following forum post:
https://weather4cast.net/neurips2023/forums/topic/announcing-final-competition-metric/

[Author]

Posts