Evaluation Metric?

Weather4cast 2023 › Forums › Weather4cast 2023 › Evaluation Metric?

Viewing 4 posts - 1 through 4 (of 4 total)
  • Author
    Posts
  • 26 October 2023 at 02:24 #1207
    ding05

      Hi W4C 2023 Organizer,

      We couldn’t find the specific evaluation metric(s) for W4C 2023. Given that this year’s task is centered on regression, I was wondering if the standard MSE will be the metric, or if there are other regression metrics in place. Could you please clarify how models will be evaluated this year? Thank you, and I look forward to your reply.

      Kind regards,
      Ding.

      26 October 2023 at 08:11 #1208
      Aleksandra Gruca (network admin)
      Participant

        Hi Ding!

        We are just in the process of creating the leaderboard.

        Please be patient; the information about the metric will be available in the next few days and we will post it on the forum.

        Aleksandra

        8 November 2023 at 08:45 #2225
        sunrui

          Hi W4C 2023 Organizer,

          Is the current evaluation metric(s) still MSE, or has it been replaced by other metric(s)?

          Sun

          10 November 2023 at 15:22 #2353
          Aleksandra Gruca (network admin)
          Participant

            Hi Sunrui!

            It has been replaced with the mean Critical Success Index, and all previous leaderboard submissions have been reassessed accordingly.

            Please refer to the following forum post:
            https://weather4cast.net/neurips2023/forums/topic/announcing-final-competition-metric/

          • Author
            Posts
          Viewing 4 posts - 1 through 4 (of 4 total)
          • You must be logged in to reply to this topic.