Weather4cast 2024 Competition #2 - Event Prediction

We are proud to present the Weather4cast 2024 Competitions.

In this Competition Challenge, you are asked to pre-train a generic model, which is then finetuned to individual Downstream Tasks.

This page here is for Downstream Task #2 – the now-casting prediction of high-precipitation events that start within 4 hours anywhere in the region.

Leaderboard and Submission links are available in the menu on the right.

A detailed description of the training dataset can be found in our forum post.

A detailed description of the submission format and the test dataset can be found in our forum post.

For finetuning your generic model, we provide the W4C24-Task-2 training dataset as zip archive of CSV text files, giving lists of high-precipitation weather events characterized by a range of variables that are discussed below.

Input EUMETSAT satellite band data provide 1h of input for the full context region for which a prediction is to be made. You are then asked to predict 5 events starting in the subsequent 4 hours. (There may be more than 5 reference events. Your predictions will be scored against the best matching references.)

As in Task 1, predictions are evaluated using a Continuous Ranked Probability Score (CRPS) that support (but do not require) probabilistic forecasts.

High-precipitation events are to be characterized by their predicted intensity, duration, start time, location, and size:

Variables to predict

Variables to be predicted in natural log values:

• maxPrec.ln: ln[maximum precipitation, in mm/h/m²]
• duration.ln: ln[min. 45 mins, max 48h, given in time slots], 1 slot = 15 mins
• start.offset.ln: ln[start time offset in slots], 1=first slot
• mid.diag: ln[a measure of the size (not area!) of the event at mid time, in hi-res OPERA rain radar pixels]; mid time is min.time + duration/2.

For these, we calculate the CRPS which reduces to the Absolute Error for point estimates. The error thus is predicted-value minus true-value on the log-scale, giving an unbiased symmetrical error; on the linerar scale this corresponds to a relative error.

For convenience, the training set includes the log values in columns with names ending in .ln, for example maxPrec.ln, duration.ln …

The event start time is offset by the end of the 4 hour prediction window time0 for this purpose, testing log[ (pred.time–time0) / (true.time–time0) ], where this offset adjusted event start time can range from a minimum of slot 1 (+15 minutes) up to a maximum of 16 slots (+4h). Both probabilistic and point predictions are therefore expected on the log-scale, for variable start.offset.ln. We therefore suggest that models are already trained to predict the log of these variables.

Variable to be predicted on a linear scale:

• mid.x, mid.y: coordinates of the center of the event at mid time, in hi-res OPERA rain radar pixel resolution

For these we directly calculate the CRPS which reduces to the Absolute Error for point estimates. We then scale the error by the true size of the event.

Prediction assessment

We finally compute the RMSE from these errors. The final score averages over events, regions, and years.

The code to calculate a score for a single event can be found below:

sqrt(
(
(maxPrec.ln.pred - maxPrec.ln.true)^2 +
(duration.ln.pred - duration.ln.true)^2 +
(start.offset.ln.pred - start.offset.ln.true)^2 +
(mid.diag.ln.pred - mid.diag.ln.true)^2 +
( (mid.x.pred -mid.x.true)/(mid.diag.true) )^2 +
( (mid.y.pred -mid.y.true)/(mid.diag.true) )^2
) / 6
)

which is represented as the following formula:

Test dataset

Input satellite band data representing 1h of input leading times for each test-id time window are now available via sftp link in a following location:

2019/
2019/HRIT/
2019/HRIT/roxi_0008.ev1test19.reflbt0.ns.h5
2019/HRIT/roxi_0009.ev1test19.reflbt0.ns.h5
2019/HRIT/roxi_0010.ev1test19.reflbt0.ns.h5
2020/
2020/HRIT/
2020/HRIT/roxi_0008.ev1test20.reflbt0.ns.h5
2020/HRIT/roxi_0009.ev1test20.reflbt0.ns.h5
2020/HRIT/roxi_0010.ev1test20.reflbt0.ns.h5

Each four subsequent images represent the input data for a specific test-id time window. The order of predictions in the submission file should be the same as the order of the test-id windows in the input test data file.

Sample submission

Submission zip archive files must have the following structure, with each file being in CSV format without headers:

2019/
2019/roxi_0008.test.events1.csv
2019/roxi_0009.test.events1.csv
2019/roxi_0010.test.events1.csv
2020/
2020/roxi_0008.test.events1.csv
2020/roxi_0009.test.events1.csv
2020/roxi_0010.test.events1.csv

Below we provide exemplary submission files:

• non-probabilistic forecasts
• probabilistic forecasts for 17 bins – representing probabilities for each of the predicted values on its own range of possible values