Home>Multi-Model Ensemble>Verification Measures

Verification Measures

This section describes four different verification measures of precipitation forecasts. Since two other commonly used forecast skill measures are very simple, those (RMSE and correlation) are not included here.

ETS and BS

The equitable threat score (ETS) is defined as?

where F is the number of grid boxes that forecast more than the threshold, O the number of grid boxes that observe more than the threshold, and H the number of grid boxes that correctly forecast more than the threshold. CH is the expected number of correct forecasts due to chance (= , where T is the total number of grid boxes inside the verification domain.?

The ETS has shown to be a reasonable estimate for the overall forecast skill. The higher the value, the better the forecast model skill is for that particular threshold.?

The score can vary from a small negative number to 1.0, where 1.0 represents a perfect forecast. This simply is the ratio of the correct forecast area to the total area of the forecast and observed precipitation. The model gets penalized for forecasting rain in the wrong place as well as not forecasting rain in the right place. Thus, the model with the highest score is generally the model with the best forecast skill.?

The bias score (BS) is a very simple equation, defined as simply as F=O. This score does not comment at all on the skill of a model forecast in terms of the placement of precipitation, but does give an indication if a model is consistently over-or under-forecasting areas of precipitation. The best model is generally the one that remains near the 1.0 line, which means that the model does not generally over-forecast precipitation or under-forecast precipitation. If the model verifies over 1.0, it is over-predicting precipitation, and if below 1.0 it is under-predicting precipitation.?