**6. Deterministic versus probabilistic rainfall forecasts, and skill scores**

In this chapter, we have shown that forecasts generated through application of an ANN are deterministic, that is numerical values are produced corresponding to the expected rainfall for a specific month. In contrast, medium-term rainfall forecasts issued to the public by the Australian Bureau of Meteorology are in the form of probabilities relative to the median seasonal rainfall, and do not differentiate between an anticipated rainfall slightly above the median and an extreme rainfall event, such as occurred in Queensland during the period December 2010 and January 2011 [14].

This is illustrated in **Figure 6** showing the seasonal forecast issued by the BOM for the period December 2010 to February 2011 for the entire continent of Australia (Bureau of Meteorology, Archive of rainfall forecasts). Although the forecast indicates that there is a probability in the range of 60–70% above median rainfall for the south-east Queensland region, there was no warning of the magnitude of the impending heavy rainfall. Furthermore, the extensive flooding that affected much of costal Queensland beyond the south-eastern region where the forecast is a 50% probability of above median. This contrasts with the specific quantities forecast for the isohyet maps shown in the previous section.

Not only is the information content of probabilistic forecasts less than corresponding deterministic forecasts, where actual numerical values of predicted rainfall are provided, studies have demonstrated that the general public and specific classes of end-users, such as farmers from south-east Queensland, often have difficulty interpreting the meaning of the probabilistic forecasts [33, 34]. The distinction between median and average was often not understood. In addition, the statement that there was a 30% probability of above the median rainfall was often misinterpreted to mean a forecast of a specific quantity of rainfall 30% higher than the median. There is confusion between a probabilistic and deterministic forecast that can be understood by the public as the expectation of higher rather than lower rainfall as intended by the forecaster.

The Bureau bases its probabilistic forecasts on the simulation of actual physical climatic processes through general circulation models, specifically POAMA [36–38]. For the period from 2002 through until 2011, seasonal rainfall predictions were made using POAMA 1.5.

In July 2011, the Bureau provided us with monthly forecasts for the period to March 2011 for 17 individual sites as simple bilinear interpolations of surrounding grid points which were calculated from the ensemble mean which in turn had been calculated from many runs of this

In **Figures 4** and **5**, it is evident that rainfall varies with topography, and was highest at the places of some altitude in both the relatively dry December of 2005, and also the flood month of December 2010. It is also evident that the ANN correctly predicted December 2010 as expe-

**Figure 5.** Forecast rainfall (mm) for December 2005 for the south-east Queensland region. A: bar chart for individual

In this chapter, we have shown that forecasts generated through application of an ANN are deterministic, that is numerical values are produced corresponding to the expected rainfall for a specific month. In contrast, medium-term rainfall forecasts issued to the public by the Australian Bureau of Meteorology are in the form of probabilities relative to the median seasonal rainfall, and do not differentiate between an anticipated rainfall slightly above the median and an extreme rainfall event, such as occurred in Queensland during the period

riencing relatively higher rainfall across the sub-region, peaking at over 700 mm.

**6. Deterministic versus probabilistic rainfall forecasts, and skill** 

**scores**

December 2010 and January 2011 [14].

sites; and B: isohyet map with 50 mm interval spacing [21].

42 Engineering and Mathematical Topics in Rainfall

**Figure 6.** Seasonal rainfall forecast for Australia by the BoM issued in November 2010. http://www.bom.gov.au/climate/ ahead/archive/rainfall/20101123.national.hrweb.gif [35].

general circulation model, POAMA 1.5. **Figure 7** shows the difference between the observed monthly rainfall, together with forecasts generated by POAMA 1.5 for the site of Miles. The forecasts generated by POAMA show little variation between the 6-year period 2004–2009 and the occurrence of heavy rainfall in 2010: each showing a maximum of approximately 100 mm. In contrast, the forecast generated by the ANN clearly show much higher rainfall of

Forecasting of Medium-term Rainfall Using Artificial Neural Networks: Case Studies…

http://dx.doi.org/10.5772/intechopen.72619

45

Evaluations of forecast skill were also made by comparisons of skill scores calculated relative to climatology from Eq. (1). This is analogous to the method used by the Bureau where a skill

Skill score = [RMSE (climatology) − RMSE (model)/RMSE (climatology)] 100% (2)

When the calculated value of the RMSE from climatology and for a particular model are equal, the forecast skill score will be zero. For a perfect forecast, the RMSE for the model will be zero, and the calculated skill score 100%. Negative values calculated from Eq. (2) indicate a forecast

**Table 4.** Forecast skill as a percentage for monthly rainfall forecasts for Miles with reference to climatology for the

composite ANN model and POAMA.

about 350 mm, closely corresponding to the observed value, **Figure 7** (**Table 3**).

score is calculated for forecasts using POAMA [39].

**Figure 7.** Forecasts for Miles from POAMA 1.5 (yellow line), and the ANN (red line) benchmarked against actual observed rainfall (blue line) for a 3-month lead for the period July 2004 to March 2011.


**Table 3.** Comparison of rainfall forecast skill for Miles using different forecasting methods at leads of 3, 6 and 9 months for the period July 2004 to March 2011.

general circulation model, POAMA 1.5. **Figure 7** shows the difference between the observed monthly rainfall, together with forecasts generated by POAMA 1.5 for the site of Miles. The forecasts generated by POAMA show little variation between the 6-year period 2004–2009 and the occurrence of heavy rainfall in 2010: each showing a maximum of approximately 100 mm. In contrast, the forecast generated by the ANN clearly show much higher rainfall of about 350 mm, closely corresponding to the observed value, **Figure 7** (**Table 3**).

Evaluations of forecast skill were also made by comparisons of skill scores calculated relative to climatology from Eq. (1). This is analogous to the method used by the Bureau where a skill score is calculated for forecasts using POAMA [39].

$$\text{Skill score} = \left[ \text{RMSE (climatory)} - \text{RMSE (model)} / \text{RMSE (climatory)} \right] 100\% \tag{2}$$

When the calculated value of the RMSE from climatology and for a particular model are equal, the forecast skill score will be zero. For a perfect forecast, the RMSE for the model will be zero, and the calculated skill score 100%. Negative values calculated from Eq. (2) indicate a forecast

**Figure 7.** Forecasts for Miles from POAMA 1.5 (yellow line), and the ANN (red line) benchmarked against actual

**Table 3.** Comparison of rainfall forecast skill for Miles using different forecasting methods at leads of 3, 6 and 9 months

for the period July 2004 to March 2011.

observed rainfall (blue line) for a 3-month lead for the period July 2004 to March 2011.

44 Engineering and Mathematical Topics in Rainfall


**Table 4.** Forecast skill as a percentage for monthly rainfall forecasts for Miles with reference to climatology for the composite ANN model and POAMA.

skill score worse than climatology. **Table 4** shows skill scores for each individual month using an ANN with a lead time of 12 months and POAMA with a lead time of 8 months. In all cases the ANN skill scores are positive and lie in the range 23–69%. In contrast, the skills cores for POAMA forecasts are negative, that is worse than climatology.

Hawthorne et al. [39] used the output from POAMA to produce monthly rainfall forecasts with up to 8 months lead time for 250 km × 250 km grid areas over continental Australia. The skill of their monthly rainfall forecasts was described as being generally low. Skill scores fell between −20% and 20%, including grid locations in coastal south-east Queensland. For lead times between 3, 4, 5, 6, 7 and 8 months, with approximately 60% of the forecasts give skill levels relative to climatology below 0%. For southeast Queensland only about 20% of the forecasts had a skill level in the 15–20% range [39] Hawthorne et al. [39] described the skill of their monthly rainfall forecasts as low, and concluded that monthly rainfall forecasting with POAMA remained a challenge.
