Atmospheric and Climate Sciences, 2012, 2, 479-491 http://dx.doi.org/10.4236/acs.2012.24042 Published Online October 2012 (http://www.SciRP.org/journal/acs)

Monthly Forecast of Indian Southwest Monsoon Rainfall Based on NCEP’s Coupled Forecast System

Dushmanta R. Pattanaik1*, Biswajit Mukhopadhyay1, Arun Kumar2 1India Meteorological Department (IMD), New Delhi, India

2Climate Prediction Center, National Centres for Environmental Prediction (NCEP)/National Oceanic and Atmospheric Administration (NOAA), Camp Springs, USA

Email: *pattanaik_dr@yahoo.co.in

Received July 27, 2012; revised August 29, 2012; accepted September 10, 2012

ABSTRACT

The monthly forecast of Indian monsoon rainfall during June to September is investigated by using the hindcast data sets of the National Centre for Environmental Prediction (NCEP)’s operational coupled model (known as the Climate Forecast System) for 25 years from 1981 to 2005 with 15 ensemble members each. The ensemble mean monthly rainfall over land region of India from CFS with one month lead forecast is underestimated during June to September. With respect to the inter-annual variability of monthly rainfall it is seen that the only significant correlation coefficients (CCs) are found to be for June forecast with May initial condition and September rainfall with August initial conditions. The CFS has got lowest skill for the month of August followed by that of July. Considering the lower skill of monthly fore- cast based on the ensemble mean, all 15 ensemble members are used separately for the preparation of probability fore- cast and different probability scores like Brier Score (BS), Brier Skill Score (BSS), Accuracy, Probability of Detection (POD), False Alarm Ratio (FAR), Threat Score (TS) and Heidke Skill Score (HSS) for all the three categories of fore- casts (above normal, below normal and normal) have been calculated. In terms of the BS and BSS the skill of the monthly probability forecast in all the three categories are better than the climatology forecasts with positive BSS values except in case of normal forecast of June and July. The “TS”, “HSS” and other scores also provide useful probability forecast in case of CFS except the normal category of July forecast. Thus, it is seen that the monthly probability forecast based on NCEP CFS coupled model during the southwest monsoon season is very encouraging and is found to be very useful. Keywords: Indian Monsoon; Coupled Model; Monthly Forecast; Probability Forecast; Brier Skill Score; Threat Score;

Heidke Skill Score

1. Introduction

In addition to the seasonal total rainfall during the south- west monsoon season from June to September the sub- seasonal (monthly) variability of Indian monsoon rainfall is also a major factor, which influences the Agricultural outputs of the country. Thus, the monthly forecast during the southwest monsoon from June to September is very essential for the planner, policy maker and also to various other users. The medium-range forecasting (up to 7 days in tropics) is an atmospheric initial value problem, where the Sea surface temperature (SST) anomaly is generally persisted beyond its initial value. Seasonal forecasting, on the other hand, relies on the slow evolu- tion of boundary conditions, like SSTs and soil moisture. The monthly forecast is at the interface between the medium-range weather forecasting and the seasonal fore- casting and also fills the gap between these two time

scales. However, the monthly forecasting is often consi- dered a difficult time range for skillful forecasts, since the forecast lead-time scale is sufficiently long so that much of the memory of the atmospheric initial conditio- ns is lost, and the time-averaging is too short such that the signal due to the influence of SST is small compared to the atmospheric noise.

The intraseasonal variation (variability of monsoon with in season) of Indian summer monsoon precipitation shows clear association with northward propagation of large-scale convective anomalies from the equator as shown by Sikka and Gadgil [1] and Pattanaik [2]. This northward propagation is known to be accompanied by eastward propagation of convective activity along the equator (Madden-Julian Oscillation; MJO) through the Rossby wave propagation. An important source of predictability on the monthly time-scale is thus, argued to be from the modes of tropical intra-seasonal variability, the MJO, which is characterized by organization on a *Corresponding author.

Copyright © 2012 SciRes. ACS

D. R. PATTANAIK ET AL. 480

global spatial scale with a period typically ranging from 30 - 60 days [3-6]. Some observational studies of the northward propagating convective bands associated with fluctuation of intra-seasonal monsoon activity have been demonstrated by many studies [1,2,7]. An accurate coupling of the fast atmosphere to the slow ocean (with long memory) is essential to simulate the MJO, which in turn can simulate the intra-seasonal variability of Indian monsoon.

The monthly forecast using dynamical model was triggered by the result of the study [8], where they showed how the pronounced blocking event of 1977 was successfully reproduced in 1-month forecasts by some general circulation models. Some of the recent studies have highlighted that the coupled models with one-tier approach can enhance the predictability of the summer monsoon precipitation [9,10]. As shown by Krishnan et al. [10], a fully coupled model will be able to better capture the observed monsoon inter-annual variability. Thus, the future climate prediction system should focus with coupled atmosphere-ocean models particularly for the extended range prediction covering the monthly forecast, which require a better representation of air-sea interaction and the coupled atmospheric-ocean phenome- non like MJO in the model.

The leading modeling centres like the European Centre for Medium range Weather Forecasting (ECMWF) and the National Centre for Environmental Prediction (NCEP) have also introduced General Circulation Model and coupled atmosphere-ocean models operationally for monthly forecast of atmospheric and oceanic compone- nts (see Ferranti et al. [11] & Vitart [12] for ECMWF and Saha et al. [13] for NCEP). As discussed by Vitart [12] the ECMWF has a dedicated monthly forecasting system, which was based on 32-day coupled ocean-atm- osphere integrations run routinely since March 2002. Though the model is integrated for 32 days the forecast is prepared on weekly basis valid for days 5 - 11, days 12 - 18, days 19 - 25 and days 26 - 32. As shown by him the model displays some skill in predicting weekly averaged 2-m temperature, precipitation, and mean sea level pressure anomalies relative to the climate of the past 12 years. The NCEP coupled modeling system (Known as the Climate Forecast System (CFS)) on the other hand is used for both monthly as well as the seasonal forecast. The skill of the NCEP’s CFS coupled modeling system for the seasonal rainfall over India during June to September as shown by Pattanaik and Kumar [14] and Pattanaik et al. [15] show some useful skill. With the availability of long hindcast data from various centres using GCMs and coupled GCMs, a number of studies [16-23] have been carried out to see the performance of different models for the monsoon prediction over India in the extended range time scale. Most of these studies have

focused on the simulation of seasonal monsoon rainfall over India. In the present study the skill of the NCEP CFS forecast system is assessed with respect to the monthly forecast solely by the use of a tier-1 retrospec- tive set of forecasts for 25 years from 1981 to 2005 for the Indian monsoon rainfall from June to September. The ensemble members are also considered separately to prepare the monthly probability forecast during the period from 1981 to 2005 and the skill of the probability forecast is also investigated.

2. Details of the Model Hindcast and the Methodology

The atmospheric component of the CFS is the NCEP atmospheric GFS model, as of February 2003 [24]. The atmospheric component is having a spectral triangular truncation of 62 waves (T62) in the horizontal and a finite differencing in the vertical with 64 sigma layers. This version of the GFS has been modified from the version of the NCEP model used for the NCEP/NCAR Reanalysis by Kalnay et al. [25] and Kistler et al. [26], with upgrades in the parameterization of solar radiation transfer [27,28], boundary layer vertical diffusion [29], cumulus convection [30], gravity wave drag [31]. The oceanic component is the GFDL Modular Ocean Model V.3 (MOM3), which is a finite difference version of the ocean primitive equations under the assumptions of Bou- ssinesq and hydrostatic approximations [32]. It uses spherical coordinates in the horizontal with a staggered Arakawa B grid and the z-coordinate in the vertical. The ocean surface boundary is computed as an explicit free surface. The domain is quasi-global extending from 74˚S to 64˚N. The zonal resolution is 1˚. The meridional reso- lution is 1/3˚ between 10˚S and 10˚N, gradually incre- asing through the tropics until becoming fixed at 1˚ poleward of 30˚S and 30˚N. There are 40 layers in the vertical with 27 layers in the upper 400 m, and the bottom depth is around 4.5 Km. Vertical mixing follows the non-local K-profile parameterization of Large et al. [33]. The horizontal mixing of momentum uses the nonlinear scheme of Smagorinsky [34]. The ocean- atmosphere coupling is nearly global (64˚N - 74˚S), instead of only in the tropical Pacific Ocean, and flux correction is no longer applied. Thus, the CFS is a fully “tier-1” forecast system. The coupling over the global ocean required an important upgrade in the ocean data assimilation as well. An extensive set of retrospective forecasts (“hindcasts”) was generated to cover a 25 years period (1981-2005), in order to obtain a history of the model. This history can be used operationally to calibrate and assess the skill of the real-time forecasts.

The CFS includes a comprehensive set of retrospective runs that are used to calibrate and evaluate the skill of its forecasts. Each run is a full nine month integration with

Copyright © 2012 SciRes. ACS

D. R. PATTANAIK ET AL. 481

15 initial conditions that span each month. Each month was divided into 3 segments centered on the pentad ocean initial conditions of 11th of the month, 21st of the month and the first day of next month. The atmospheric initial states of 9th, 10th, 11th, 12th and 13th of the month used the same pentad ocean initial conditions of 11th. Similarly, the atmospheric states of 19th to 23rd used the same pentad ocean initial condition of 21st and the remaining five atmospheric states (last two days of the month and first three days of the next month) used the same pentad ocean initial conditions of first day of next month. Thus, these 15 initial conditions were carefully selected to span the evolution of both the atmosphere and ocean in a continuous fashion. In the present analysis the hindcast analysis obtained with 15 initial conditions of the months for the forecasting of monthly and seasonal monsoon rainfall during June to September. CFS forecast for the simulation of Indian monsoon rainfall during June to September. The atmospheric initial conditions were from the NCEP/DOE Atmospheric Model Inter-compa- rison Project (AMIP) II Reanalysis (R2) data & the ocean initial conditions were from the NCEP Global Ocean Data Assimilation (GODAS), which was made opera- tional at NCEP in September 2003 [35].

The skill of monthly forecast rainfall for country as a whole during June to September from NCEP CFS hind- cast with one month lead is evaluated, initially by consi- dering the ensembles mean as the deterministic forecast. Subsequently, the 15 ensemble members are also used to obtain the probability forecast in the three categories (above normal, normal and below normal) by calculating different verification scores used for the verification of probability forecast. The distribution of observed cate- gories of above normal, below normal and normal rain- fall over India in each month from June to September is determined by using the observed monthly rainfall series of India available from India Meteorological Department (http://www.imd.gov.in). For the quantitative verification purpose the observed monthly rainfall series of IMD based on observational network over Indian land region is used (hereafter referred as IMD rainfall). However, since the numerical model also gives rainfall over the Ocean region the verification of rainfall forecast from the model require similar rainfall distribution including the land and ocean region. Thus, for the eye ball verification purpose the rainfall analysis obtained from the global monthly precipitation using gauge observations, satellite estimates and numerical model outputs is used from Xie and Arkin, [36]; hereafter known as Xie-Arkin rainfall.

3. Skill of Monthly Rainfall Forecast from CFS during June to September

3.1. Simulation of Mean Monsoon Rainfall

The CFS hindcast climatology of monsoon rainfall is

prepared by using one month lead forecast valid for June, July, August and September. The model climatology is represented here by retrospective forecasts (or “model simulations”), made with a 15-member ensemble, over the 25-year period from 1981 to 2005. Therefore, for each new forecast, there is a reference set of 375 (15 25) simulations. The Xie-Arkin rainfall climatology on monthly scale from June to September during the period from 1981 to 2005 along with the corresponding CFS forecast climatology with lag-1 is shown in Figure 1 for the eye ball verification purpose. It is seen from Figure 1 that the monthly forecast climatology of rainfall from CFS forecast (Figures 1 (e)-(h)) during June to Sept- ember clearly shows two rainfall maxima (one over the Bay of Bengal and other over the west coast region) as in the observation (Figures 1(a)-(d)). Thus, the two maxima are well captured in the model, particularly during June to August, although the west coast maximum is stretched and extends westward into the Arabian Sea with slight overestimation in the CFS forecast. It is also seen that the west coast maximum is over estimated in the CFS forecast over the Arabian Sea region during all four months. During the onset phase of monsoon (June) the forecast climatology from CFS (Figure 1(e)) matches well with the corresponding observed climatology (Fig- ure 1(a)), although the rainfall over northeast India and west coast of India is slightly overestimated in the CFS forecast.

During the peak monsoon months of July and August the rainfall over the central parts of India is underesti- mated in the CFS forecast (Figures 1(f) and (g) respec- tively) compared to the corresponding observed climato- logy (Figures 1(b) and (c)). During the withdrawal phase of monsoon the CFS forecast indicates overestimation of west coast rainfall (Figure 1(h)) compared to that in observation (Figure 1(d)). The zone of less rainfall over the northwest India and the rain shadow region of Tamilnadu is also well captured in the CFS climatology (Figures 1(e)-(h)).

Thus, the CFS forecast simulates excessive rainfall over the northeastern parts of the country stretching westward along Nepal, Gangetic and Brahmaputra valley stretching from the Bay of Bengal region for all the four months.

During the peak monsoon months of July and August the rainfall over the central parts of India is underes- timated in the CFS forecast (Figures 1(f) and (g) respe- ctively) compared to the corresponding observed clima- tology (Figures 1(b) and (c)). During the withdrawal phase of monsoon (September) the CFS forecast indicates overestimation of west coast rainfall (Figure 1(h)) compared to that in observation (Figure 1(d)). The zone of less rainfall over the northwest India and the rain shadow region of Tamilnadu is also well captured in the CFS climatology (Figures 1(e)-(h)). Thus, the CFS fore-

Copyright © 2012 SciRes. ACS

D. R. PATTANAIK ET AL.

Copyright © 2012 SciRes. ACS

482

Figure 1. (a) Climatological observed June rainfall (mm/day) for 25 years period from 1981 to 2005 with more than 7 mm/day is shaded and (e) the corresponding CFS forecast climatology for June with lag-1 (May initial conditions here). The corre- sponding figures are for July (b, f) to September (d, h).

D. R. PATTANAIK ET AL. 483

cast simulates excessive rainfall over the northeastern parts of the country stretching westward along Nepal, Gangetic and Brahmaputra valley stretching from the Bay of Bengal region for all the four months.

In order to quantify the monthly mean rainfall over land region of India the CFS forecast rainfall over land only region of India along with the mean rainfall from IMD during different phases of monsoon is given in Table 1. As seen from Table 1, although the rainfall over west coast of India along with adjoining Arabian Sea and the Bay of Bengal regions overestimate in the CFS forecast the rainfall over the land only region of India is slightly underestimated in the CFS for all the four months from June to September. It is also seen from Table 1 that the coefficient of variability (CV) in case of CFS forecast is significantly less for the peak monsoon months of July and August with CV of 5.1% and 7.5% respectively compared to its corresponding observed CV of 13.9% and 15.3% during July and August respectively. In case of CFS the CV is computed based on the ensemble mean, and thus, this behavior is expected.

3.2. Simulation of Inter-Annual Variability of Monthly Rainfall

In order to see the inter-annual variability of monthly rainfall from June to September the land only rainfall from CFS forecast (with lag-1) along with the correspon- ding rainfall departure from IMD observation is shown in Figure 2. It is seen from Figure 2, and also from Table 1, that the monthly forecast in the form of ensemble mean in CFS has got lowest skill for the month of August with lag-1 (July ensembles). Similarly the July has also got very less skill with lag-1 (June ensembles). The only significant CCs during all the 4 months are found to be for the June rainfall with lag-1 forecast (ensembles of May) and September with lag-1 forecasts (ensembles of August). Thus, the analysis indicates that the monthly forecasts with the current CFS although shows some encouraging results the skill is not significant for all the 4 months. During July 2002 there was unprecedented deficiency in rainfall over India (Figure 2(b)). However, although the CFS forecasts captured the negative departure the anomalies are underestimated. Since the month of July has got the lowest CV (Table 1) in the

CFS forecast the variance inflated/deflated forecast could be useful in improving the anomalies on individual occasions. As shown earlier by Pattanaik et al. [15], like the lower skill of monthly monsoon rainfall in CFS the skill of inter-annual variability of seasonal rainfall during JJAS is also found to be having lower skill with April ensembles having slightly higher CC (0.44) compared to that of March (0.24) and May ensembles (0.30) during the same 25 years of hindcast period from 1981 to 2005.

4. Monthly Probability Forecast Based on CFS Ensembles

As it is seen earlier the raw skill of CFS for monthly forecast is not highly encouraging, with CCs not signifi- cant even at 95% level for the peak monsoon months of July and August. Thus, the peak monsoon months of July and August are having the lowest skill. Although the deterministic forecast in the form of ensemble mean (when there is large number of ensemble members) pro- vides on average better skill than an individual forecast [37], but it represents just a part of the information con- tained in the ensemble. Palmer [38] has shown that the use of an ensemble mean forecasts, generated from adja- cent start dates, also appeared to perform very close to a climatological forecast, thus showing almost poorer skill for the prediction of seasonal anomalies. Thus, there is a need to have the probability forecast. Another useful in-formation lie with ensemble members is the spread among the members. But a simple relationship between skill and spread has not been found. Taking advantage of a multi-ensemble member framework it will be useful to use the same in the probability formulation, where the ensemble may give information on the possible outcomes. Such information will be better as the spread increases. As emphasized by Palmer et al. [39] the chaotic nature of forecasts associated with the spread of the ensembles requiring the need for a forecast in probabilistic sense.

In view of the forecast uncertainty of deterministic forecasts, there is also a need to see the skill of CFS in case of probability forecast. The basic principle of prob- ability forecast is the different ensemble members are associated with slightly different initial conditions. The ensemble members are having spread from one member

Table 1. Mean, Correlation Coefficients (CC) and the Coefficient of Variability (CV).

Rainfall over Indian land only region IMD’s rainfall Mean (CV in %) CFS’s hindcast rainfall Mean (CV in %) CC

June 5.18 mm (23.7%) 4.36 mm (19.2%) 0.5199

July 9.49 mm (13.9%) 7.02 mm (5.10%) 0.32

August 8.36 mm (15.3%) 6.07 mm (7.50%) 0.31

September 5.73 mm (23.6%) 4.17 mm (12.7%) 0.4698

CC is between IMD monthly rainfall and model hindcast rainfall during 25 years period (1981 to 2005) for June to September on monthly scale with lag-1 initial conditions. The significance level of CCs are indicated as superscript.

Copyright © 2012 SciRes. ACS

D. R. PATTANAIK ET AL. 484

-40

-30

-20

-10

0

10

20

30

40

1981

1982

1983

1984

1985

1986

1987

1988

1989

1990

1991

1992

1993

1994

1995

1996

1997

1998

1999

2000

2001

2002

year

% D

ep o

f m

onth

ly rai

nfa

ll

2003

2004

2005

CFS Jun RF (May, CC=0.51)IMD Jun RF

(a)

(a)

-60

-50

-40

-30

-20

-10

0

10

20

30

40

1981

1982

1983

1984

1985

1986

1987

1988

1989

1990

1991

1992

1993

1994

1995

1996

1997

1998

1999

2000

2001

2002

2003

2004

2005

year

% D

ep o

f m

onth

ly rai

nfa

ll

CFS Jul RF (Jun, CC=0.32)

IMD Jul RF(b)

(b)

-30

-20

-10

0

10

20

30

1981

1982

1983

1984

1985

1986

1987

1988

1989

1990

1991

1992

1993

1994

1995

1996

1997

1998

1999

2000

2001

2002

year

% D

ep o

f m

onth

ly rai

nfa

ll

2003

2004

2005

CFS Aug RF (Jul, CC=0.31)

IMD Aug RF

(c)

(c)

-50

-40

-30

-20

-10

0

10

20

30

40

50

1981

1982

1983

1984

1985

1986

1987

1988

1989

1990

1991

1992

1993

1994

1995

1996

1997

1998

1999

2000

2001

2002

year

% D

ep o

f m

onth

ly rai

nfa

ll

2003

2004

2005

CFS Sep RF (Aug,CC=0.46)IMD Sep RF

(d)

(d)

Figure 2. (a)Year-to-year variation of % departure of observed June rainfall over India along with that of rainfall from CFS forecast with one month lead for the month of June over Indian land mass (May ICs); (b), (c) and (d) for observed and CFS rainfall from July to September respectively.

Copyright © 2012 SciRes. ACS

D. R. PATTANAIK ET AL.

ACS

485

to other. Thus, if the ensemble members have large stan-dard deviation, which indicates it has large spread or the ensemble members deviates from one another (good for the probability forecast). The spread of a variable indi-cates the forecast diversity of the ensemble mean among all the members. Small spread indicates low forecast un-certainty, large spread high forecast uncerta- inty. Large spread should not be taken as a reason not to issue a forecast. In that case the best strategy will be to issue a forecast based on the ensemble mean but to be careful in the formulations and try to indicate possible alternatives. The spread will also indicate what is not likely to happen, which at times might be as important as knowing what is likely to happen. In order to see the spread of ensemble members, the 15 individual member monthly forecasts rainfall averaged over land only region of India with lag-1 ensemble valid for June to September is shown in Figure 3. Along with the ensemble members the ensem-ble spread in the form of Standard Deviation (SD) during each monthly forecast is also plotted in Figure 3 (in secondary Y-axis). Thus, the spread is calculated as SD with respect to ensemble mean for each monthly forecast

separately. As it is seen from Figure 3(a) the forecast for June rainfall during the period from 1981 to 2005 shows large spread of ensemble members in most of the years with mean SD is found to be 1.15 mm/day (Figure 3(a)). However, the month of July and August have got the lower spread among the members, which indicates the members are closer to the ensemble mean. The mean SD for July and August during the 25 years period as seen from Figures 3(b) and (c) is found to be 0.86 mm/day and 0.90 mm/day respectively. For the month of Sep-tember as seen from Figure 3(d) it is seen that the en-semble spread is slightly higher than that during July and August with mean SD during the 25 years period (1981-2005) is found to be 0.98 mm/day. The spread of the ensemble members in case of monthly forecast for all 4 months are the basis of preparing monthly probability forecast.

There are different methods of generating probability forecasts. Based on the ensemble members the probabil- ity of above normal, below normal or normal may be calculated at each grid point by using the climatological information of CFS hindcast over the region for 25 years

(a) June (May IC) (b) July (Jun IC)

(c) August (Jul IC) (d) September (August IC)

Figure 3. (a) Maximum, minimum and mean values of the 15 ensemble members of the forecast for June along with the en-semble spread (in terms of standard deviation) during the period from 1981 to 2005; (b) Same as “a” but for July; (c) Same as “a” but for August; (d) Same as “a” but for September.

Copyright © 2012 SciRes.

D. R. PATTANAIK ET AL. 486

(1981-2005). In case of hindcast data of CFS the 15 days of initial conditions are consisting of date 9th to 13th, 19th to 23rd and the last five days of the month. Since the operational CFS T62L64/MOM3—is initialized 4 times daily from 00Z, 06Z, 12Z and 18Z (with lag-1) 4 ensemble members are available for each day of atmos- pheric initial condition. Hence by choosing 15 days of atmospheric initial conditions of current month a total of 60 ensemble members are used to generate the real time probability forecast for subsequent months (schematic diagram is shown in Figure 4). Thus, these 15 initial con- ditions were carefully selected not only to span the evo- lution of both the atmosphere and ocean in a continuous fashion but also to keep the symmetry with the hindcast run of 25 years used for the preparation of model clima- tology as discussed in Section 2. As shown in Figure 4 the hindcast climatology of 25 year model run is used for categorizing into three categories viz., the above normal (exceeding upper tercile), the below normal (below lower tercile) and the normal category (forecasts between lower and upper terciles). The upper and lower tercile values of observed monthly rainfall are also calculated based on the distribution of observed monthly rainfall over India during June to September, which is used to categorize the month in a given year as either above normal (PAB), below normal (PBL) or normal (PNN) monsoon rainfall month. Verification of the three categories of the monthly probability forecast of rainfall over India with one month lead from NCEP CFS coupled model during 1981 to 2005 valid for June to September is given in Ta- bles 2 to 5 respectively.

5. Verification of Probability Forecast

A major difficulty with a probabilistic forecast is to evaluate its actual skill as discussed in Murphy [40]. The verification of probability forecast is determined in terms of Brier Score (BS) and Brier Skill Score (BSS). Krish- namurti et al. [41] and many other studies have used these scores for the verification of forecast of seasonal monsoon rainfall As suggested by Murphy [40] there factors are need to be considered when verifying a fore-

cast viz., the Consistency (forecasts agree with fore- caster’s true belief about the future weather), Quality (Good correspondence between observations and fore- casts-verification) and Value (increase or decrease in economic or other kind of value to someone as a result of using the forecast-decision theory). Different scores are calculated and analysed for the verification of probability forecast of monthly rainfall from NCEP CFS during the period from 1981 to 2005.

5.1. Brier Score (BS)

The Brier score is a proper score function that measures the accuracy of a set of probability assessments. It was proposed by Brier [42]. It measures the average squared deviation between predicted probabilities for a set of events and their outcomes, so a lower score represents higher accuracy. Nowadays, the most common formula- tion of the Brier score is

2

1

1 N

t tt

BS f ON

in which ft is the probability that was forecast, Ot the ac- tual outcome of the event at instance (0 if it doesn’t hap- pen and 1 if it happens) and N is the number of fore- casting instances. This formulation is mostly used for binary events (for example “rain” or “no rain”; “above normal” or “no above normal”). Brier score is analogous to a Mean Square Error (MSE), but it is negatively ori- ented, with perfect forecasts exhibiting “BS” = 0. Less accurate forecasts receive higher Brier scores, but since individual forecasts and observations are bounded by zero and one, the score can take values only in the range 0 ≤ BS ≤ 1. The best score achievable for BS is “0” and the worst score achievable is “1”. There will be different BSs for different category of probability forecasts (like above normal, below normal and normal).

5.2. Brier Skill Score (BSS)

The Brier skill score is in the usual skill score format, (score for the forecast—score for the standard forecast)/

Probability forecast (in %) for month 1 onwards

Above Normal (Exceeding upper tercile) Below Normal (Below lower tercile)

Normal (Between lower and upper terciles)

Total 60 forecast ensemble members valid for Month 1 onwards

4 Ensembles/day with 15 days of the Month (0)

Corresponding CFS hindcast climatology from 25 years (1981-2005)

for Month 1 onwards

Figure 4. Schematic diagram shows how the probability forecast is generated from the CFS forecast.

Copyright © 2012 SciRes. ACS

D. R. PATTANAIK ET AL. 487

Table 2. Forecast verifications scores for June.

Scores Above Normal Below Normal Normal

Accuracy 0.64 0.80 0.60

POD 0.70 0.80 0.20

FAR 0.46 0.50 0.50

CSI (TS) 0.44 0.44 0.17

HSS 0.29 0.49 0.07

Table 3. Forecast verifications scores for July.

Scores Above Normal Below Normal Normal

Accuracy 0.76 0.60 0.52

POD 0.50 0.58 0.14

FAR 0.50 0.42 0.86

CSI (TS) 0.33 0.41 0.08

HSS 0.34 0.20 –0.19

Table 4. Forecast verifications scores for August.

Scores Above Normal Below Normal Normal

Accuracy 0.72 0.56 0.60

POD 0.12 0.57 0.60

FAR 0.00 0.67 0.50

CSI (TS) 0.12 0.27 0.38

HSS 0.16 0.10 0.19

Table 5. Forecast verifications scores for September.

Scores Above Normal Below Normal Normal

Accuracy 0.64 0.68 0.76

POD 0.60 0.58 0.50

FAR 0.70 0.30 0.33

CSI (TS) 0.25 0.47 0.40

HSS 0.18 0.35 0.41

(perfect score—score for the standard forecast). In this sense, it measures the difference between the score for the forecast and the score for the unskilled standard forecast, normalized by the total possible improvement that can be achieved. Skill scores have a range of – to 1. Negative values indicate that the forecast is less accurate than the standard forecast. “Standard” forecasts can be any unskilled forecast; the two most often used are cli- matology and persistence. Climatology is most often used as the standard. Since the perfect Brier score is 0, the BSS can be written as,

1 refBSS BS BS

where the BSref is calculated by using the forecast prob- ability based on the long term climatology, estimated from each station’s climatological records. The BS and BSS for monthly forecast of rainfall over India with one month lead time and valid for June, July, August and September are calculated from CFS handcast during the period from 1981 to 2005 as shown in Figure 5. As seen from Figure 5(a) the BS is found to be between 0.16 to 0.26, with lower positive value indicating relatively bet- ter forecast compared to higher value. The month-wise BS indicate best forecast for below normal category in June, above normal category in July, both below and above normal categories in August and finally above normal category in September. The BSS on the other hand is mainly positive for all cases except in the normal category of June and July (Figure 5(b)). The highest value of BSS is 0.12 is in case of below normal category of June forecast. Though the BSS values are small the positive values indicate the forecast skill is better than climatology forecast in most of the cases (except the normal category of June and July forecast). Palmer et al [39] have indicated that it is not fixed what will be the perfect-model average BSS. As pointed out by them the theoretical maximum value of Brier Skill Score (which is actually one) is not a reasonable upper bound that can be achieved in principal since inevitable uncertainties of the initial conditions lead to chaotic variability within the ensemble even with a perfect model. Though the skill of the monthly probability forecast is better than the clima- tology in most of the cases (positive values in Figure 5(b)) the negative BSS in case of normal forecast of June and July indicate poor forecast compared to climatology forecast.

5.3. Other Verification Scores

The probability forecast can also be verified by consid- ering it as a dichotomous (yes/no) forecasts. To verify this type of forecast we start with a contingency table that shows the frequency of “yes” and “no” forecasts and occurrences. The four combinations of forecasts (yes or no) and observations (yes or no), called the joint distri- bution, are:

hit (H)—event forecast to occur, and did occur miss(M)—event forecast not to occur, but did occur false alarm (F)—event forecast to occur, but did not

occur correct negative (CN)—event forecast not to occur,

and did not occur and the total number (N) = (hits + misses + false alarm +

correct negative) Three contingency tables are prepared for each cate-

gory of forecast (PAB, PBL and PNN) for each month

Copyright © 2012 SciRes. ACS

D. R. PATTANAIK ET AL. 488

0.23

0.16

0.26

0.18

0.21

0.26

0.21

0.24

0.21

0.10

0.12

0.14

0.16

0.18

0.20

0.22

0.24

0.26

0.28

June July August

Monthly Forecast (Lag -1 forecast)

Mo

nth

ly F

ore

ca

st

Bri

er

Sc

ore

(B

S

0.16

0.26

0.21

Septemper

)

BS (PAB)

BS (PBL)

BS(PNN)

(a)

0.12

0.07

0.04

0.06

0.04

0.00

0.03

-0.04

-0.06

-0.10

-0.05

0.00

0.05

0.10

0.15

June July August

Monthly Forecast (Lag -1 forecast)

Mo

nth

ly F

ore

ca

st

Bri

er

Sk

ill S

co

re (

BS

S)

0.08

0.06

0.04

Septemper

BSS (PAB)

BSS (PBL)

BSS (PNN)

(b)

Figure 5. (a) Brier Score and (b) Brier Skill Score of the monthly rainfall forecast over the land region of India with one month lag based on CFS hindcast during the period from 1981-2005. The scores are computed for all the three categories of above normal, below normal and normal. from June to September by comparing the forecast of monthly rainfall averaged for the country as a whole from NCEP CFS with corresponding observed monthly rainfall over India as a whole. For the quantitative veri- fication many verification scores can be calculated using the contingency table such as Accuracy, Probability of Detection (POD), False Alarm Ration (FAR), Critical Success Index (CSI) or commonly known as Threat Score (TS), Heidke Skill Score (HSS) etc. The Relative Operating Characteristics (ROC) also used for the verifi- cation of probability forecast by Kharin and Zwiers [43]. The ROC is a representation of the skill of a forecasting system in which the hit rate and the false-alarm rate are compared [44,45]. The different verification scores used in the present study are discussed below:

1) Accuracy (fraction correct) = (H + CN)/N This gives overall, what fraction of the forecasts were

correct. It is very simple but it is heavily influenced by most common category, usually “no event” in the case of rare weather. The range is between “0” to “1” with per- fect score indicating “1”.

2) Probability of Detection (POD) = H/(H + M)

The range is 0 to 1 with latter for perfect score. The POD gives, what fraction of the observed “yes” were correctly forecasts? It is sensitive to hits, but ignores false alarm.

POD is also very sensitive to the climatological fre- quency of the event. POD is also an important compo- nent of the ROC used for probability forecasts.

3) False Alarm Ratio (FAR) = F/(H + F) This gives what fraction of the predicted “yes” events

actually did not occur and the range is between 0 and 1 with perfect score is “0”. The FAR is sensitive to false alarms, but ignores misses and is very sensitive to the climatological frequency of the event. It is always advis- able to use “POD” and “FAR” in conjunction to one an- other.

4) Threat score (TS) or also known as Critical Success Index (CSI) = H/(H + M + F)

This gives how well did the forecast “yes” events cor- respond to the observed “yes” events? The range is be- tween “0” and “1” with “0” indicates no skill and “1” indicates perfect score. CIS measures the fraction of ob- served and/or forecast events that were correctly pre- dicted. It can be thought of as the accuracy when correct negatives have been removed from consideration, that is, TS is only concerned with forecasts that count. Sensitive to hits, penalizes both misses and false alarms. Depends on climatological frequency of events (poorer scores for rarer events) since some hits can occur purely due to random chance.

5) Heidke Skill Score (HSS) The Heidke Skill score is in the usual skill score for-

mat and is defined as = (score value – score for the standard forecast)/(perfect score – score for the standard forecast). Or using the contingency Table “HSS” can be written as

random

random

– expected correctHSS

– expected correct

H CN

N

where (expected correct)random = [(H + M)(H + F) + (CN + M)(CN + F)]/N

For the HSS, the “score” is the number correct or the proportion correct. The “standard forecast” is usually the number correct by chance or the proportion correct by chance. The HSS measures the fractional improvement of the forecast over the standard forecast. Like most skill scores, it is normalized by the total range of possible im- provement over the standard, which means HSS can safely be compared on different datasets. The range of the HSS is −∞ to 1. Negative values indicate that the chance forecast is better, “0” means no skill, and a per- fect forecast obtains a HSS of 1. The HSS is a popular score, partly because it is relatively easy to compute and perhaps also because the standard forecast, chance, is relatively easy to beat.

Copyright © 2012 SciRes. ACS

D. R. PATTANAIK ET AL. 489

The scores discussed above are calculated for each monthly forecast of rainfall over India from June to Sep- tember from NCEP CFS hindcast during the period from 1981 to 2005. The values for June to September forecasts are given in Tables 2-5 respectively for all the three cate- gories of the forecast (PAB, PBL and PNN). As seen from the values, the monthly probability forecast in terms of the three categories clearly able to provide useful guidance. In terms the accuracy of the forecast it is seen from Tables 2-5 that it is 60% or higher in all the three categories for all four months except for the forecasts of normal category of July (52%) and below normal cate- gory of August (56%). As discussed above the accuracy may be high but it does not penalize for misses and false alarms and more appropriate scores are “TS” and “HSS”. Although there is no definite cutoff value of TS or HSS above which the forecast can be considered to be good forecast the positive value and the threshold exceeding around 0.2 could be considered as a very good score. The “TS” and “HSS” for June and July is found to be greater than 0.2 for above and below normal categories but is less than 0.2 in case of normal categories with July fore- cast in the normal category giving negative HSS value. The negative value indicates that the chance forecast is better than the forecast. Also it is seen that during the first half of the season the TS and HSS scores are higher in the month of June compared to that of July indicating May ensembles give better forecast for June compared to June ensembles for July forecasts. Similarly for the sec- ond half of the season the “TS” and “HSS” scores give higher values for September forecasts compared to that of forecast for August indicating better skill in Septem- ber rainfall compared to August rainfall. With respect to the “POD” the forecast for August in the above normal category is the lowest with only 12% followed by 14% for the normal category of July and 20% in the normal category of June. The “POD” is found to be more than 70% for June forecast in the above and below normal categories and more than 50% during July and Sep- tember forecasts in the above and below normal catego- ries. The “FAR” is about 50% or less in case of June and July for above and below normal categories and much higher in case of below normal category of August and above normal category of September.

Except the normal category of July the HSS is positive and the forecasts are better than chance forecast. In case of normal category of July even the BSS was also nega- tive indicating it is worst than Climatology. The analysis based on the scores given in Tables 2-5 also indicates that the probability forecasts particularly in the category of above and below normal monthly rainfall shows very useful skill during June, July and September with slightly lower skill in August. Thus, it is seen that the probability forecast on monthly scale during the southwest monsoon

season is very encouraging and is found to be very useful. Hence, considering the very low skill of ensemble mean monthly forecast the probability forecast can give some useful guidance in the real time, although there is a need to further improve intrinsic capability of MJO prediction in the coupled model.

6. Summary

The skill of the prediction of monthly rainfall in terms of ensemble mean of CFS during the period from 1981- 2005 is found to be very useful with correlation co-effi- cients (CCs) found to be significant for June rainfall with May initial conditions and September rainfall with Au- gust initial conditions. It is also seen that the monthly forecast in CFS has got lowest skill for the month of August forecasts (initial conditions of July) followed by that of skill for July rainfall. It is also seen that the coef- ficient of variability (CV) in case of CFS forecast is less with the peak monsoon months of July and August showing significantly less (5.1% and 7.5% respectively) compared to its corresponding observed (13.9% and 15.3% respectively) CV.

Considering the lower skill of monthly forecast based on the ensemble mean, all the ensemble members are used separately for the preparation of probability forecast. The ensemble spread measured in terms of the Standard Deviation is found to be highest in June followed by that of September, August and July. In terms of the Brier Score (BS) and Brier Skill Score (BSS) it is seen that the skill of the monthly probability forecast in all the three categories (Above normal, below normal and normal) is better than the climatology forecast in most of the cases with positive BSSs except in case of normal category forecast of June and July rainfall. The negative BSS in case of normal forecast of July is also associated with negative Heidke Skill score. With respect to the other verification score like the “Threat Score” the probability forecasts particularly in the category of above and below normal monthly rainfall shows very useful skill during June, July and September with slightly lower skill in August.

Thus, it is seen that the probability forecast on mon- thly scale during the southwest monsoon season is very encouraging and is found to be very useful. Hence, con- sidering the very low skill of ensemble mean monthly forecast the probability forecast can give some useful guidance in the real time, although there is scope to fur- ther improve intrinsic capability of MJO prediction in the coupled model through better representation of air-sea interaction in the coupled model.

7. Acknowledgements

The authors are thankful to the Director General, IMD,

Copyright © 2012 SciRes. ACS

D. R. PATTANAIK ET AL. 490

New Delhi for providing all facilities to carry out this work. We also sincerely acknowledge the National Cen- tre for Environmental Prediction (NCEP) for providing the CFS hindcasts used in the present study. Thanks are also due to reviewers for very useful suggestions, which helped in improving the quality of the paper.

REFERENCES [1] D. R. Sikka and S. Gadgil, “On the Maximum Cloud

Zone and the ITCZ over Indian Longitudes during the Southwest Monsoon,” Monthly Weather Review, Vol. 108, No. 11, 1980, pp. 1840-1853. doi:10.1175/1520-0493(1980)108<1840:OTMCZA>2.0.CO;2

[2] D. R. Pattanaik, “The Northward Moving Low Frequency Mode: A Case Study of 2001 Monsoon Season,” MAU- SAM, Vol. 54, 2003, pp. 937-940.

[3] R. Madden and P. Julian, “Detection of a 40-50 Day Os- cillation in the Zonal Wind in the Tropical Pacific,” Jour- nal of the Atmospheric Sciences, Vol. 28, No. 5, 1971, pp. 702-708. doi:10.1175/1520-0469(1971)028<0702:DOADOI>2.0.CO;2

[4] R. Madden and P. Julian, “Description of Global-Scale Circulation Cells in the Tropics with a 40-50 Day Period,” Journal of the Atmospheric Sciences, Vol. 29, No. 6, 1972, pp. 1109-1123. doi:10.1175/1520-0469(1972)029<1109:DOGSCC>2.0.CO;2

[5] R. Madden and P. Julian, “Observations of the 40-50 Day Tropical Oscillation: A Review,” Monthly Weather Re- view, Vol. 112, No. 5, 1994, pp. 814-837.

[6] C. Zhang, “Madden-Julian Oscillation,” Reviews of Geo- physics, Vol. 43, No. RG2003, 2005, pp. 1-36. doi:10.1029/2004RG000158

[7] T. N. Krishnamurti, D. Subrahmanyam, “The 30-50-Day Mode at 850 mb during MONEX,” Journal of the At- mospheric Sciences, Vol. 39, No. 9, 1982, pp. 2088-2095. doi:10.1175/1520-0469(1982)039<2088:TDMAMD>2.0.CO;2

[8] K. Miyakoda, T. Gordon, R. Caverly, W. Stern, J. Sirutis and W. Bourke, “Simulation of a blocking event in Janu- ary 1977,” Monthly Weather Review, Vol. 111, No. 4, 1983, pp. 846-869. doi:10.1175/1520-0493(1983)111<0846:SOABEI>2.0.CO;2

[9] B. Wang, J. Y. Lee, I. S. Kang, J. Shukla, C. K. Park, A. Kumar, J. Schemn, S. Cocke, J. S. Kug, J. J. Luo, T. Zhou, B. Wang, X. Fu, W. T. Yun, O. Alves, E. K. Jin, J. Kinter, B. Kirtman, T. N. Krishnamurti, N. C. Lau, W. Lau, P. Liu, P. Pegion, T. Rosati, S. Schubert, W. Stern, M. Suarez and T. Yamagata, “Advance and Prospects of Seasonal Prediction: Assessment of the APCC/CliPAS 14-Model Ensemble Retrospective Seasonal Prediction (1980-2004),” Climate Dynamics, Vol. 33, No. 1, 2008, pp. 93-117. doi:10.1007/s00382-008-0460-0

[10] R. Krishnan, S. Sundaram, P. Swapna, V. Kumar, D. C.

Ayantika and M. Mujumdar, “Crucial Role of Ocean-At- mosphere Coupling on the Indian Monsoon Anomalous Response during Dipole Events,” Climate Dynamics, Vol. 37, No. 1-2, 2010, pp. 1-17. doi:10.1007/s00382-010-0830-2

[11] L. Ferranti, T. N. Palmer, F. Molteni and E. Klinker, “Tropical-Extratropical Interaction Associated with the 30-60-Day Oscillation and Its Impact on Medium and Ex- tended Range Prediction,” Journal of the Atmospheric Sciences, Vol. 47, No. 18, 1990, pp. 2177-2199. doi:10.1175/1520-0469(1990)047<2177:TEIAWT>2.0.CO;2

[12] V. Frederic, “Monthly Forecasting at ECMWF,” Monthly Weather Review, Vol. 132, No. 12, 2004, pp. 2761-2779. doi:10.1175/MWR2826.1

[13] S. Saha, S. Nadiga, C. Thiaw, J. Wang, W. Wang, Q. Zhang, H. M. Van den Dool, H. L. Pan, S. Moorthi, D. Behringer, D. Stokes, M. Pena, S. Lord, G. White, W. Ebisuzki, P. Peng and P. Xie, “The NCEP Climate Forecast System,” Journal of Climate, Vol. 19, No. 15, 2006, pp. 3483-3517. doi:10.1175/JCLI3812.1

[14] D. R. Pattanaik and A. Kumar, “Prediction of Summer Monsoon Rainfall over India Using the NCEP Climate Forecast System,” Climate Dynamics, Vol. 34, No. 4, 2010, pp. 557-572. doi:10.1007/s00382-009-0648-y

[15] D. R. Pattanaik, A. Tyagi and A. Kumar, “Dynamical- Empirical Forecast for the Indian Monsoon Rainfall Us- ing the NCEP Coupled Modelling System—Application for Real Time Monsoon Forecast,” MAUSAM, Vol. 63, pp. 433-448.

[16] I. S. Kang, J. Lee and C. K. Park, “Potential Predictability of Summer Mean Precipitation in a Dynamical Seasonal Prediction System with Systematic Error Correction,” Journal of Climate, Vol. 17, No. 4, 2004, pp. 834-844.

[17] S. C. Kar, N. Acharya, U. C. Mohanty and M. A. Kul- karni, “Skill of Mean of Distribution of Monthly Rainfall over India during July Using Multi-Model Ensemble Schemes,” International Journal of Climatology, Vol. 32, No. 8, 2011, pp. 1271-1286. doi:10.1002/joc.2334

[18] K. K. Kumar, M. Hoerling and B. Rajagopalan, “Ad- vancing Dynamical Prediction of Indian Monsoon Rain- fall,” Geophysical Research Letters, Vol. 32, No. L08704, 2005, 4 pp. doi:10.1029/2004GL021979

[19] B. Q. Wang, X. Ding, I-S Fu, K. Kang, J. S. Jin and F. Doblas-Reyes, “Fundamental Challenge in Simulation and Prediction of Summer Monsoon Rainfall,” Geo- physical Research Letters, Vol. 32, No. L15711, 2005, 4 pp. doi:10.1029/2005GL022734

[20] S. Joseph, A. K. Sahai and B. N. Goswami, “Boreal Sum- mer Intraseasonal Oscillations and Seasonal Indian Mon- soon Prediction in DEMETER Coupled Models,” Climate Dynamics, Vol. 35, No. 4, 2010, pp. 651-667. doi:10.1007/s00382-009-0635-3

[21] M. Rajeevan, C. K. Unnikrishnan and B. Preethi, “Evaluation of the ENSEMBLES Multi-Model Seasonal Forecasts of Indian Summer Monsoon Variability,” Cli- mate Dynamics, Vol. 38, No. 11-12, 2011, pp. 2257-2274. doi:10.1007/s00382-011-1061-x

[22] M. A. Kulkarni, N. Acharya, S. C. Kar, U. C. Mohanty,

Copyright © 2012 SciRes. ACS

D. R. PATTANAIK ET AL.

Copyright © 2012 SciRes. ACS

491

M. K. Tippett, A.W. Robertson, J. J. Luo and Toshio Yamagata, “Probabilistic Prediction of Indian Summer Monsoon Rainfall Using Global Climate Models,” Theo- retical and Applied Climatology, Vol. 107, No. 3-4, 2011, pp. 441-450. doi:10.1007/s00704-011-0493-x

[23] B. K. Samala, R. Krishnan and M. roxy, “Assessment of One Month Forecasts of Weak Indian Monsoons Based on the NCEP Climate Forecast System (CFS),” Meteoro- logical Applications, Vol. 19, No. 2, 2012, pp. 189-199. doi:10.1002/met.1331

[24] S. Moorthi, H. L. Pan and P. Caplan, “Changes to the 2001 NCEP Operational MRF/AVN Global Analysis/ Forecast System,” NWS Technical Procedures Bulletin, Vol. 484, 2001, pp. 1-14. http://www.nws.noaa.gov/om/tpb/484.htm

[25] E. Kalnay E and Coauthors, “The NCEP/NCAR 40-Year Reanalysis Project,” Bull Amer Meteor Soc, Vol. 77, 1996, pp. 1057-1072.

[26] R. Kistler, E. Kalnay, W. Collins, S. Saha, G. White, J. Woollen, M. Chelliah, W. Ebisuzaki, M. Kanamitsu, V. Kousky, H. van den Dool, R. Jenne and M. Fiorino, “The NCEP-NCAR 50-Year Reanalysis: Monthly Means CD-ROM and Documentation,” Bulletin of the American Meteorological Society, Vol. 82, No. 2, 2001, pp. 247-268. doi:10.1175/1520-0477(2001)082<0247:TNNYRM>2.3.CO;2

[27] Y. T. Hou, K.A. Campana and S. K. Yang, “Shortwave Radiation Calculations in the NCEP’s Global Model,” In- ternational Radiation Symposium, IRS-96, Fairbanks, 19-24 August 1996.

[28] Y. T. Hou, S. Moorthi and K. Campana, “Parameteriza- tion of Solar Radiation Transfer in the NCEP Models,” 2002. http://www.emc.ncep.noaa.gov/officenotes/FullTOC.html#2000.

[29] S. Y. Hong and H. L. Pan, “Nonlocal Boundary Layer Vertical Diffusion in a Medium-Range Forecast Model,” Monthly Weather Review, Vol. 124, No. 10, 1996, pp. 2322-2339. doi:10.1175/1520-0493(1996)124<2322:NBLVDI>2.0.CO;2

[30] S. Y. Hong and H. L. Pan, “Convective Trigger Function for a Mass-Flux Cumulus Parameterization Scheme,” Monthly Weather Review, Vol. 126, No. 10, 1998, pp. 2599-2620.

[31] Y. J. Kim and A. Arakawa, “Improvement of Orographic Gravity Wave Parameterization Using a Mesoscale Grav- ity Wave Model,” Journal of the Atmospheric Sciences, Vol. 52, No. 11, 1995, pp. 1875-1902.

[32] R. C. Pacanowski and S. M. Griffies, “MOM 3.0 Manual,” NOAA/Geophysical Fluid Dynamics Laboratory, Prince- ton, 1998, p. 08542.

[33] W. G. Large, J. C. McWilliams and S. C. Doney, “Oce- anic Vertical Mixing: A Review and a Model with

Non-Local Boundary Layer Parameterization,” Reviews of Geophysics, Vol. 32, No. 4, 1994, pp. 363-403. doi:10.1029/94RG01872

[34] J. Smagorinsky, “General Circulation Experiments with the Primitive Equations: I. The Basic Experiment,” Mon- thly Weather Review, Vol. 91, No. 3, 1963, pp. 99-164.

[35] M. Kanamitsu, W. Ebisuzaki, J. Woollen, S. K. Yang, J. J. Slingo, M. Fiorino and G. L. Potter, “NCEP-DOE AMIP- II Reanalysis (R-2),” Bulletin of the American Meteoro- logical Society, Vol. 83, No. 11, 2002, pp. 1631-1643. doi:10.1175/BAMS-83-11-1631

[36] P. Xie and P. A. Arkin, “Analysis of Global Monthly Precipitation Using Gauge Observations, Satellite Esti- mates, and Numerical Model Predictions,” Journal of Climate, Vol. 9, No. 4, 1996, pp. 840-858. doi:10.1175/1520-0442(1996)009<0840:AOGMPU>2.0.CO;2

[37] C. E. Leith, “Theoretical Skill of Monte Carlo Forecasts,” Monthly Weather Review, Vol. 102, No. 6, 1974, pp. 409- 418. doi:10.1175/1520-0493(1974)102<0409:TSOMCF>2.0.CO;2

[38] T. N. Palmer, “Chaos and Predictability in Forecasting the Monsoon,” Proceedings of the Indian National Sci- ence Academy, Vol. 60, 1994, pp. 57-66.

[39] T. N. Palmer, C. Branković and D. S. Richardson, “A Probability and Decision-Model Analysis of PROVOST Seasonal Multi-Model Ensemble Integrations,” Quarterly Journal of the Royal Meteorological Society, Vol. 126, 2000, pp. 2013-2033. doi:10.1256/smsqj.56702

[40] A. H. Murphy, “Comparative Evaluation of Categorical and Probabilistic Forecasts. Two Alternatives to the Tra- ditional Approach,” Monthly Weather Review, Vol. 114, 1986, pp. 2363-2401. doi:10.1175/1520-0493(1986)114<0245:CEOCAP>2.0.CO;2

[41] T. N. Krishnamurti, L. Stefanova, A. Chakraborty, T. S. V. Vijaya Kumar, S. Cocke, D. Bachiochi and B. Mackey, “Seasonal Forecasts of precipitation anomalies for North American and Asian Monsoons,” Florida State University (FSU), Tallahassee, 2001, pp. 1-81.

[42] Brier, “Verification of Forecasts Expressed in Terms of Probability,” Monthly Weather Review, Vol. 78, No. 1, 1950, pp. 1-3.

[43] V. V. Kharin and F. W. Zwiers, “Improved Seasonal Probability Forecasts,” Journal of Climate, Vol. 16, No. 11, 2003, pp. 1684-1701. doi:10.1175/1520-0442(2003)016<1684:ISPF>2.0.CO;2

[44] J. A. Swets, “The Relative Operating Characteristic in Psychology,” Science, Vol. 182, No. 4116, 1973, pp. 990- 1000. doi:10.1126/science.182.4116.990

[45] I. Mason, “A Model for Assessment of Weather Fore- casts,” Australian Meteorological Magazine, Vol. 30, 1982, pp. 291-303.