An Experience on the Selection of Members for Simplifying a Multimodel Hydrological Ensemble Prediction System

Darwin Brochero, François Anctil and Christian Gagné

Abstract - In hydrology, as in many applications, it is accepted that there is no superior model for every application under all circumstances. In that sense, a multimodel approach may circumvent structural errors or inductive biases of the models. On the other hand, a Hydrology Ensemble Prediction System (HEPS), obtained by forcing a rainfall-runoff model with forecasts from a Meteorological Ensemble Prediction Systems (MEPS), have been recognized as a useful approach to quantify streamflow forecast uncertainties. Larger HEPS may thus be envisioned, simultaneously considering the model structure and the meteorological forecast sources of uncertainty. The complexity of such grand HEPS rapidly becomes an operational burden when one has to evaluate several hundreds of scenarios at each time step. This situation may even be more dramatic considering the current trend of also targeting several MEPS.
The main objective of the current work is to assess the degree of simplification (reduction of members) of a HEPS configured with 16 lumped hydrological models driven by the 50 weather ensemble forecasts from the European Centre for Medium-range Weather Forecasts (ECMWF). Here, the selection of the most relevant members from an initial 800-member grand ensemble is proposed using a Backward greedy technique with k-fold cross validation. This represents a direct measure to evaluate weight that each model must represent within a subset that offers the same or better performance than the reference set of 800 members.
Several statistical measures should be considered concurrently to assess the quality of ensemble forecasts, drawing the following question. Which optimization criterion would lead to the best simplification of a hydrological ensemble prediction system with the proposed technique of selection of members? Results exploiting the following five different optimization criteria are thus confronted: the continuous ranked probability score (CRPS), the ignorance score, the reliability diagram, the rank histogram, and the coefficient of variation. The combination of all of them is also tested.
Results, for ten French catchments, support the idea that selecting HEPS members is viable. It is in general even possible to expect better balance of scores in the 30-member subset than in the original much larger 800-member ensemble. However, the degree of reduction of members may be established in terms of the number of members required (complexity of the HEPS) or of the relationship between the different scores (performance).
The evaluation of five individual scores as criteria for optimizing the selection process revealed the complexity of the relationship between them. In many situations, improving one score is achieved at the expense of another one. For instance, the CRPS often the primary score used for evaluating HEPS performance is not a good choice for member selection. In fact, it was often possible to preserve or minimize the CRPS using another objective criterion. The design of a combined criterion turned out to be an important methodological improvement that integrates many characteristics of each score. The ratio of the rank histogram is the best single optimization criteria, not very distant to the achievements of the combined criterion.

download document


    author    = { Darwin Brochero and François Anctil and Christian Gagné },
    title     = { An Experience on the Selection of Members for Simplifying a Multimodel Hydrological Ensemble Prediction System },
    booktitle = { CSHS Workshop: Operational River Flow and Water Supply Forecasting },
    year      = { 2011 },
    month     = { October 6-7 },
    location  = { Vancouver (BC), Canada }

Last modification: May 20 2013 11:44AM by cgagne


©2002-. Computer Vision and Systems Laboratory. All rights reserved