Comparing expert judgement and numerical criteria for hydrograph evaluation

This paper investigates the relationship between expert judgement and numerical criteria when evaluating hydrological model performance by comparing simulated and observed hydrographs. Using a web-based survey, we collected the visual evaluations of 150 experts on a set of high- and low-flow hydrographs. We then compared these answers with results from 60 numerical criteria. Agreement between experts was found to be more frequent in absolute terms (when rating models) than in relative terms (when comparing models), and better for high flows than for low flows. When comparing the set of 150 expert judgements with numerical criteria, we found that most expert judgements were loosely correlated with a numerical criterion, and that the criterion that best reflects expert judgement varies from expert to expert. Overall, we identified two groups of 10 criteria yielding an equivalent match with the expertise of the 150 participants in low and high flows, respectively. A single criterion common to both groups (the Hydrograph Matching Algorithm with mean absolute error) may represent a good indicator for the overall evaluation of models based on hydrographs. We conclude that none of the numerical criteria examined here can fully replace expert judgement when rating hydrographs, and that both relative and absolute evaluations should be based on the judgement of multiple experts. Editor D.