Assessing the Calibration of High-Dimensional Ensemble Forecasts Using Rank Histograms

Details

• Journal Title:
  Journal of Computational and Graphical Statistics
• Personal Author:
  Thorarinsdottir, Thordis L. ; Scheuerer, Michael ; Heinz, Christopher
• NOAA Program & Office:
  OAR (Oceanic and Atmospheric Research) ; ESRL (Earth System Research Laboratory) ; CIRES (Cooperative Institute for Research in Environmental Sciences)
• Description:
  Any decision-making process that relies on a probabilistic forecast of future events necessarily requires a calibrated forecast. This article proposes new methods for empirically assessing forecast calibration in a multivariate setting where the probabilistic forecast is given by an ensemble of equally probable forecast scenarios. Multivariate properties are mapped to a single dimension through a prerank function and the calibration is subsequently assessed visually through a histogram of the ranks of the observation’s preranks. Average ranking assigns a prerank based on the average univariate rank while band depth ranking employs the concept of functional band depth where the centrality of the observation within the forecast ensemble is assessed. Several simulation examples and a case study of temperature forecast trajectories at Berlin Tegel Airport in Germany demonstrate that both multivariate ranking methods can successfully detect various sources of miscalibration and scale efficiently to high-dimensional settings. Supplemental material in form of computer code is available online.
• Keywords:
  Computer Programs Numerical Weather Forecasting Computer Programs Numerical Weather Forecasting
• Source:
  Journal of Computational and Graphical Statistics, 25:1, 105-122, 2016
• DOI:
  https://doi.org/10.1080/10618600.2014.977447
• Document Type:
  Journal Article
• Rights Information:
  Accepted Manuscript
• Rights Statement:
  The NOAA IR provides access to this content under the authority of the government's retained license to distribute publications and data resulting from federal funding. While users may legally access this content, the copyright owners retain rights that govern the reproduction, redistribution, and re-use of this work. The user is solely responsible for complying with applicable copyright law.
• Compliance:
  Submitted
• Main Document Checksum:
  urn:sha-512:a5f4b9c9c5cc80a7b876e170eb3d998e4a6f1c6142c3a1a5a91db6c4f83bae3a487e5c340c35a20bf2714f25aee0ba614615f30bc53a49508e72904ea9d0c2a0
• Download URL:
  https://repository.library.noaa.gov/view/noaa/33235/noaa_33235_DS1.pdf
• File Type:
  [PDF - 357.64 KB ]