Tara Jensen-March 12

A Community Effort for Convection Allowing Model (CAM) Scorecarding

The use of scorecards to synthesize the skill of a forecast system has been suggested in literature describing best practices for designing ensemble prediction systems (Sandgathe et al., 2011, Sandgathe et al., 2013).  Scorecards have been used in the development of the National Center for Environmental Prediction (NCEP) Environmental Modeling Center (EMC) Global Forecast System (GFS – Yang, 2010-2017 and Zhou et al., 2016) and a data assimilation framework (Kuhl et al., 2013).  The National Center for Atmospheric Research (NCAR), through the Developmental Testbed Center (DTC), has been using scorecards for synthesizing the results of extensive testing and evaluation for many years.  The statistics and significance were computed by the Model Evaluation Tools, which is a verification package supported out to the community by the DTC.  Examples of these scorecards can be found in final reports on the DTC website (Wolff et al., 2010-2017).  Flexible scorecarding capability was added to the database and display package called METViewer in 2017 and is now being used by DTC, EMC and other entities that have access to METViewer.

An ongoing United States Weather Research Program (USWRP) testbed project between NCAR and the National Severe Storms Laboratory (NSSL) is focused on determining which metrics and fields to include in a comprehensive scorecard to assess improvements to Convection Allowing Models (CAMs) and making that capability available to the Hazardous Weather Testbed (HWT).  This effort is also being discussed and supported by a community-led group focused on guiding the Next Generation Global Prediction System project and EMC on development of the upcoming Unified Forecast System (UFS).  Both the HWT Spring Warning and Forecast Experiments s(HWT/SWE and HWT/SFE) are contributing to development of a scorecard focused on storm-attributes, while EMC, NOAA/Earth System Research Laboratory (ESRL) and others are contributing additional requirements to assess other aspects of CAM performance.  This work leverages work already performed by the Warn-on-Forecast group developing scorecards for Convection Allowing Models (CAMs) providing forecasts on the 0-6hr time frame.  This presentation will introduce the idea of scorecarding and provide an update on what will be available during the HWT this year.