NOAA Technical Information Series NESDIS
DSMR-00125 Version 1.0

doi: 10.25923/z5p4-cm07

Data Stewardship Maturity Report for NOAA JPSS Advanced Technology Microwave Sounder (ATMS) Sensor Data Record (SDR) from IDPS

<table>
  <tr>
    <th colspan="6">Table 1 Legend</th>
  </tr>
  <tr>
    <th>Level 1</th>
    <th>Level 2</th>
    <th>Level 3</th>
    <th>Level 4</th>
    <th>Level 5</th>
  </tr>
  <tr>
    <td>Ad Hoc</td>
    <td>Minimal</td>
    <td>Intermediate</td>
    <td>Advanced</td>
    <td>Optimal</td>
  </tr>
  <tr>
    <td>Little or no management</td>
    <td>Limited Management</td>
    <td>Defined Management, partially implemented</td>
    <td>Well-defined Management, fully implemented</td>
    <td>Full Management, audited, measured, controlled</td>
  </tr>
</table>

<table>
  <tr>
    <th colspan="3">Table 1. Scores for the Nine DSMM Key Components at a Glance</th>
  </tr>
  <tr>
    <th>Preservability - 4.5</th>
    <th>Accessibility - 5</th>
    <th>Usability - 4.5</th>
  </tr>
  <tr>
    <td>Production Sustainability - 5</td>
    <td>Data Quality Assurance - 4</td>
    <td>Data Quality Control/Monitoring - 3.5</td>
  </tr>
  <tr>
    <td>Data Quality Assessment - 3</td>
    <td>Transparency/Traceability - 4</td>
    <td>Data Integrity - 5</td>
  </tr>
</table>
Cover Image: Data Stewardship Rating Diagram for NOAA JPSS Advanced Technology Microwave Sounder (ATMS) Sensor Data Record (SDR) from IDPS

Shades of green are used to represent level 1 through level 5 ratings; denoting Ad Hoc, Minimal, Intermediate, Advanced, and Optimal stages for each of the nine key components, respectively. The dark green level indicates all the practices are completely satisfied. The lighter green levels indicate only some of the practices are satisfied. The lightest green level indicates none of the practices are satisfied.

The stewardship maturity of NCEI data product, NOAA JPSS Advanced Technology Microwave Sounder (ATMS) Sensor Data Record (SDR) from IDPS, is assessed based on a reference stewardship maturity framework. The current maturity ratings of NOAA JPSS Advanced Technology Microwave Sounder (ATMS) Sensor Data Record (SDR) from IDPS are at Level 1 or higher for all nine key components with zero Level 1, zero Level 2, two Level 3, four Level 4, and three Level 5 key components.
NOAA Technical Information Series NESDIS DSMR-00125 Version 1.0

The National Environmental Satellite, Data, and Information Service (NESDIS) manages the Nation’s civil Earth-observing satellite systems, as well as global national data bases for meteorology, oceanography, geophysics, and solar-terrestrial sciences. From these sources, it develops and disseminates environmental data and information products critical to the protection of life and property, national defense, and the national economy, energy development and distribution, global food supplies, and the development of natural resources.

Publication in the NOAA Technical Memorandum series does not preclude later publication in scientific journals in expanded or modified form. The NESDIS series of NOAA Technical Reports is a continuation of the former NESS and EDIS series of NOAA Technical Reports and the NESC and EDS series of Environmental Science Services Administration (ESSA) Technical Reports.

Copies of earlier reports may be available by contacting NESDIS Chief of Staff, NOAA/NESDIS, 1335 East-West Highway, SSMC1, Silver Spring, MD 20910, (301) 713-3578.
ASSESSMENT REVISION HISTORY

<table>
  <tr>
    <th>Revision</th>
    <th>Description</th>
    <th>Date</th>
  </tr>
  <tr>
    <td>V01r00</td>
    <td>Initial Release</td>
    <td>12/16/2021</td>
  </tr>
</table>
NOAA Technical Information Series NESDIS DSMR-00125
Version 1.0
doi: 10.25923/z5p4-cm07

Data Stewardship Maturity Report for NOAA JPSS Advanced Technology Microwave Sounder (ATMS) Sensor Data Record (SDR) from IDPS

Lori Hager, Paul Lemieux III
NOAA's National Centers of Environmental Information (NCEI)
151 Patton Avenue, Asheville, NC 28801, (828) 271-4800

Recommended Citation

Lori Hager, Paul Lemieux III. (2021), Data Stewardship Maturity Report for NOAA JPSS Advanced Technology Microwave Sounder (ATMS) Sensor Data Record (SDR) from IDPS

NOAA Technical Information Series NESDIS DSMR-00125 Version 1.0, 21pp., doi: 10.25923/z5p4-cm07
Table of Contents

List of Tables ..................................................................................................................... 7
Preface ................................................................................................................................. 8
1. Introduction ..................................................................................................................... 9
2. Results ............................................................................................................................ 10
3. Acknowledgment ........................................................................................................... 15
4. References .................................................................................................................... 16
Appendix I  The Scientific Data Stewardship Maturity Matrix (DSMM) ......................... 18
List of Tables

Table 1. Scores for the Nine DSMM Key Components at a Glance .................................................. 1
Table 2. Dataset and Data Stewardship Maturity Assessment Metadata ............................................. 11
Table 3. Stewardship Maturity Levels and Detailed Justifications for Each of Nine DSMM Key Components for the Dataset. ..................................................................................................................... 12
Preface

In response to the President's Open Government Initiative and related policies, NOAA has committed to providing improved public access to all of its environmental information, to enable research and commercial innovation through ease of data discovery and use [Casey, 2016].

OneStop supports NOAA's efforts by leveraging existing access technologies and infusing specific innovations to provide improved discover, access, and visualization services for NOAA's data. Also, OneStop is viewed by a NESDIS as a pathfinder effort with an initial focus on selected high-priority datasets from NESDIS and other program data meeting OneStop standards, but eventually scalable across NOAA's data. Lastly, OneStop is implementing the USGEO Common Framework for Earth Observation Data and leveraging/supporting the NOAA Big Data Project (BDP) and Big Earth Data Initiative (BEDI) [Casey, 2016].

As with any process of improvement planning, agencies need to find out where they are in terms of their compliance to the federal regulations and what they need to do if any areas of non-compliance are identified. To this end, a unified framework would be beneficial for assessing the current stage of stewardship practices applied to individual datasets and for providing a road map that will guide future investments towards enhanced stewardship of environmental datasets. The value and quality of a dataset depends in part on the stewardship practices applied after its development and production. Therefore, a unified framework providing a holistic view of the quality of stewardship practices applied to individual datasets is beneficial to data stewards and users [Casey, 2016].

The Data Stewardship Maturity Matrix (DSMM), jointly developed by domain (data management, technology, and science) subject matter experts from NOAA’s National Centers for Environmental Information (NCEI) and Cooperative Institute for Climate and Satellites – North Carolina (CICS-NC), provides such a consistent framework [Peng et al., 2016]. The DSMM, leveraging institutional knowledge and community practices and standards, defines a graduated maturity scale for each of nine key components of scientific data stewardship to enable a consistent assessment of the measureable stewardship practices applied to a given data set or product.

The NOAA Data Stewardship Maturity Technical Series captures stewardship maturity assessment results for individual datasets, provides consistent representation and citable documents of those assessments, ensures transparency, and allows better data quality information integration and content-based search and discovery of NOAA data.
NOAA Technical Information Series NESDIS DSMR-00125 Version 1.0

Data Stewardship Maturity Report for NOAA JPSS Advanced Technology Microwave Sounder (ATMS) Sensor Data Record (SDR) from IDPS

1. Introduction

1.1 Purpose

The purpose of this document is to describe the results of stewardship maturity assessment for NOAA Climate Data Record for Mean Layer Temperature (Upper Troposphere & Lower Stratosphere from UCAR, Version 2, utilizing the Scientific Data Stewardship Maturity Matrix or DSMM [Peng, et al, 2016]. DSMM defines levels of stewardship maturity stages for Preservability, Accessibility, Usability, Production Sustainability, Data Quality Assurance, Data Quality Control/Monitoring, Data Quality Assessment, Transparency/Traceability, and Data Integrity key components. Each of these components is ranked from ‘Ad hoc’ to ‘Optimal’ (see Appendix I). This report is based on evaluation performed by NOAA OneStop metadata specialists working with Subject Matter Experts and utilizing the DSMM template [Peng, 2016].

1.2 Scope

Assessing stewardship maturity - the current state of how datasets are documented, preserved, stewarded, and made accessible publicly, is a critical step towards meeting U.S. federal regulations, organizational requirements, and user needs [Peng et al., 2016]. The goal of this document is to provide consistent and transparent stewardship maturity information to data users and decision-makers.

1.3 Dataset Abstract

The Advanced Technology Microwave Sounder (ATMS) on board Suomi NPP is a 22 channel microwave sounder that provides continuous cross-track scanning in an 824 km sun-synchronous orbit. This Sensor Data Record (SDR) from ATMS provides Level 1b calibrated and geolocated radiance data produced from the ATMS Raw Data Record (RDR) and Temperature Data Record (TDR). This operational SDR is used to produce ATMS Environmental Data Records (EDR) for measuring atmospheric temperature and moisture. ATMS SDR data obtained from the NOAA Comprehensive Large Array-Data Stewardship System (CLASS) are distributed as aggregated 8 minute files consisting of 15 granules in the Hierarchical Data Format v.5 (HDF5) with metadata attributes included.
1.4 Document Maintenance

This document is generated and maintained by NOAA’s National Centers for Environmental Information. More on policy is available at https://www.ncei.noaa.gov/.

2. Results

The data stewardship maturity assessment information is summarized in Table 1. Each component is displayed along with its corresponding score in a color-coded table.
<table>
  <tr>
    <th colspan="2">Table 2. Dataset and Data Stewardship Maturity Assessment Metadata</th>
  </tr>
  <tr>
    <th>Dataset Title</th>
    <td>NOAA JPSS Advanced Technology Microwave Sounder (ATMS) Sensor Data Record (SDR) from IDPS</td>
  </tr>
  <tr>
    <th>Dataset Information URL</th>
    <td>https://doi.org/10.7289/V5BR8Q7N</td>
  </tr>
  <tr>
    <th>Data Provider POC (Name; Email; Affiliation)</th>
    <td>Customer Engagement Branch, ncei.sat.info@noaa.gov, DOC/NOAA/NESDIS/NCEI > National Centers for Environmental Information, NESDIS, NOAA, U.S. Department of Commerce</td>
  </tr>
  <tr>
    <th>Dataset POC (Name; Email; Affiliation)</th>
    <td>Fuzhong Weng, Fuzhong.Weng@noaa.gov, DOC/NOAA/NESDIS/STAR > Center for Satellite Applications and Research, NESDIS, NOAA, U.S. Department of Commerce</td>
  </tr>
  <tr>
    <th>SMM Version (Document ID and Version Number)</th>
    <td>NCDC-CICS-SMM_0001_Rev.1 12/09/2014</td>
  </tr>
  <tr>
    <th>SMM POC (Name; E-mail; Affiliation)</th>
    <td>Ge Peng, ge.peng@uah.edu, University of Alabama-Huntsville</td>
  </tr>
  <tr>
    <th>SMM Template Version (Document ID and Version Numbers)</th>
    <td>NCDC-CICS-SMM_0001_Rev.1 v4.0 06/23/2015</td>
  </tr>
  <tr>
    <th>SMM Template POC</th>
    <td>Ge Peng, ge.peng@uah.edu, University of Alabama-Huntsville</td>
  </tr>
  <tr>
    <th>SMM Assessment Version (v=xn>r=mm>, e.g., v01r00)</th>
    <td>v01r05</td>
  </tr>
  <tr>
    <th>SMM Assessment Date (MM/DD/YYYY)</th>
    <td>03/08/2017</td>
  </tr>
  <tr>
    <th>SMM Assessment POC (Name; E-mail; Affiliation)</th>
    <td>Paul Lemieux III, paul.lemieux@noaa.gov, Earth Resources Technology, Inc.</td>
  </tr>
  <tr>
    <th>Stewardship Maturity Ratings (each key component) (kc1/kc2/kc3/kc4/kc5/kc6/kc7/kc8/kc9)</th>
    <td>4.5 / 5 / 4.5 / 5 / 4 / 3.5 / 3 / 4 / 5</td>
  </tr>
  <tr>
    <th>SMM Original Assessment Date (MM/DD/YYYY)</th>
    <td>07/14/2016</td>
  </tr>
  <tr>
    <th>SMM Original Assessment POC (Name; E-mail; Affiliation)</th>
    <td>Paul Lemieux III, paul.lemieux@noaa.gov, Earth Resources Technology, Inc.</td>
  </tr>
  <tr>
    <th>SMM Last Modified Date (MM/DD/YYYY)</th>
    <td>10/07/2021</td>
  </tr>
  <tr>
    <th>SMM Last Modification POC (Name; E-mail; Affiliation)</th>
    <td>Lori Hager, lori.hager@noaa.gov , CASE Consultants International</td>
  </tr>
  <tr>
    <th>SMM Modified Date (MM/DD/YYYY)</th>
    <td>04/24/2019</td>
  </tr>
  <tr>
    <th>SMM Modification POC (Name; E-mail; Affiliation)</th>
    <td>Paul Lemieux III, paul.lemieux@noaa.gov, Riverside Technology, Inc.</td>
  </tr>
</table>
<table>
  <tr>
    <th>DSMM Key Component</th>
    <th>Stewardship Maturity Rating, Justification, and Comments</th>
  </tr>
  <tr>
    <td><b>Preservability</b></td>
    <td>
      Level 4.5<br>
      • Archived by NCEI which is a NOAA designated archive compliant to NARA standards.<br>
      • Metadata following ISO 19115-2.<br>
      • Compliant to OIAS RM.<br>
      • Plans to update metadata to ISO 19115-1 at a later date and may be a pilot dataset for the OneStop initiative.<br>
      • Using CLASS archiving system which is CMMI-Level 3 compliant.<br><br>
      Comments:<br>
      No known audits on the archiving processes
    </td>
  </tr>
  <tr>
    <td><b>Accessibility</b></td>
    <td>
      Level 5<br>
      • Collection level searchable online<br>
      • Direct file download available CLASS FTP: ftp://ftp-npp.bou.class.noaa.gov<br>
      • CLASS ordering:<br>
      https://www.avl.class.noaa.gov/saa/products/search?sub_id=0&datatype_family=ATMS_SDR<br>
      • Granules searchable and orderable via CLASS.<br>
      • CLASS has dissemination reports available internally and externally. Users have to e-mail the CLASS Help Desk to request access to the metrics tools.<br>
      • New technology for OneStop search and discovery planned (i.e. ElasticSearch, Hyrax Servers, etc.) This is part of the JPSS/NPP data group that will be enhanced for OneStop readiness.<br><br>
      Comments:<br>
      A CLASS improvement (GEARS) is in the works.
    </td>
  </tr>
  <tr>
    <td><b>Usability</b></td>
    <td>
      Level 4.5<br>
      • Satellite user community standard HDF5 data format.<br>
      • ATBD [Chu & Northrop Grumman, 2011] and other supporting documentation available online here: https://www.star.nesdis.noaa.gov/jpss/Docs.php<br>
      • Data visualization capability via JPSS Long Term Monitoring (LTM) system: https://www.star.nesdis.noaa.gov/icvs/status_NPP_ATMS.php<br>
      • CLASS has deaggregating/subsetting options for downloads.<br><br>
      Comments:<br>
      No known external rankings.
    </td>
  </tr>
  <tr>
    <td><b>Production Sustainability</b></td>
    <td>
      Level 5<br>
      • Product improvement process in place, product has reached validated maturity level 1 and there are plans to advance it to level 2. Product may transition from IDPS to the newer NDE processing system.<br>
      • Long-term Institutional commitment through NOAA/JPSS in place.<br>
      • National and international commitment from Numerical Weather Prediction (NWP) centers, NCEP, Navel Research Lab (NRL), UK MetOffice, and ECMWF.<br><br>
      Comments:<br>
      No comments.
    </td>
  </tr>
</table>
<table>
  <tr>
    <th>DSMM Key Component</th>
    <th>Stewardship Maturity Rating, Justification, and Comments</th>
  </tr>
  <tr>
    <td>Data Quality Assurance</td>
    <td>Level 4<br>
      • Product maturity document [Weng, 2013] available online that outlines the rigorous QA/QC controls implemented on the product: https://www.star.nesdis.noaa.gov/jpss/AlgorithmMaturity.php<br>
      • STAR JPSS Quality Assurance Plan [Mikles and Liu, 2016] well documented and fully implemented.<br>
      • Quality flags included at the file level that can be considered limited data quality assurance metadata<br><br>
      Comments:<br>
      No known external reviews on data quality metadata.
    </td>
  </tr>
  <tr>
    <td>Data Quality Control/Monitoring</td>
    <td>Level 3.5<br>
      • Data quality control and monitoring steps are available online per the JPSS validation requirements. Anomaly detection procedure, reporting, and remedies are also available as a requirement for the validated (operational) maturity phase. See the CalVal presentation [Weng, 2013] Weng, F., (2013), Suomi NPP ATMS CalVal overview [PowerPoint slides], retrieved from: https://www.star.nesdis.noaa.gov/jpss/AlgorithmMaturity.php (Accessed 19 January 2017).] available online here: https://www.star.nesdis.noaa.gov/jpss/AlgorithmMaturity.php<br>
      • Long term data quality monitoring application available to the public: https://www.star.nesdis.noaa.gov/icvs/status_NPP_ATMS.php<br><br>
      Comments:<br>
      No data quality information in the metadata record.<br>
      I’m not sure how users provide feedback, but there are several instances of user feedback given in the maturity review documents so there must a system in place to do it. I think users can e-mail team leads directly at STAR.
    </td>
  </tr>
  <tr>
    <td>Data Quality Assessment</td>
    <td>Level 3<br>
      • ATBD [Chu and Northrop Grumman, 2011] available online here: https://www.star.nesdis.noaa.gov/jpss/Docs.php<br>
      • Research product assessed in literature [Weng, Yang, Tian, et al., 2013] available online here: https://doi.org/10.1002/jgrd.50840<br>
      • Operational assessment available in literature [Weng, Zou, Wang, et al., 2017] available online here: https://doi.org/10.1029/2012JD018144<br><br>
      Comments:<br>
      No data quality information in the metadata record and no metadata assessment.<br>
      No known external ranking
    </td>
  </tr>
</table>
<table>
  <tr>
    <th>DSMM Key Component</th>
    <th>Stewardship Maturity Rating, Justification, and Comments</th>
  </tr>
  <tr>
    <td rowspan="2">Transparency / Traceability</td>
    <td>Level 4<br>
      • ATBD [Chu and Northrop Grumman, 2011] available here: https://www.star.nesdis.noaa.gov/jpss/Docs.php<br>
      • OAD [Guenther, Gu, Siebels, et al., 2015] available online here:<br>
      • https://www.jpss.noaa.gov/sciencedocs.html<br>
      • DOI: 10.7289/V5BR8Q7N<br>
      • Product information available in literature [Weng, Yang, Tian, et al., 2013]: https://doi.org/10.1002/jgrd.50840<br>
      • CLASS Assigned OID: ATMS_SDR<br>
      • Configuration Management Plan [Zhao, 2014] available online here: https://www.star.nesdis.noaa.gov/jpss/Docs.php<br><br>
      Comments:<br>
      No comments.
    </td>
  </tr>
  <tr>
    <td>Level 5<br>
      • CLASS offers data signatures option for downloads and checksums available.<br>
      • File level checksums are verified at ingest and at archive.<br>
      • CLASS maintains a copy of the checksum for validation during dissemination and that checksum is used for validation during the staging process when an order is fulfilled.<br>
      • Discrepancies identified in checksum validation are automatically reported to CLASS team for investigative purposes.<br><br>
      Comments:<br>
      No comments.
    </td>
  </tr>
  <tr>
    <td>Data Integrity</td>
    <td></td>
  </tr>
</table>
3. Acknowledgment

This work is supported by the NOAA OneStop Project.

We thank the dataset POCs for their valuable input, as well as the collaborative efforts of the OneStop teams, especially the Metadata team. We would also like to show appreciation to Ge Peng for her contributions.

The draft of this data stewardship maturity report is systematically generated by a tool created by Kieran Hodnett and populated with the stewardship maturity assessment done by the author(s) of this report. The tool was developed based on a Word template created collaboratively by Robert Partee II, Raisa Ionin, Paul Lemieux III, Ge Peng, Don Collins, and Sonny Zinn with helpful input from the NOAA Central Library and the NCEI Communication Team.
4. References

Casey, K. (2016), The NOAA OneStop data discover and access framework project, Version:June 3, 2016. https://cdn.ioos.noaa.gov/media/2017/12/OneStop-IOOS-DMAC-03-June-2016.pdf

Peng, G. (2015) The scientific data stewardship maturity assessment model template, Version: NCDC-CICS-SMM-0001-Rev.1 v4.0 6/23/2015. doi:10.6084/m9.figshare.1211954.

Peng, G., J.L. Privette, E.J. Kearns, N.A. Ritchey, and S. Ansari (2015), A unified framework for measuring stewardship practices applied to digital environmental datasets, Data Science Journal, 13, 231-253, doi: 10.2481/dsj.14-049.

Peng, G., J. Lawrimore, V. Toner, C. Lief , R. Baldwin, N. Ritchey, D. Brinegar, and S. A. Delgreco (2016) assessing stewardship naturity of the global historical climatology network-monthly (GHCN-M) dataset: use case study and lessons learned, D-Lib Magazine, 22, doi:10.1045/november2016-peng.

Chu, R., and Northrop Grumman, (2011), Joint Polar Satellite System (JPSS) Advanced Technology Microwave Sounder (ATMS) SDR radiometric calibration Algorithm Theoretical Basis Document (ATBD), _Rep. 474-00043_, NASA Goddard Space Flight Center, Greenbelt, MD., retrieved online: https://www.star.nesdis.noaa.gov/jpss/Docs.php (Accessed 19 January 2017).

Mikles, V., and Liu, X., (2016), STAR JPSS quality assurance plan, NOAA Center for Weather and Climate Prediction (NCWCP), College Park, MD., retrieved online: https://www.star.nesdis.noaa.gov/jpss/Docs.php (Accessed 24 January 2017).

Weng, F., Zou, X., Tian, M., Blackwell, W., Sun, N., Yang, H., Wang, X., Lin, L., Anderson, K., (2013), Calibration of Suomi National Polar-orbiting Partnership (NPP) Advanced Technology Microwave Sounder (ATMS), _Journal of Geophysical Research Atmospheres_, 118(19), 11,187—11,200, doi:10.1002/jgrd.50840.

Weng, F., Zou, X., Wang, X., Yang, S., Goldberg, M., (2012), Introduction to Suomi NPP ATMS for NWP and tropical cyclone applications, _Journal of Geophysical Research Atmospheres_, 117, D19112, doi:10.1029/2012JD018144.
Zhao, Y., (2014) STAR JPSS algorithms integration team configuration management plan, NOAA Center for Satellite Applications and Research (STAR), College Park, MD., retrieved online: https://www.star.nesdis.noaa.gov/jpss/Docs.php (Accessed 06 January 2017).
Appendix I: The Scientific Data Stewardship Maturity Matrix (DSMM)

Table A1: This matrix (Version: NCDC-CICS-SMM-0001-Rev.1. 12/09/2014) describes the criterion used to evaluate data stewardship maturity for each of the nine DSMM key components [Peng et al., 2015].

<table>
  <tr>
    <th>DSMM Component</th>
    <th>Level 1<br><i>Ad hoc</i><br>Little or no management</th>
    <th>Level 2<br><i>Minimal</i><br>Limited management</th>
    <th>Level 3<br><i>Intermediate</i><br>Defined management, partially implemented</th>
    <th>Level 4<br><i>Advanced</i><br>Well-defined management, fully implemented</th>
    <th>Level 5<br><i>Optimal</i><br>Full management, audited, measured, controlled</th>
  </tr>
  <tr>
    <td rowspan="2"><i>Preservability</i><br>(<i>The state of being preservable</i>)</td>
    <td>Any storage location<br>Data only</td>
    <td>Non-designated repository<br>Redundancy<br>Limited archiving metadata</td>
    <td>Designated archive<br>Redundancy<br>Community-standard archiving metadata<br>Conforming to limited archiving standards</td>
    <td>Level 3 +<br>Conforming to community archiving standards</td>
    <td>Level 4 +<br>Archiving process performance controlled, measured, and audited<br>Future archiving standard changes planned</td>
  </tr>
  <tr>
    <td></td>
    <td></td>
    <td></td>
    <td></td>
    <td></td>
  </tr>
  <tr>
    <td rowspan="2"><i>Accessibility</i><br>(<i>The state of being searchable and accessible publicly</i>)</td>
    <td>Not publically available person-to-person</td>
    <td>Publically available direct file download (e.g., via anonymous FTP server)<br>Collection or dataset level searchable online</td>
    <td>Level 2 +<br>Non-standard data service<br>Limited data server performance<br>Granule/file level searchable<br>Limited search metrics</td>
    <td>Level 3 +<br>Community-standard data service<br>Enhanced data server performance<br>Conforming to community search metrics<br>Dissemination report metrics defined and implemented internally</td>
    <td>Level 4 +<br>Dissemination reports available online<br>Future technology and standard changes planned</td>
  </tr>
  <tr>
    <td></td>
    <td></td>
    <td></td>
    <td></td>
    <td></td>
  </tr>
</table>
<table>
  <tr>
    <th><i>Usability<br>(The state of being easy to use)</i></th>
    <td>Extensive product-specific knowledge required<br>No documentation online</td>
    <td>Non-standard data format<br>Limited documentation (e.g., user’s guide online)</td>
    <td>Community standard-based interoperable format & metadata<br>Documentation (e.g. source code, product algorithm document, processing or/and data flow diagram) online</td>
    <td>Level 3 +<br>Basic capability (e.g., subsetting, aggregating) & data characterization overall/global, e.g., climatology, error estimates) available online</td>
    <td>Level 4 +<br>Enhanced online capability (e.g., visualization, multiple data formats)<br>Community metrics of data characterization (regional/cell) online<br>External ranking</td>
  </tr>
  <tr>
    <th><i>Production Sustainability<br>(The state of data production being sustainable and extendable)</i></th>
    <td>Ad Hoc or Not applicable<br>To obligation or deliverable requirement</td>
    <td>Short-term Individual PI’s commitment (grant obligations)</td>
    <td>Medium-term Institutional commitment (contractual deliverables with specs and schedule defined)</td>
    <td>Long-term Institutional commitment<br>Product improvement process in place</td>
    <td>Level 4 +<br>National or international commitment<br>Changes for technology planned</td>
  </tr>
  <tr>
    <th><i>Data Quality Assurance<br>(The state of data quality being assured)</i></th>
    <td>Data quality assurance (DQA) procedure unknown or none</td>
    <td>Ad Hoc and random<br>QA procedure not defined and documented</td>
    <td>DQA procedure defined and documented and partially implemented</td>
    <td>DQA procedure well documented, fully implemented and available online with master reference data<br>Limited data quality assurance metadata</td>
    <td>Level 4 +<br>DQA procedure monitored and reported<br>Conforming to community quality metadata & standards<br>External review</td>
  </tr>
</table>
<table>
  <tr>
    <th>Data Quality Control/ Monitoring<br><i>The state of data quality being controlled and monitored</i></th>
    <th>None or Sampling unknown or spotty<br>Analysis unknown or random in time</th>
    <th>Sampling and analysis are regular in time and space<br>Limited product-specific metrics defined & implemented</th>
    <th>Level 2 +<br>Sampling and analysis are frequent and systematic but not automatic<br>Community metrics defined and partially implemented<br>Procedure documented and available online</th>
    <th>Level 3 +<br>Anomaly detection procedure well-documented and fully implemented using community metrics, automatic, tracked and reported<br>Limited quality monitoring metadata</th>
    <th>Level 4 +<br>Cross-validation of temporal & spatial characteristics<br>Physical consistency check<br>Conforming to community quality metadata & standards</th>
  </tr>
  <tr>
    <th>Data Quality Assessment<br><i>(The state of data quality being assessed)</i></th>
    <th>Algorithm/ method/model<br>Theoretical basis assessed (methods and results online)</th>
    <th>Level 1 +<br>Research product assessed (methods and results online)</th>
    <th>Level 2 +<br>Operational product assessed (methods and results online)</th>
    <th>Level 3 +<br>Quality metadata assessed<br>Limited quality assessment metadata</th>
    <th>Level 4 +<br>Assessment performed on a recurring basis<br>Conforming to community quality metadata & standards<br>External ranking</th>
  </tr>
  <tr>
    <th>Transparency/ Traceability<br><i>(The state of being transparent, trackable, and traceable)</i></th>
    <th>Limited product information available<br>Person-to-person</th>
    <th>Product information available in literature</th>
    <th>Algorithm Theoretical Basis Document (ATBD) & source code online<br>Dataset configuration managed (CM)<br>Unique Object Identifier (OID) assigned (dataset, documentation, source code)<br>Data citation tracked (e.g., utilizing Digital Object Identifier</th>
    <th>Level 3 +<br>Operational Algorithm Description (OAD) online, OID assigned, and under CM</th>
    <th>Level 4 +<br>System information online<br>Complete data provenance online</th>
  </tr>
</table>
<table>
  <tr>
    <th>Data Integrity<br>(The state of data integrity being verifiable)</th>
    <th>Unknown or no data ingest integrity check</th>
    <th>Data ingest integrity verifiable (e.g., checksum technology)</th>
    <th>(DOI) system Level 2 + Data archive integrity verifiable</th>
    <th>Level 3 + Data access integrity verifiable<br>Conforming to community data integrity technology standard</th>
    <th>Level 4 +<br>Data authenticity verifiable (e.g., data signature technology)<br>Performance of data integrity check monitored and reported</th>
  </tr>
</table>
Table 1 Legend
Level 1	Level 2	Level 3	Level 4	Level 5
Ad Hoc	Minimal	Intermediate	Advanced	Optimal
Little or no management	Limited Management	Defined Management, partially implemented	Well-defined Management, fully implemented	Full Management, audited, measured, controlled
Table 1. Scores for the Nine DSMM Key Components at a Glance
Preservability - 4.5	Accessibility - 5	Usability - 4.5
Production Sustainability - 5	Data Quality Assurance - 4	Data Quality Control/Monitoring - 3.5
Data Quality Assessment - 3	Transparency/Traceability - 4	Data Integrity - 5
Table 2. Dataset and Data Stewardship Maturity Assessment Metadata
Dataset Title	NOAA JPSS Advanced Technology Microwave Sounder (ATMS) Sensor Data Record (SDR) from IDPS
Dataset Information URL	https://doi.org/10.7289/V5BR8Q7N
Data Provider POC (Name; Email; Affiliation)	Customer Engagement Branch, ncei.sat.info@noaa.gov, DOC/NOAA/NESDIS/NCEI > National Centers for Environmental Information, NESDIS, NOAA, U.S. Department of Commerce
Dataset POC (Name; Email; Affiliation)	Fuzhong Weng, Fuzhong.Weng@noaa.gov, DOC/NOAA/NESDIS/STAR > Center for Satellite Applications and Research, NESDIS, NOAA, U.S. Department of Commerce
SMM Version (Document ID and Version Number)	NCDC-CICS-SMM_0001_Rev.1 12/09/2014
SMM POC (Name; E-mail; Affiliation)	Ge Peng, ge.peng@uah.edu, University of Alabama-Huntsville
SMM Template Version (Document ID and Version Numbers)	NCDC-CICS-SMM_0001_Rev.1 v4.0 06/23/2015
SMM Template POC	Ge Peng, ge.peng@uah.edu, University of Alabama-Huntsville
SMM Assessment Version (v=xn>r=mm>, e.g., v01r00)	v01r05
SMM Assessment Date (MM/DD/YYYY)	03/08/2017
SMM Assessment POC (Name; E-mail; Affiliation)	Paul Lemieux III, paul.lemieux@noaa.gov, Earth Resources Technology, Inc.
Stewardship Maturity Ratings (each key component) (kc1/kc2/kc3/kc4/kc5/kc6/kc7/kc8/kc9)	4.5 / 5 / 4.5 / 5 / 4 / 3.5 / 3 / 4 / 5
SMM Original Assessment Date (MM/DD/YYYY)	07/14/2016
SMM Original Assessment POC (Name; E-mail; Affiliation)	Paul Lemieux III, paul.lemieux@noaa.gov, Earth Resources Technology, Inc.
SMM Last Modified Date (MM/DD/YYYY)	10/07/2021
SMM Last Modification POC (Name; E-mail; Affiliation)	Lori Hager, lori.hager@noaa.gov , CASE Consultants International
SMM Modified Date (MM/DD/YYYY)	04/24/2019
SMM Modification POC (Name; E-mail; Affiliation)	Paul Lemieux III, paul.lemieux@noaa.gov, Riverside Technology, Inc.
DSMM Key Component	Stewardship Maturity Rating, Justification, and Comments
Preservability	Level 4.5 • Archived by NCEI which is a NOAA designated archive compliant to NARA standards. • Metadata following ISO 19115-2. • Compliant to OIAS RM. • Plans to update metadata to ISO 19115-1 at a later date and may be a pilot dataset for the OneStop initiative. • Using CLASS archiving system which is CMMI-Level 3 compliant. Comments: No known audits on the archiving processes
Accessibility	Level 5 • Collection level searchable online • Direct file download available CLASS FTP: ftp://ftp-npp.bou.class.noaa.gov • CLASS ordering: https://www.avl.class.noaa.gov/saa/products/search?sub_id=0&datatype_family=ATMS_SDR • Granules searchable and orderable via CLASS. • CLASS has dissemination reports available internally and externally. Users have to e-mail the CLASS Help Desk to request access to the metrics tools. • New technology for OneStop search and discovery planned (i.e. ElasticSearch, Hyrax Servers, etc.) This is part of the JPSS/NPP data group that will be enhanced for OneStop readiness. Comments: A CLASS improvement (GEARS) is in the works.
Usability	Level 4.5 • Satellite user community standard HDF5 data format. • ATBD [Chu & Northrop Grumman, 2011] and other supporting documentation available online here: https://www.star.nesdis.noaa.gov/jpss/Docs.php • Data visualization capability via JPSS Long Term Monitoring (LTM) system: https://www.star.nesdis.noaa.gov/icvs/status_NPP_ATMS.php • CLASS has deaggregating/subsetting options for downloads. Comments: No known external rankings.
Production Sustainability	Level 5 • Product improvement process in place, product has reached validated maturity level 1 and there are plans to advance it to level 2. Product may transition from IDPS to the newer NDE processing system. • Long-term Institutional commitment through NOAA/JPSS in place. • National and international commitment from Numerical Weather Prediction (NWP) centers, NCEP, Navel Research Lab (NRL), UK MetOffice, and ECMWF. Comments: No comments.
DSMM Key Component	Stewardship Maturity Rating, Justification, and Comments
Data Quality Assurance	Level 4 • Product maturity document [Weng, 2013] available online that outlines the rigorous QA/QC controls implemented on the product: https://www.star.nesdis.noaa.gov/jpss/AlgorithmMaturity.php • STAR JPSS Quality Assurance Plan [Mikles and Liu, 2016] well documented and fully implemented. • Quality flags included at the file level that can be considered limited data quality assurance metadata Comments: No known external reviews on data quality metadata.
Data Quality Control/Monitoring	Level 3.5 • Data quality control and monitoring steps are available online per the JPSS validation requirements. Anomaly detection procedure, reporting, and remedies are also available as a requirement for the validated (operational) maturity phase. See the CalVal presentation [Weng, 2013] Weng, F., (2013), Suomi NPP ATMS CalVal overview [PowerPoint slides], retrieved from: https://www.star.nesdis.noaa.gov/jpss/AlgorithmMaturity.php (Accessed 19 January 2017).] available online here: https://www.star.nesdis.noaa.gov/jpss/AlgorithmMaturity.php • Long term data quality monitoring application available to the public: https://www.star.nesdis.noaa.gov/icvs/status_NPP_ATMS.php Comments: No data quality information in the metadata record. I’m not sure how users provide feedback, but there are several instances of user feedback given in the maturity review documents so there must a system in place to do it. I think users can e-mail team leads directly at STAR.
Data Quality Assessment	Level 3 • ATBD [Chu and Northrop Grumman, 2011] available online here: https://www.star.nesdis.noaa.gov/jpss/Docs.php • Research product assessed in literature [Weng, Yang, Tian, et al., 2013] available online here: https://doi.org/10.1002/jgrd.50840 • Operational assessment available in literature [Weng, Zou, Wang, et al., 2017] available online here: https://doi.org/10.1029/2012JD018144 Comments: No data quality information in the metadata record and no metadata assessment. No known external ranking
DSMM Key Component	Stewardship Maturity Rating, Justification, and Comments
Transparency / Traceability	Level 4 • ATBD [Chu and Northrop Grumman, 2011] available here: https://www.star.nesdis.noaa.gov/jpss/Docs.php • OAD [Guenther, Gu, Siebels, et al., 2015] available online here: • https://www.jpss.noaa.gov/sciencedocs.html • DOI: 10.7289/V5BR8Q7N • Product information available in literature [Weng, Yang, Tian, et al., 2013]: https://doi.org/10.1002/jgrd.50840 • CLASS Assigned OID: ATMS_SDR • Configuration Management Plan [Zhao, 2014] available online here: https://www.star.nesdis.noaa.gov/jpss/Docs.php Comments: No comments.
Transparency / Traceability	Level 5 • CLASS offers data signatures option for downloads and checksums available. • File level checksums are verified at ingest and at archive. • CLASS maintains a copy of the checksum for validation during dissemination and that checksum is used for validation during the staging process when an order is fulfilled. • Discrepancies identified in checksum validation are automatically reported to CLASS team for investigative purposes. Comments: No comments.
Data Integrity
DSMM Component	Level 1 Ad hoc Little or no management	Level 2 Minimal Limited management	Level 3 Intermediate Defined management, partially implemented	Level 4 Advanced Well-defined management, fully implemented	Level 5 Optimal Full management, audited, measured, controlled
Preservability (The state of being preservable)	Any storage location Data only	Non-designated repository Redundancy Limited archiving metadata	Designated archive Redundancy Community-standard archiving metadata Conforming to limited archiving standards	Level 3 + Conforming to community archiving standards	Level 4 + Archiving process performance controlled, measured, and audited Future archiving standard changes planned
Preservability (The state of being preservable)
Accessibility (The state of being searchable and accessible publicly)	Not publically available person-to-person	Publically available direct file download (e.g., via anonymous FTP server) Collection or dataset level searchable online	Level 2 + Non-standard data service Limited data server performance Granule/file level searchable Limited search metrics	Level 3 + Community-standard data service Enhanced data server performance Conforming to community search metrics Dissemination report metrics defined and implemented internally	Level 4 + Dissemination reports available online Future technology and standard changes planned
Usability (The state of being easy to use)	Extensive product-specific knowledge required No documentation online	Non-standard data format Limited documentation (e.g., user’s guide online)	Community standard-based interoperable format & metadata Documentation (e.g. source code, product algorithm document, processing or/and data flow diagram) online	Level 3 + Basic capability (e.g., subsetting, aggregating) & data characterization overall/global, e.g., climatology, error estimates) available online	Level 4 + Enhanced online capability (e.g., visualization, multiple data formats) Community metrics of data characterization (regional/cell) online External ranking
Production Sustainability (The state of data production being sustainable and extendable)	Ad Hoc or Not applicable To obligation or deliverable requirement	Short-term Individual PI’s commitment (grant obligations)	Medium-term Institutional commitment (contractual deliverables with specs and schedule defined)	Long-term Institutional commitment Product improvement process in place	Level 4 + National or international commitment Changes for technology planned
Data Quality Assurance (The state of data quality being assured)	Data quality assurance (DQA) procedure unknown or none	Ad Hoc and random QA procedure not defined and documented	DQA procedure defined and documented and partially implemented	DQA procedure well documented, fully implemented and available online with master reference data Limited data quality assurance metadata	Level 4 + DQA procedure monitored and reported Conforming to community quality metadata & standards External review
Data Quality Control/ Monitoring The state of data quality being controlled and monitored	None or Sampling unknown or spotty Analysis unknown or random in time	Sampling and analysis are regular in time and space Limited product-specific metrics defined & implemented	Level 2 + Sampling and analysis are frequent and systematic but not automatic Community metrics defined and partially implemented Procedure documented and available online	Level 3 + Anomaly detection procedure well-documented and fully implemented using community metrics, automatic, tracked and reported Limited quality monitoring metadata	Level 4 + Cross-validation of temporal & spatial characteristics Physical consistency check Conforming to community quality metadata & standards
Data Quality Assessment (The state of data quality being assessed)	Algorithm/ method/model Theoretical basis assessed (methods and results online)	Level 1 + Research product assessed (methods and results online)	Level 2 + Operational product assessed (methods and results online)	Level 3 + Quality metadata assessed Limited quality assessment metadata	Level 4 + Assessment performed on a recurring basis Conforming to community quality metadata & standards External ranking
Transparency/ Traceability (The state of being transparent, trackable, and traceable)	Limited product information available Person-to-person	Product information available in literature	Algorithm Theoretical Basis Document (ATBD) & source code online Dataset configuration managed (CM) Unique Object Identifier (OID) assigned (dataset, documentation, source code) Data citation tracked (e.g., utilizing Digital Object Identifier	Level 3 + Operational Algorithm Description (OAD) online, OID assigned, and under CM	Level 4 + System information online Complete data provenance online