Iterative Error‐Driven Ensemble Labeling (IEDEL) Algorithm for Enhanced Data Quality Control for the Atmospheric Radiation Measurement (ARM) Program User Facility

Li, Lishan; Kehoe, Kenneth E.; Hu, Jiaxi; Peppler, Randy A.; Sockol, Alyssa J.; Godine, Corey A.

doi:10.1029/2024jh000192

Advanced Search

Select up to three search categories and corresponding keywords using the fields to the right. Refer to the Help section for more detailed instructions.

Search our Collections & Repository

Advanced Search
Custom Query

All these words:

For very narrow results

This exact word or phrase:

When looking for a specific result

Any of these words:

Best used for discovery & interchangable words

None of these words:

Recommended to be used in conjunction with other fields

Language:

Dates

Publication Date Range:

to

Document Data

Title:

Document Type:

Library

Collection:

Series:

People

Author:

Clear All

Query Builder

Query box

Clear All

For additional assistance using the Custom Query please check out our Help Page

The NOAA IR serves as an archival repository of NOAA-published products including scientific findings, journal articles, guidelines, recommendations, or other information authored or co-authored by NOAA or funded partners. As a repository, the NOAA IR retains documents in their original published format to ensure public access to scientific information.

i

Iterative Error‐Driven Ensemble Labeling (IEDEL) Algorithm for Enhanced Data Quality Control for the Atmospheric Radiation Measurement (ARM) Program User Facility

2024
By Li, Lishan ; Kehoe, Kenneth E. ; Hu, Jiaxi ; ...
Source: Journal of Geophysical Research: Machine Learning and Computation, 1(3)

[PDF-4.30 MB]

Details You May Also Like

Details:

Journal Title:

Journal of Geophysical Research: Machine Learning and Computation
Personal Author:

Li, Lishan ; Kehoe, Kenneth E. ; Hu, Jiaxi ; Peppler, Randy A. ; Sockol, Alyssa J. ; ... More +

Li, Lishan ; Kehoe, Kenneth E. ; Hu, Jiaxi ; Peppler, Randy A. ; Sockol, Alyssa J. ; Godine, Corey A. Less -
NOAA Program & Office:

OAR (Oceanic and Atmospheric Research) ; CIWRO (Cooperative Institute for Severe and High-Impact Weather Research and Operations) ; NSSL (National Severe Storms Laboratory)

OAR (Oceanic and Atmospheric Research) ; CIWRO (Cooperative Institute for Severe and High-Impact Weather Research and Operations) ; NSSL (National Severe Storms Laboratory) Less -
Description:

For over three decades, the Atmospheric Radiation Measurement (ARM) Program user facility has provided researchers with invaluable benchmark atmospheric data. Ensuring the accuracy and integrity of ARM data is vital, and to achieve this, the ARM Data Quality Office (DQO) has implemented customized quality control tests tailored to each variable, with guidance from instrument mentors. These tests are designed to pinpoint common issues, such as data exceeding valid ranges or persisting with little change over extended periods, and ARM offers tools for users to review and exclude contaminated data efficiently. However, certain quality issues, such as spikes in time series or data drift over time, sometimes evade detection by existing tests and require manual identification by data analysts and instrument mentors through visualization tools. To tackle these challenges more efficiently, the DQO has developed and implemented the Iterative Error‐Driven Ensemble Labeling (IEDEL) algorithm with unanimous voting and transfer learning techniques to efficiently generate labeled data at scale. This initiative has empowered the creation of high‐performing machine learning models, enabling real‐time monitoring of data quality issues within the ARM data and thereby enhancing data integrity and accessibility.
Source:

Journal of Geophysical Research: Machine Learning and Computation, 1(3)
DOI:

https://doi.org/10.1029/2024jh000192
ISSN:

2993-5210;2993-5210;
Format:

pdf
Publisher:

American Geophysical Union (AGU)
Document Type:

Journal Article
Funding:

Grant no. NA21OAR4320204
License:

https://creativecommons.org/licenses/by/4.0/
Rights Information:

CC BY
Compliance:

Library
Main Document Checksum:

[+]

urn:sha-512:ae6e27b9731985bbfc94667d1f5caea399e7b44d4de371f4d9106ecef009a7b9c5cad6dfa3614d7d5c71b5d5e6c06e0924710405b86bff3e906fde40de6e19b3
Download URL:

https://repository.library.noaa.gov/view/noaa/67919/noaa_67919_DS1.pdf
File Type:

No Additional Files

More +

Iterative Error‐Driven Ensemble Labeling (IEDEL) Algorithm for Enhanced Data Quality Control for the Atmospheric Radiation Measurement (ARM) Program User Facility

Details:

Supporting Files

You May Also Like