The NOAA IR serves as an archival repository of NOAA-published products including scientific findings, journal articles, guidelines, recommendations, or other information authored or co-authored by NOAA or funded partners.
As a repository, the NOAA IR retains documents in their original published format to ensure public access to scientific information.
i
Increasing the Reproducibility and Replicability of Supervised AI/ML in the Earth Systems Science by Leveraging Social Science Methods
-
2024
-
-
Source: Earth and Space Science, 11(7)
[PDF-2.62 MB]
Details:
-
Journal Title:Earth and Space Science
-
Personal Author:
-
NOAA Program & Office:
-
Description:Artificial intelligence (AI) and machine learning (ML) pose a challenge for achieving science that is both reproducible and replicable. The challenge is compounded in supervised models that depend on manually labeled training data, as they introduce additional decision‐making and processes that require thorough documentation and reporting. We address these limitations by providing an approach to hand labeling training data for supervised ML that integrates quantitative content analysis (QCA)—a method from social science research. The QCA approach provides a rigorous and well‐documented hand labeling procedure to improve the replicability and reproducibility of supervised ML applications in Earth systems science (ESS), as well as the ability to evaluate them. Specifically, the approach requires (a) the articulation and documentation of the exact decision‐making process used for assigning hand labels in a “codebook” and (b) an empirical evaluation of the reliability” of the hand labelers. In this paper, we outline the contributions of QCA to the field, along with an overview of the general approach. We then provide a case study to further demonstrate how this framework has and can be applied when developing supervised ML models for applications in ESS. With this approach, we provide an actionable path forward for addressing ethical considerations and goals outlined by recent AGU work on ML ethics in ESS.
-
Source:Earth and Space Science, 11(7)
-
DOI:
-
ISSN:2333-5084;2333-5084;
-
Format:
-
Publisher:
-
Document Type:
-
License:
-
Rights Information:CC BY-NC-ND
-
Compliance:Library
-
Main Document Checksum:
-
Download URL:
-
File Type:
