The NOAA IR serves as an archival repository of NOAA-published products including scientific findings, journal articles, guidelines, recommendations, or other information authored or co-authored by NOAA or funded partners.
As a repository, the NOAA IR retains documents in their original published format to ensure public access to scientific information.
i
Small values in big data: The continuing need for appropriate metadata
-
2018
-
-
Source: Ecological Informatics, 45, 26-30
[PDF-710.44 KB]
Details:
-
Journal Title:Ecological Informatics
-
Personal Author:
-
NOAA Program & Office:
-
Description:Compiling data from disparate sources to address pressing ecological issues is increasingly common. Many ecological datasets contain left-censored data – observations below an analytical detection limit. Studies from single and typically small datasets show that common approaches for handling censored data — e.g., deletion or substituting fixed values — result in systematic biases. However, no studies have explored the degree to which the documentation and presence of censored data influence outcomes from large, multi-sourced datasets. We describe left-censored data in a lake water quality database assembled from 74 sources and illustrate the challenges of dealing with small values in big data, including detection limits that are absent, range widely, and show trends over time. We show that substitutions of censored data can also bias analyses using 'big data' datasets, that censored data can be effectively handled with modern quantitative approaches, but that such approaches rely on accurate metadata that describe treatment of censored data from each source.
-
Source:Ecological Informatics, 45, 26-30
-
DOI:
-
ISSN:1574-9541
-
Format:
-
Publisher:
-
Document Type:
-
Rights Information:Accepted Manuscript
-
Rights Statement:The NOAA IR provides access to this content under the authority of the government's retained license to distribute publications and data resulting from federal funding. While users may legally access this content, the copyright owners retain rights that govern the reproduction, redistribution, and re-use of this work. The user is solely responsible for complying with applicable copyright law.
-
Compliance:Library
-
Main Document Checksum:
-
Download URL:
-
File Type:
