i
Accessibility of Big Data Imagery for Next Generation Machine Learning Applications
-
2019
-
Details:
-
Personal Author:
-
Corporate Authors:
-
NOAA Program & Office:
-
Description:NOAA generates tens of terabytes of data a day, also known as “big data” from satellites, radars, ships, weather models, optical technologies, and other sources. This unprecedented growth of data collection in recent years has resulted from enhanced sampling technologies and faster computer processing. While these data are publicly available, there is not yet sufficient access to the data by next generation processing technologies, such as machine learning (ML) algorithms that are able to improve processing efficiencies. Accessibility is the key component for utilizing analytical tools and ensuring our processing meets 21st century data needs. This report focuses on the challenges of accessibility of imagery (defined as still images and video) from the marine environment. Vast amounts of imagery are collected from optical technologies used in marine ecosystem monitoring and ocean observation programs. While technologies have dramatically increased the spatial and temporal resolution of data and increased our understanding of marine ecosystems, the drastic increase in big data, specifically imagery, presents numerous challenges. Case studies discussed in this report highlight that big data imagery are readily being collected and stored, yet the foundation for the long term storage and accessibility of big data must be based on the necessary guidance for its architecture, infrastructure, and applications to enhance the accessibility and use of these data to help fulfill NOAA’s cross-functional missions. Additionally, the report highlights key considerations and recommendations for NOAA’s data modernization efforts that align with mandates such as Public Access to Research Results, the Evidence-Based Policy Making Act, Department of Commerce Strategic Plan, the President’s Management Agenda, and White House Executive Order on Artificial Intelligence (AI). As big data and analytical tools become more commonplace for NOAA’s research and scientific operations, there is an increasing need to create end-to-end data management practices that improve data accessibility for analytical tools that utilize AI, computer vision (AI applied to the visual world), and ML. The development and application of AI and ML analytics will progress as long as there is accessibility of big data with enriched metadata; however, accessibility appears to be the primary challenge to fully utilize ML analytics. Rapid, optimal access to entire imagery and data collections is critical to create annotated imagery libraries for supervised analysis using ML algorithms. This report highlights the common need to implement accessibility solutions to facilitate efficient imagery processing using available analytical tools.
-
Keywords:
-
Series:
-
Document Type:
-
Rights Information:Public Domain
-
Compliance:Submitted
-
Main Document Checksum:
-
Download URL:
-
File Type: