WaterGAN: Unsupervised Generative Network to Enable Real-time Color Correction of Monocular Underwater Images

Details

• Journal Title:
  IEEE Robotics and Automation Letters
• Personal Author:
  Li, Jie ; Skinner, Katherine A. ; Eustice, Ryan M. ; Johnson-Roberson, Matthew
• NOAA Program & Office:
  OAR (Oceanic and Atmospheric Research)
• Description:
  This paper reports on WaterGAN, a generativeadversarial network (GAN) for generating realistic underwaterimages from in-air image and depth pairings in an unsupervisedpipeline used for color correction of monocular underwaterimages. Cameras onboard autonomous and remotely operatedvehicles can capture high resolution images to map the seafloor;however, underwater image formation is subject to the complexprocess of light propagation through the water column. The rawimages retrieved are characteristically different than imagestaken in air due to effects such as absorption and scattering,which cause attenuation of light at different rates for differentwavelengths. While this physical process is well describedtheoretically, the model depends on many parameters intrinsicto the water column as well as the structure of the scene. Thesefactors make recovery of these parameters difficult withoutsimplifying assumptions or field calibration; hence, restorationof underwater images is a non-trivial problem. Deep learninghas demonstrated great success in modeling complex nonlinearsystems but requires a large amount of training data, which isdifficult to compile in deep sea environments. Using WaterGAN,we generate a large training dataset of corresponding depth,in-air color images, and realistic underwater images. This dataserves as input to a two-stage network for color correctionof monocular underwater images. Our proposed pipeline isvalidated with testing on real data collected from both a purewater test tank and from underwater surveys collected in thefield. Source code, sample datasets, and pretrained models aremade publicly available.
• Keywords:
  Generative Adversarial Network Image Restoration Monocular Vision Underwater Vision
• Source:
  IEEE Robotics and Automation Letters
• DOI:
  https://doi.org/10.1109/LRA.2017.2730363
• Document Type:
  Journal Article
• Funding:
  Grant no. NA14OAR0110265
• Rights Information:
  Accepted Manuscript
• Rights Statement:
  The NOAA IR provides access to this content under the authority of the government's retained license to distribute publications and data resulting from federal funding. While users may legally access this content, the copyright owners retain rights that govern the reproduction, redistribution, and re-use of this work. The user is solely responsible for complying with applicable copyright law.
• Compliance:
  Submitted
• Main Document Checksum:
  urn:sha-512:53160f3ac452caf2907b738e81fcaa17ee80539e47b047b9c8b7719b8f856771ca7cea715cb954fc445498875d37281b0098d46ace46bbe312b7e7b720547a15
• Download URL:
  https://repository.library.noaa.gov/view/noaa/32890/noaa_32890_DS1.pdf
• File Type:
  [PDF - 4.97 MB ]