低照度图像处理难点：

增强微光图像并不是一件容易的事，因为噪声通常很容易被放大

发展历史

直方图均衡化（Histogram Equalization, HE）

直方图均衡化通过扩展动态范围来增强低照度图像，从而提供意外放大的密集噪声的不良照明。

  • 优点：采用传统算法，速度快。
  • 缺点：最早的低照度增强方法是均匀调整光照，容易造成曝光过度和曝光不足，如直方图均衡化(HE)。在没有局部自适应的情况下，增强会导致不希望的光照和强烈的噪声。一些方法通过对倒置的微光图像应用去模糊方法来增强可见度。在这些方法中，应用离线去噪操作来抑制噪声，这有时也会导致细节模糊。

    Retinex-2018【code】

    基于Retinex理论的方法分别分解和处理图像的两个层次，反射层和照明层。
    基于Retinex的方法通过将图像分解成照明层和反射层并自适应地调整它们来执行联合照明调整和噪声抑制。利用各种先验(例如结构感知先验、加权变化和多个照度导数)来指导这两层的操作。开发了Retinex模型的变量，例如单尺度Retinex、多尺度Retinex、自然度Retinex和鲁棒Retinex，以促进微光图像增强。这些方法在光照调节和小噪声去除方面取得了令人印象深刻的效果。然而，由于只有手工制作的约束，这些方法的适应性不够强，而且它们的结果显示出强烈的噪声，有时曝光不足和曝光过多的局部细节。

    EnlightenGAN-2021【code】

    EnlighttenGAN，其中只需要包含低照度/普通光图像(不必要的配对)的数据集。该方法证明了利用非配对数据学习进行低照度增强的可行性。然而，如果没有配对的监督，精细的细节就无法恢复，强化效果中密集的噪音仍然存在。

基于深度学习的低照度图像增强技术带来了令人印象深刻的性能提升。

  • Loreet等人进行了第一次尝试，提出了一种名为低照度网络(LLNet)的深度自动编码器，用于增强对比度和消除噪声。随后，针对不同的网络设计提出了各种方法。这些方法都是在配对数据集上训练的，它们的增强性能在很大程度上取决于数据集。由于合成数据不能完全刻画真实场景中的退化特征，而真实捕获的配对数据包括有限种类的场景，这些方法的结果仍然不完善，特别是不能处理密集的噪声。
  • 也有一些工作是基于深度学习的原始图像增强，或者是微光图像增强和高级计算机视觉任务的联合任务，如人脸检测、目标检测等。

深度递归频带网络(DRBN)-2020

主要关注RGB格式的低照度图像的感知质量的提高。与以往的相关研究不同，其开发了一个半监督框架，在该框架中，来自配对和非配对数据集的有用知识被联合使用来为细节信号建模和全局光照、颜色和对比度恢复提供感知指导。
作者提供了pytorch的代码

Datasets

Papers

2021

  • RUAS [Web] [Pdf] [Code]
    • Liu, Risheng, Long Ma, Jiaao Zhang, Xin Fan, and Zhongxuan Luo. “Retinex-Inspired Unrolling with Cooperative Prior Architecture Search for Low-Light Image Enhancement.” In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 10561–70, 2021.
    • 🔖 retinex
  • Deep denoising of flash and no-flash pairs for photography in low-light environments [Pdf]
    • Zhihao Xia, Michael Gharbi, Federico Perazzi, Kalyan Sunkavalli, and Ayan Chakrabarti. “Deep Denoising of Flash and No-Flash Pairs for Photography in Low-Light Environments.” In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2063–72, 2021.
  • HORUS [Pdf]
    • Moseley, Ben, Valentin Bickel, Ignacio G. Lopez-Francos, and Loveneesh Rana. “Extreme Low-Light Environment-Driven Image Denoising over Permanently Shadowed Lunar Regions with a Physical Noise Model.” In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 6317–27, 2021.
  • Learning temporal consistency for low light video enhancement from single images [Pdf] [Code]
    • Zhang, Fan, Yu Li, Shaodi You, and Ying Fu. “Learning Temporal Consistency for Low Light Video Enhancement from Single Images.” In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 4967–76, 2021.
  • Nighttime visibility enhancement by increasing the dynamic range and suppression of light effects [Pdf]
    • Sharma, Aashish, and Robby T. Tan. “Nighttime Visibility Enhancement by Increasing the Dynamic Range and Suppression of Light Effects.” In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 11977–86, 2021.
  • SDSD [Pdf] [Code]
    • Ruixing Wang, Xiaogang Xu, Chi-Wing Fu, Jiangbo Lu, Bei Yu, and Jiaya Jia. “Seeing Dynamic Scene in the Dark: A High-Quality Video Dataset with Mechatronic Alignment.” In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), 9700–9709, 2021.
  • DeepHDRVideo: [Pdf] [Web] [Code]
    • Chen, Guanying, Chaofeng Chen, Shi Guo, Zhetong Liang, Kwan-Yee K. Wong, and Lei Zhang. “HDR Video Reconstruction: A Coarse-to-Fine Network and a Real-World Benchmark Dataset.” In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), 2502–11, 2021.
  • MID: [Pdf] [Web] [Code]
  • Song, Wenzheng, Masanori Suganuma, Xing Liu, Noriyuki Shimobayashi, Daisuke Maruta, and Takayuki Okatani. “Matching in the Dark: A Dataset for Matching Image Pairs of Low-Light Scenes.” In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), 6029–38, 2021.
  • UTVNet: [Pdf] [Code]
    • Zheng, Chuanjun, Daming Shi, and Wentian Shi. “Adaptive Unfolding Total Variation Network for Low-Light Image Enhancement.” In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), 4439–48, 2021.
  • LLVIP [Pdf] [Code] [Web]
  • Jia, Xinyu, Chuang Zhu, Minzhen Li, Wenqi Tang, and Wenli Zhou. “LLVIP: A Visible-Infrared Paired Dataset for Low-Light Vision.” In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV) Workshops, 3496–3504, 2021.
  • Low-Light-Level Image Super-Resolution Reconstruction Basedon a Multi-Scale Features Extraction Network基于多尺度特征提取网络的微光图像超分辨率重建[paper][code] 该文主要是基于低照度图像的复原重建，侧重于去模糊过程，与亮度提升无关。
  • Low-light Image Restoration with Short- and Long-exposure Raw Pairs[paper][code] 同上，侧重点不同
  • Low-Light Demosaicking and Denoising for Small Pixels Using Learned Frequency Selection[paper][code] 同上

image.png

  • Learning to restore light fields under low-light imaging[paper][code] 黑图—》清晰的黑白图

image.png

2020

  • Low-Light Image Enhancement with Attention and Multi-level Feature Fusion [Pdf]
    • L. Wang, G. Fu, Z. Jiang, G. Ju and A. Men, “Low-Light Image Enhancement with Attention and Multi-level Feature Fusion,” 2019 IEEE International Conference on Multimedia & Expo Workshops (ICMEW), Shanghai, China, 2019
  • Zero-DCE [Web] [Code] [Pdf]
    • C. Guo et al., “Zero-Reference Deep Curve Estimation for Low-Light Image Enhancement,” in 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Seattle, WA, USA, Jun. 2020, pp. 1777–1786, doi: 10.1109/CVPR42600.2020.00185.
  • Learning to Restore Low-Light Images via Decomposition-and-Enhancement [Pdf]
    • K. Xu, X. Yang, B. Yin and R. W. H. Lau, “Learning to Restore Low-Light Images via Decomposition-and-Enhancement,” 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Seattle, WA, USA, 2020, pp. 2278-2287, doi: 10.1109/CVPR42600.2020.00235.
  • DRBN [Pdf][Paper Link] [Project Page] [Slides]
    • W. Yang, S. Wang, Y. Fang, Y. Wang and J. Liu, “From Fidelity to Perceptual Quality: A Semi-Supervised Approach for Low-Light Image Enhancement,” 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Seattle, WA, USA, 2020, pp. 3060-3069, doi: 10.1109/CVPR42600.2020.00313.
  • STARnet[Web] [Code] [Pdf]
  • Muhammad Haris, Greg Shakhnarovich, and Norimichi Ukita, “Space-Time-Aware Multi-Resolution Video Enhancement”, Proc. of IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2020.
  • DeepLPF [Code][Pdf]
  • Moran, S., Marza, P., McDonagh, S., Parisot, S., Slabaugh, G. DeepLPF: Deep Local Parametric Filters for Image Enhancement, in: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2020.
  • Image-Adaptive-3DLUT [Code] [Pdf]
    • H. Zeng, J. Cai, L. Li, Z. Cao and L. Zhang, “Learning Image-adaptive 3D Lookup Tables for High Performance Photo Enhancement in Real-time,” in IEEE Transactions on Pattern Analysis and Machine Intelligence, 2020.
  • Meng et al, GIA-Net: Global Information Aware Network for Low-light Imaging. [paper][code]
  • Kwon et al, DALE : Dark Region-Aware Low-light Image Enhancement. [paper][code]
  • Yang et al, From Fidelity to Perceptual Quality: A Semi-Supervised Approach for Low-Light Image Enhancement. [paper][code]———-深度递归频带网络(DRBN)-2020CVPR
  • Atoum et al, Color-wise Attention Network for Low-light Image Enhancement. [paper][code]
  • Lv et al, Attention Guided Low-light Image Enhancement with a Large Scale Low-light Simulation Dataset. [paper][code]
  • Guo et al, Zero-Reference Deep Curve Estimation for Low-Light Image Enhancement. [paper][code]
  • Wei et al, A Physics-based Noise Formation Model for Extreme Low-light Raw Denoising. [paper][code]
  • Fu et al, Learning an Adaptive Model for Extreme Low-light Raw Image Processing. [paper][code]
  • Wang et al, Extreme Low-Light Imaging with Multi-granulation Cooperative Networks. [paper][code]
  • Karadeniz et al, Burst Denoising of Dark Images. [paper][code]
  • Xiong et al, Unsupervised Real-world Low-light Image Enhancement with Decoupled Networks. [paper][code]
  • Liang et al, Deep Bilateral Retinex for Low-Light Image Enhancement. [paper][code]
  • Zhang et al, ATTENTION-BASED NETWORK FOR LOW-LIGHT IMAGE ENHANCEMENT. [paper][code]
  • Li et al, Visual Perception Model for Rapid and Adaptive Low-light Image Enhancement. [paper][code]
  • Zhang et al, Self-supervised Image Enhancement Network: Training with Low Light Images Only. [paper][code]

  • Xu et al, Learning to Restore Low-Light Images via Decomposition-and-Enhancement. [paper][code]

    2019

  • Wang et al, Underexposed Photo Enhancement using Deep Illumination Estimation. [paper][code]

  • Loh et al, Low-light image enhancement using Gaussian Process for features retrieval. [paper][code]
  • Zhang et al, Kindling the Darkness: A Practical Low-light Image Enhancer. [paper][code]
  • Ren et al, Low-Light Image Enhancement via a Deep Hybrid Network. [paper][code]
  • Jiang et al, EnlightenGAN: Deep Light Enhancement without Paired Supervision. [paper][code]

  • Wang et al, RDGAN: RETINEX DECOMPOSITION BASED ADVERSARIAL LEARNING FOR LOW-LIGHT ENHANCEMENT. [paper][code]

    2018

  • Chen et al, Learning to See in the Dark. [paper][code]

  • Wei et al, Deep Retinex Decomposition for Low-Light Enhancement. [paper][code]
  • Wang et al, GLADNet: Low-Light Enhancement Network with Global Awareness. [paper][code]
  • Lv et al, MBLLEN: Low-light Image/Video Enhancement Using CNNs. [paper][code]
  • Jiang et al, Deep Refinement Network for Natural Low-Light Image Enhancement in Symmetric Pathways. [paper][code]

  • Cai et al, Learning a Deep Single Image Contrast Enhancer from Multi-Exposure Images. [paper][code]

    2017

  • GHARBI et al, Deep Bilateral Learning for Real-Time Image Enhancement. [paper][code]

  • Shen et al, MSR-net:Low-light Image Enhancement Using Deep Convolutional Network. [paper][code]
  • Tao et al, LLCNN: A convolutional neural network for low-light image enhancement. [paper][code]

  • Ying et al, A Bio-Inspired Multi-Exposure Fusion Framework for Low-light Image Enhancement. [paper][code]

    2016

  • Lore et al, LLNet: A Deep Autoencoder approach to Natural Low-light Image Enhancement. [paper][code]

  • Guo et al, LIME: Low-Light Image Enhancement via Illumination Map Estimation. [paper][code]

    Image Quality Assessment Metrics

  • MSE (Mean Square Error)

  • LOE (Lightness Order Error) [matlab code]
  • VIF (Visual Quality) [matlab code]
  • PSNR (Peak Signal-to-Noise Ratio) [matlab code] [python code]
  • SSIM (Structural Similarity) [matlab code] [python code]
  • FSIM (Feature Similarity) [matlab code]
  • NIQE (Naturalness Image Quality Evaluator) [matlab code][python code]
  • PIQE (Perception based Image Quality Evaluator) [matlab code]
  • BRISQUE (Blind Image Spatial Quality Evaluator) [matlab code]