Learning to Predict Indoor Illumination from a Single Image

We propose an automatic method to infer high dynamic range illumination from a single, limited field-of-view, low dynamic range photograph of an indoor scene. In contrast to previous work that relies on specialized image capture, user input, and/or simple scene models, we train an end-to-end deep neural network that directly regresses a limited field-of-view photo to HDR illumination, without strong assumptions on scene geometry, material properties, or lighting. We show that this can be accomplished in a three step process: 1) we train a robust lighting classifier to automatically annotate the location of light sources in a large dataset of LDR environment maps, 2) we use these annotations to train a deep neural network that predicts the location of lights in a scene from a single limited field-of-view photo, and 3) we fine-tune this network using a small dataset of HDR environment maps to predict light intensities. This allows us to automatically recover high-quality HDR illumination estimates that significantly outperform previous state-of-the-art methods. Consequently, using our illumination estimates for applications like 3D object insertion, we can achieve results that are photo-realistic, which is validated via a perceptual user study.

Paper

Marc-André Gardner, Kalyan Sunkavalli, Ersin Yumer, Xiaohui Shen, Emiliano Gambaretto, Christian Gagné, and Jean-François Lalonde
Learning to Predict Indoor Illumination from a Single Image
ACM Transactions on Graphics (SIGGRAPH Asia), 9(4), 2017
[arXiv pre-print] [BibTeX]

Supplementary material

We provide additional results in this supplementary page.

Dataset

We provide the HDR dataset we used to train the network. It is free to use for non commercial purposes. The complete dataset (EXR latlong format) can be downloaded here.

Demo

Coming soon!

Talk

Coming soon!

Acknowledgements

The authors gratefully acknowledge the following funding sources:

  • A NSERC Alexander Graham Bell Canada Doctoral Scholarship to Marc-André Gardner
  • A generous donation from Adobe to Jean-Francois Lalonde
  • NVIDIA Corporation with the donation of the Tesla K40 and Titan X GPUs used for this research.
  • FRQ-NT New Researcher Grant 2016‐NC‐189939
  • REPARTI Strategic Network

Ulaval logo
Adobe logo