A Perceptual Measure for Deep Single Image Camera Calibration

Most current single image camera calibration methods rely on specific image features or user input, and cannot be applied to natural images captured in uncontrolled settings. We propose directly inferring camera calibration parameters from a single image using a deep convolutional neural network. This network is trained using automatically generated samples from a large-scale panorama dataset, and considerably outperforms other methods, including recent deep learning-based approaches, in terms of standard L2 error. However, we argue that in many cases it is more important to consider how humans perceive errors in camera estimation. To this end, we conduct a large-scale human perception study where we ask users to judge the realism of 3D objects composited with and without ground truth camera calibration. Based on this study, we develop a new perceptual measure for camera calibration, and demonstrate that our deep calibration network outperforms other methods on this measure. Finally, we demonstrate the use of our calibration network for a number of applications including virtual object insertion, image retrieval and compositing.

Paper

Yannick Hold-Geoffroy, Kalyan Sunkavalli, Jonathan Eisenmann, Matt Fisher, Emiliano Gambaretto, Sunil Hadap, and Jean-Fran├žois Lalonde
A Perceptual Measure for Deep Single Image Camera Calibration
IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2018.
[arXiv pre-print] [BibTeX coming soon]


Supplementary material

We provide additional results in this supplementary page.

Demo

A working demo will be available soon, stay tuned!

Poster

You can download the poster in PDF format.

Acknowledgements

The authors gratefully acknowledge the following funding sources:

  • A FRQ-NT Ph.D. scholarship to Yannick Hold-Geoffroy
  • A generous donation from Adobe to Jean-Francois Lalonde
  • NVIDIA Corporation with the donation of the Tesla K40 and Titan X GPUs used for this research.
  • NSERC Discovery GRANT RGPIN-2014-05314
  • REPARTI Strategic Network

Ulaval logo
Adobe logo