Computational Photography @ ICIP 2016

Responsive image

Credit: Ron Niebrugge

In the last decade, computational photography has emerged as a vibrant field of research. A computational camera uses a combination of unconventional optics and novel algorithms to produce images that cannot otherwise be captured with traditional cameras. The design of such cameras involves the following two main aspects:

Optical coding
modifying the design of a traditional camera by introducing programmable optical elements and light sources to capture maximal amount of scene information in images;
Algorithm design
developing algorithms that take information captured by conventional or modified cameras, and create a visual experience that goes beyond the capabilities of traditional systems.

Examples of computational cameras that are already making an impact in the consumer market include: wide field-of-view (Omnicam), light-field (Lytro), high dynamic range (Red, Blackmagic Design), mobile (iPhone, Nexus), multispectral, motion sensing (Leap Motion) and depth cameras (Kinect).

This course serves as an introduction to the basic concepts in programmable optics and computational image processing needed for designing a wide variety of computational cameras, as well as an overview of the recent work in the field.

14:00-14:20
Opening remarks and a brief history of photography [J-F Lalonde]
14:20-15:30
Coded photography [M. Gupta]
15:30-15:40
Break
15:40-16:50
Augmented photography [J-F Lalonde]
16:50-17:10
Future and impact of photography [M. Gupta]
17:10-17:30
Q&A and concluding remarks

1. A brief history of photography

  • From camera obscura to the computational camera
PDF PDF (printable) PPT Keynote

2. Coded photography Novel camera designs and functionalities

  • Optical coding approaches: aperture, image plane, and illumination coding; camera arrays;
  • Novel functionalities: light field cameras, extended DOF cameras, hyperspectral cameras, ultra high-resolution cameras (Gigapixel), HDR cameras, post-capture refocusing and post-capture resolution trade-offs,
  • Depth cameras: structured light, time-of-flight,
  • Compressive sensing: single pixel and high speed cameras;
PDF PPT

3. Augmented photography Algorithmic tools for novel visual experiences

  • Inverting the imaging pipeline: deconvolution, PSF estimation, demosaicking;
  • Capturing bursts of photos: denoising, deblurring;
  • Advanced image editing: automatic cropping, contrast/tone/color adjustments or transfer, distractor removal, and shallow depth of field;
  • 2D image, 3D scene: advanced image editing beyond the image plane, scene geometry, materials, and light estimation for "behind-the-image" editing;
PDF PDF (printable) PPT Keynote

4. Future and impact of photography

  • "Social/collaborative photography" or the Internet of Cameras;
  • Wearable and flexible cameras;
  • Seeing the invisible: seeing around corners, through walls, laser speckle photography;
  • Image forensics;
  • Next generation applications (personalized health monitoring, robotic surgery, self-driving cars, astronomy).
PDF PPT

Bibliography

PDF BibTeX
Mohit Gupta

Mohit Gupta
Assistant Professor, University of Wisconsin, Madison

Mohit Gupta is an assistant professor in the CS department at the University of Wisconsin-Madison. Earlier, he was a research scientist in the CAVE lab at Columbia University. He received a B.Tech. in computer science from Indian Institute of Technology Delhi, and a Ph.D. from the Robotics Institute, Carnegie Mellon University. His research interests are broadly in computer vision and computational imaging. His focus is on designing computational cameras that enable computer vision systems to perform robustly in extreme real-world scenarios, as well as capture novel kinds of information about the physical world that is not possible with conventional cameras. Details can be found here.

Jean-Francois Lalonde

Jean-François Lalonde
Assistant Professor, Laval University

Jean-François Lalonde is an assistant professor in ECE at Laval University, Quebec City. Previously, he was a Post-Doctoral Associate at Disney Research, Pittsburgh. He received a B.Eng. degree in Computer Engineering with honors from Laval University, Canada, in 2004. He earned his M.S at the Robotics Institute at Carnegie Mellon University in 2006 and received his Ph.D., also from Carnegie Mellon, in 2011. After graduation, he became a Computer Vision Scientist at Tandent, where he helped develop LightBrush™, the first commercial intrinsic imaging application. His work focuses on lighting-aware image understanding and synthesis by leveraging large amounts of data. Details can be found here.