Final project: Evaluation of LDR to HDR

Overview

In the final project, we are going to compare serveral different inverse tone mapping operators (iTMO). High dynamic range (HDR) imaging is currently popular due to to the importance of HDR in application, the two main fields are image based lighting (IBL) and HDR displays. There are many advantages in HDR imaging. In the previous howework, we had generated high dynamic range image, which required to photograph a scene with different exposures and post-processing. Despite capturing HDR contents become more and more popular, HDR content is not always possible to capture, currently the majority content is still low dynamic range (LDR) image. One interest question is can we recover/expand the dynamic range from LDR image?
The expanded HDR image from LDR image might be used as a light source to render an object or directly display on a HDR monitor. Akyuz et al.[2] confirmed that people really do prefer HDR displays to LDR displays. In this project we mainly follow Banterle's paper [1] to compare several iTMO.


Method

Here we briefly introduce the methods used to expand LDR to HDR and the evaluation methods in this project.

Expansion methods

The expansion methods could be seperated into 3 different type. The global methods apply a global expansion function on the LDR image at each pixel[2-4]. These global methods are easy to use however they may also amplify the noise. The expand map methods use an expand map as a guidance to direct the expansion of the LDR image[5-8]. Some methods [6-8] use bilateral filter to presever the edge contrast. These methods mainly focus on the bright region in an image. The global and expand map methods are able to increase the dynamic range, however they apply a single expand method on the LDR image and do not care about the real luminance in the scene. Several classfication methods use different expansion parameters to expand the dynamic range of an image by classifying image content as bright/dim or outdoor/indoor etc[9-11].
In this project, we first generate a LDR image by applying a tone mapping operator[12] to the original HDR image. Then we use different expansion methods to expand the LDR. We provide a maximum luminance from groundtruth HDR to all the expansion methods.

Evaluation methods

The main applications of HDR are image visualization and image based lighting. We evaluate different expansion methods with two experiments according to the HDR applications as in [1]. We do not ask people to help determine the similarity between groundtruth and expanded HDR, we use the latest HDR-VDP[13] to prediect the quanlity of the expanded HDR.

Result

First, let's compare the visual difference between HDR and LDR that display on a HDR monitor, we use HDR-VDP to indicate the probability that human observer can detect the differences.

The first column shows the VDP difference between expanded HDR and the groundtruth HDR. For each expaned HDR, we also generate a LDR image with Reinhard02 tone mapping operator, the second column.
The expansion methods from top to bottom are: [2],[5],[9],[8],[11],[3],[6].

Then we use the HDR images to render an object, next we will show the image based lighting experiment.

References

[1] Banterle, Francesco, et al. "A psychophysical evaluation of inverse tone mapping techniques." Computer Graphics Forum. Vol. 28. No. 1. Blackwell Publishing Ltd, 2009.
[2] Akyüz, Ahmet Oǧuz, et al. "Do HDR displays support LDR content?: a psychophysical evaluation." ACM Transactions on Graphics (TOG). Vol. 26. No. 3. ACM, 2007.
[3] Masia, Belen, et al. "Evaluation of reverse tone mapping through varying exposure conditions." ACM Transactions on Graphics (TOG). Vol. 28. No. 5. ACM, 2009
[4] Landis, Hayden. "Production-ready global illumination." Siggraph course notes 16.2002 (2002): 11.
[5] Banterle, Francesco, et al. "Expanding low dynamic range videos for high dynamic range applications." Proceedings of the 24th Spring Conference on Computer Graphics. ACM, 2008.
[6] Rempel, Allan G., et al. "Ldr2hdr: on-the-fly reverse tone mapping of legacy video and photographs." ACM Transactions on Graphics (TOG). Vol. 26. No. 3. ACM, 2007.
[7] Kovaleski, Rafael Pacheco, and Manuel M. Oliveira. "High-quality brightness enhancement functions for real-time reverse tone mapping." The Visual Computer 25.5-7 (2009): 539-547.
[8] Kovaleski, Rafael P., and Manuel M. Oliveira. "High-quality reverse tone mapping for a wide range of exposures." Graphics, Patterns and Images (SIBGRAPI), 2014 27th SIBGRAPI Conference on. IEEE, 2014.
[9] Meylan, Laurence, Scott Daly, and Sabine Süsstrunk. "The reproduction of specular highlights on high dynamic range displays." Color and Imaging Conference. Vol. 2006. No. 1. Society for Imaging Science and Technology, 2006.
[10] Didyk, Piotr, et al. "Enhancement of bright video features for HDR displays." Computer Graphics Forum. Vol. 27. No. 4. Blackwell Publishing Ltd, 2008.
[11] Kuo, Pin-Hung, Chi-Sun Tang, and Shao-Yi Chien. "Content-adaptive inverse tone mapping." Visual Communications and Image Processing (VCIP), 2012 IEEE. IEEE, 2012.
[12] Reinhard, Erik, et al. "Photographic tone reproduction for digital images." ACM Transactions on Graphics (TOG). Vol. 21. No. 3. ACM, 2002.
[13] Mantiuk, Rafat, et al. "HDR-VDP-2: a calibrated visual metric for visibility and quality predictions in all luminance conditions." ACM Transactions on Graphics (TOG). Vol. 30. No. 4. ACM, 2011.

Acknowledgement

At last, thanks professor Jean-François Lalonde giving us such wonderful class!