Towards Mobile HDR Video

by Alexandre Chapiro and Tassio Knop,
Done as coursework for graduate course at IMPA given by Luiz Velho - Aug/Dec 2010

Overview

The dynamic range of sensors used in digital photography is many times less than that of the human eye. This calls for algorithmic solutions, namely, HDR images/videos.

We have created an application for the Nokia N900 cellphone with the FrankenCamera API that allows us to capture scenes with varying exposure values. These images are then processed by our algorithm, which creates a High Definition Range video, with the same ammount of frames as the original captures and good object movement capture.


Capturing the Frames

An application for the Nokia N900 cellphone with the FrankenCamera API was created to capture sequences of images for the creation of the HDR video. These sequences are comprised of three photographs each, one with a low exposure value, another with the optimal exposure value as given by the autoexposure routine and the last with a greater value. The applications outline is shown below.


image

Figure 1: Application outline.


A sequence of captured frames is shown below.


image

Figure 2: Sequence of captured frames.



Creating the HDR video

Once the video with LDR images with varying exposure values is captured, we use Luiz Velho's method in order to create the HDR frames for the new video. This method can be seen on this webpage. One of several HDR construction methods can be chosen. These methods are better explained in our Results Video, which can be seen below. The code for the capture process can be downloaded below as well. The code for the HDR video construction will be available here soon.

Documents

Code

Acknowledgements

Special thanks to all of our friends at the Visgraf Laboratory for their help and support!


References

[1] A. Adams, E.-V. Talvala, S. H. Park, D. E. Jacobs, B. Ajdin, N. Gelfand, J. Dolson, D. Vaquero, J. Baek, M. Tico, H. P. A. Lensch, W. Matusik, K. Pulli, M. Horowitz, and M. Levoy. The frankencamera: an experimental platform for computational photography. ACM Trans. Graph., 2010.

[2] P. E. Debevec and J. Malik. Recovering high dynamic range radiance maps from photographs. In SIGGRAPH 97: Proceedings of the 24th annual conference on Computer graphics and interactive techniques, pages 369 to 378, New York, NY, USA, 1997. ACM Press/Addison-Wesley Publishing Co.

[3] F. Drago, K. Myszkowski, T. Annen, and N. Chiba. Adaptive logarithmic mapping for displaying high contrast scenes. Computer Graphics Forum, 22, 419 to 426, 2003. 4

[4] M. Grossberg and S. Nayar. Determining the Camera Response from Images: What is Knowable? IEEE Transactions on Pattern Analysis and Machine Intelligence, 25(11):1455-1467, Nov 2003.

[5] S. B. Kang, M. Uyttendaele, S. Winder, and R. Szeliski. High dynamic range video. In SIGGRAPH 03: ACM SIGGRAPH 2003 Papers, pages 319 to 325, New York, NY, USA, 2003. ACM. 1

[6] M. Levoy. Experimental platforms for computational photography. IEEE Computer Graphics and Applications, 30, 81 to 87, 2010.

[7] P.-Y. Lu, T.-H. Huang, M.-S. Wu, Y.-T. Cheng, and Y.-Y. Chuang. High dynamic range image reconstruction from hand-held cameras. Computer Vision and Pattern Recognition, IEEE Computer Society Conference on, 0, 509 to 516, 2009. 2

[8] T.-H. Min, R.-H. Park, and S. Chang. Histogram based ghost removal in high dynamic range images. In ICME 09: Proceedings of the 2009 IEEE international conference on Multimedia and Expo, pages 530 to 533, Piscataway, NJ, USA, 2009. IEEE Press.

[9] T. Mitsunaga and S. Nayar. Radiometric Self Calibration. In IEEE Conference on Computer Vision and Pattern Recognition (CVPR), volume 1, pages 374 to 380, Jun 1999. 1, 2

[10] K. Myszkowski. High Dynamic Range Video. Morgan and Claypool Publishers, 2008.

[11] M. Robertson, S. Borman, and R. Stevenson. Dynamic range improvement through multiple exposures. In Proceedings of the IEEE International Conference on Image Processing, volume 3, pages 159 to 163, Kobe, Japan, Oct. 1999. IEEE.

[12] A. M. Sa. High Dynamic Range Image Reconstruction. Morgan & Claypool Publishers, 2008.

[13] L. Velho. Histogram-based hdr video. In SIGGRAPH 07: ACM SIGGRAPH 2007 posters, page 62, New York, NY, USA, 2007. ACM.

[14] G. Ward. Fast, robust image registration for compositing high dynamic range photographs from handheld exposures. JOURNAL OF GRAPHICS TOOLS, 8, 17 to 30, 2003. 2