ICT

GL

Compressive Light Transport Sensing

ACM Transactions on Graphics, Volume 28, Issue 1 (January 2009)

CT Technical Report, ICT-TR-05-2008, May 2008

Pieter Peers¹ Dhruv K. Mahajan² Bruce Lamond¹ Abhijeet Ghosh¹
Wojciech Matusik⁴ Ravi Ramamoorthi^2,3 Paul Debevec¹

University of Southern California, Institute for Creative Technologies¹
Columbia University²
University of Berkely³
Adobe Inc.⁴

Three scenes captured using only 1000 non-adaptive compressive measurements, and relit using a novel conditions. The 128x128 reflectance functions of each camera pixel is reconstructed using our hierarhical reconstruction algorithm using 128 Haar wavelet coefficients per function from the observed compressive measurements.

Abstract:

In this paper we propose a new framework for capturing light transport data of a real scene, based on the recently developed theory of compressive sensing. Compressive sensing offers a solid mathematical framework to infer a sparse signal from a limited number of non-adaptive measurements. Besides introducing compressive sensing for fast acquisition of light transport to computer graphics, we develop several innovations that address specific challenges for image-based relighting, and which may have broader implications. We develop a novel hierarchical decoding algorithm that improves reconstruction quality by exploiting inter-pixel coherency relations. Additionally, we design new non-adaptive illumination patterns that minimize measurement noise and further improve reconstruction quality. We illustrate our framework by capturing detailed highresolution reflectance fields for image-based relighting.

Material:

ACM Transactions on Graphics Paper

ACM TOG External Link

ICT Technical Report (May 2008)

ICT-TR-05-2008.pdf, 9.43 MB. ( Adobe Acrobat )

Errata:

Oops: An unfortunate error crept into both the TOG paper and the technical report. Eq. 15 is obviously incorrect. However, one can still apply Eq. 16 to compute a sparse representation of the difference.

Concurrent Related Work:

We have recently become aware of concurrent and independent work performed by Pradeep Sen and Soheil Darabi at the University of New Mexico (link) that applies compressive sensing to Dual Photography. This work tackles the problem of measuring high resolution reflectance fields in a similar manner. The main difference is that they directly apply Bernoulli noise measurement patterns that work well in a Dual Photography setup.

RELATED PROJECTS:

Light Stage 1:
- Acquiring the Reflectance Field of a Human Face, SIGGRAPH 2000
- Facial Reflectance Field Demo, SIGGRAPH 2000 Creative Applications Laboratory
- Realistic Human Face Scanning and Rendering, ICT Graphics Lab 2001

Light Stage 2:
- A Photometric Approach to Digitizing Cultural Artifacts, VAST 2001
- Animatable Facial Reflectance Fields, EGSR 2004
- Reflectance Field Rendering of Human Faces for "Spider-Man 2", SIGGRAPH 2004 Sketch

Light Stage 3:
- A Lighting Reproduction Approach to Live-Action Compositing, SIGGRAPH 2002
- Optimizing Color Matching in a Lighting Reproduction System for Complex Subject and Illuminant Spectra, EGSR 2003

Light Stage 5:
- Postproduction Re-Illumination of Live Action Using Interleaved Lighting, SIGGRAPH 2004 Poster
- Performance Geometry Capture for Spatially Varying Relighting, SIGGRAPH 2005 Sketch

Light Stage 6:
- Relighting Human Locomotion with Flowed Reflectance Fields, EGSR 2006 Paper
- Relighting Human Locomotion with Flowed Reflectance Fields, SIGGRAPH 2006 Sketch
- Light Stage Data Gallery.

Reflectance Transfer:
- Post-production Facial Performance Relighting using Reflectance Transfer, SIGGRAPH 2007 Paper

Dual Light Stage:
- A Dual Light Stage, EGSR 2005 Paper