Simulating Spatially Varying Lighting on a Live Performance
3rd European Conference on Visual Media Production (CVMP 2006), Nov 2006
SIGGRAPH 2005 Sketch
Andrew Jones    Andrew Gardner    Mark Bolas*   
Ian McDowall**    Paul Debevec   
USC Institute for Creative Technologies    *USC School of Cinematic Arts    **Fakespace Labs, Inc.   
Abstract

We present an image-based technique for relighting dynamic human performances under spatially varying illumination. Our system generates a time-multiplexed LED basis and a geometric model recovered from high-speed structured light patterns. The geometric model is used to scale the intensity of each pixel differently according to its 3D position within the spatially varying illumination volume. This yields a first-order approximation of the correct appearance under the spatially varying illumination. A global illumination process removes indirect illumination from the original lighting basis and simulates spatially varying indirect illumination. We demonstrate this technique for a human performance under several spatially varying lighting environments.


One of 29 basis lighting directions


One of 24 structured light patterns


Face relit by spatially-varying light

Downloads

CVMP 2006 Paper
spatial_cvmp.pdf (1.6MB)

SIGGRAPH 2005 Sketch
spatial_sketch.pdf (1.2MB)

Videos
spatial_relighting_divx.avi (8.7MB)
spatial_relighting_mpeg4.avi (4.5MB)

Related Projects

Light Stage 1
Acquiring the Reflectance Field of a Human Face, SIGGRAPH 2000
Facial Reflectance Field Demo, SIGGRAPH 2000 Creative Applications Laboratory
Realistic Human Face Scanning and Rendering, ICT Graphics Lab 2001

Light Stage 2
A Photometric Approach to Digitizing Cultural Artifacts, VAST 2001
Animatable Facial Reflectance Fields, EGSR 2004
Reflectance Field Rendering of Human Faces for "Spider-Man 2", SIGGRAPH 2004 Sketch

Light Stage 3
A Lighting Reproduction Approach to Live-Action Compositing, SIGGRAPH 2002
Optimizing Color Matching in a Lighting Reproduction System for Complex Subject and Illuminant Spectra, EGSR 2003

Light Stage 5
Postproduction Re-Illumination of Live Action Using Interleaved Lighting, SIGGRAPH 2004 Poster
Performance Geometry Capture for Spatially Varying Relighting, SIGGRAPH 2005 Sketch

Light Stage 6
Relighting Human Locomotion with Flowed Reflectance Fields, EGSR 2006 Paper
Relighting Human Locomotion with Flowed Reflectance Fields, SIGGRAPH 2006 Sketch
Light Stage Data Gallery.