USC

ICT

GL

Relighting Human Locomotion with Flowed Reflectance Fields
Relighting Human Locomotion with Flowed Reflectance Fields
2006 Eurographics Symposium on Rendering
Per Einarsson   Charles-Felix Chabert   Andrew Jones   Wan-Chun Ma*   Bruce Lamond
Tim Hawkins   Mark Bolas**   Sebastian Sylwan   Paul Debevec
University of Southern California Centers for Creative Technologies       National Taiwan University*
University of Southern California School of Cinematic Arts**



Multiple instances of a running subject are composited into an image-based lighting environment with matched viewpoint and illumination. The subject was captured running in a human-scale light stage apparatus under a time multiplexed lighting basis to allow for image-based relighting. A novel combination of light field rendering and view interpolation was used to render the subject from an arbitrary 3D viewpoint.

 


Abstract:

We present an image-based approach for capturing the appearance of a walking or running person so they can be rendered realistically under variable viewpoint and illumination. In our approach, a person walks on a treadmill at a regular rate as a turntable slowly rotates the person's direction. As this happens, the person is filmed with a vertical array of high-speed cameras under a time-multiplexed lighting basis, acquiring a seven-dimensional dataset of the person under variable time, illumination, and viewing direction in approximately forty seconds. We process this data into a flowed reflectance field using an optical flow algorithm to correspond pixels in neighboring camera views and time samples to each other, and we use image compression to reduce the size of this data. We then use image-based relighting and a hardware-accelerated combination of view morphing and light field rendering to render the subject under user-specified viewpoint and lighting conditions. To composite the person into a scene, we use an alpha channel derived from back lighting and a retroreflective treadmill surface and a visual hull process to render the shadows the person would cast onto the ground. We demonstrate realistic composites of several subjects into real and virtual environments using our technique.

 


Material:

EGSR 2006 Paper:

EGSR 2006 Video:

SIGGRAPH 2006 Sketch:

LS6 Video Presented at the Director's Guild of America's Digital Day Conference, July 29, 2006:

Relighting Human Locomotion
SIGGRAPH 2006 Computer Animation Festival


Related Projects: