|Post-production Facial Performance Relighting using Reflectance Transfer||ACM Transactions on Graphics (Proceedings of SIGGRAPH 2007)|
|Pieter Peers1 Naoki Tamura2,3 Wojciech Matusik3 Paul Debevec1|
|University of Southern California, Institute for Creative Technologies1 The University of Tokyo2 MERL3|
From left to right: A reference (source) subject relit with the desired illumination. A frame of the performance (target) actor captured under uniform illumination. The quotient image of the reference actor relit using the target illumination and the uniform performance illumination (warped into the pose of the performance actor). The performance actor with the desired shading transferred from the relit performance actor using the quotient image.
We propose a novel post-production facial performance relighting system for human actors. Our system uses just a dataset of view-dependent facial appearances with a neutral expression, captured for a static subject using a Light Stage apparatus. For the actual performance, however, a potentially different actor is captured under known, but static, illumination. During post-production, the reflectance field of the reference dataset actor is transferred onto the dynamic performance, enabling image-based relighting of the entire sequence. Our approach makes post-production relighting more practical and could easily be incorporated in a traditional production pipeline since it does not require additional hardware during principal photography. Additionally, we show that our system is suitable for real-time post-production illumination editing.
SIGGRAPH 2007 Paper:
- SIGGRAPH2007_PFPR_high.pdf, 17.9 MB. ( Adobe Acrobat )
- SIGGRAPH2007_PFPR_low.pdf, 0.99 MB. ( Adobe Acrobat )
SIGGRAPH 2007 Video:(4:30 min)
- SIG2007_PFPR_Hires.mp4, 1024x768, 57.1 MB. ( Quicktime )
- SIG2007_PFPR_Lores.mp4, 640x480, 33.0 MB. ( Quicktime )
- Light Stage 1:
- Light Stage 2:
- A Photometric Approach to Digitizing Cultural Artifacts, VAST 2001
- Animatable Facial Reflectance Fields, EGSR 2004
- Reflectance Field Rendering of Human Faces for "Spider-Man 2", SIGGRAPH 2004 Sketch
- Light Stage 3:
- Light Stage 5:
- Postproduction Re-Illumination of Live Action Using Interleaved Lighting, SIGGRAPH 2004 Poster
- Performance Geometry Capture for Spatially Varying Relighting, SIGGRAPH 2005 Sketch
- Light Stage 6:
- Relighting Human Locomotion with Flowed Reflectance Fields, EGSR 2006 Paper
- Relighting Human Locomotion with Flowed Reflectance Fields, SIGGRAPH 2006 Sketch
- Light Stage Data Gallery.