Post-production Facial Performance Relighting using Reflectance Transfer
Post-production Facial Performance Relighting using Reflectance Transfer
ACM Transactions on Graphics (Proceedings of SIGGRAPH 2007)
Pieter Peers1   Naoki Tamura2,3  Wojciech Matusik3   Paul Debevec1
University of Southern California, Institute for Creative Technologies1 The University of Tokyo2 MERL3

From left to right: A reference (source) subject relit with the desired illumination. A frame of the performance (target) actor captured under uniform illumination. The quotient image of the reference actor relit using the target illumination and the uniform performance illumination (warped into the pose of the performance actor). The performance actor with the desired shading transferred from the relit performance actor using the quotient image.



We propose a novel post-production facial performance relighting system for human actors. Our system uses just a dataset of view-dependent facial appearances with a neutral expression, captured for a static subject using a Light Stage apparatus. For the actual performance, however, a potentially different actor is captured under known, but static, illumination. During post-production, the reflectance field of the reference dataset actor is transferred onto the dynamic performance, enabling image-based relighting of the entire sequence. Our approach makes post-production relighting more practical and could easily be incorporated in a traditional production pipeline since it does not require additional hardware during principal photography. Additionally, we show that our system is suitable for real-time post-production illumination editing.


SIGGRAPH 2007 Paper:

SIGGRAPH 2007 Video: (4:30 min)

Related Projects: