|Practical Modeling and Acquisition of Layered Facial Reflectance|
|ACM SIGGRAPH Asia 2008|
|Abhijeet Ghosh Tim Hawkins Pieter Peers Sune Frederiksen Paul Debevec|
|USC Institute for Creative Technologies|
Layers of skin reflectance which are modeled by our technique and used to render faces for novel viewpoints and lighting.
We present a practical method for modeling layered facial reflectance consisting of specular reflectance, single scattering, and shallow and deep subsurface scattering. We estimate parameters of appropriate reflectance models for each of these layers from just 20 photographs recorded in a few seconds from a single viewpoint. We extract spatially-varying specular reflectance and singlescattering parameters from polarization-difference images under spherical and point source illumination. Next, we employ direct-indirect separation to decompose the remaining multiple scattering observed under cross-polarization into shallow and deep scattering components to model the light transport through multiple layers of skin. Finally, we match appropriate diffusion models to the extracted shallow and deep scattering components for different regions on the face. We validate our technique by comparing renderings of subjects to reference photographs recorded from novel viewpoints and under novel illumination conditions.
This paper aims to develop a practical appearance model which is in addition easy to incorporate in existing rendering systems. The detail in the facial appearance model should be such that full-screen close-ups can be faithfully reproduced. Additionally, working with live subjects requires fast acquisition to avoid registration problems, temporal changes in the appearance (e.g., due to sweat or blood flow), and to enable capture of facial appearance of natural expressions which are difficult to hold for more than a few seconds.
To achieve these goals, we model facial skin reflectance as a combination of different layers: specular reflectance, single scattering, and shallow and deep multiple scattering (Fig. 1). A suitable reflectance or scattering model is selected for each layer, and parameters are obtained using a single high-resolution still camera to capture a small set of 20 photographs under environmental and projected lighting conditions. For each reflectance component, we estimate or infer high-frequency details such as albedo and normals per pixel based on the environmental illumination patterns, while modeling lower-frequency BRDF and scattering behavior per region based on the projected patterns. This allows for fast acquisition and straightforward processing, while achieving a high level of realism in the resulting models.
Measured Reflectance Components:
ACM SIGGRAPH Asia 2008 Paper:
SIGGRAPH 2008 Talk:
- A Photometric Approach to Digitizing Cultural Artifacts, VAST 2001
- Animatable Facial Reflectance Fields, EGSR 2004
- Reflectance Field Rendering of Human Faces for "Spider-Man 2", SIGGRAPH 2004 Sketch
- Postproduction Re-Illumination of Live Action Using Interleaved Lighting, SIGGRAPH 2004 Poster
- Performance Geometry Capture for Spatially Varying Relighting, SIGGRAPH 2005 Sketch