USC

ICT

GL

Practical Modeling and Acquisition of Layered Facial Reflectance
ACM SIGGRAPH Asia 2008
Abhijeet Ghosh    Tim Hawkins    Pieter Peers    Sune Frederiksen    Paul Debevec
USC Institute for Creative Technologies


Layers of skin reflectance which are modeled by our technique and used to render faces for novel viewpoints and lighting.

 

Abstract:

We present a practical method for modeling layered facial reflectance consisting of specular reflectance, single scattering, and shallow and deep subsurface scattering. We estimate parameters of appropriate reflectance models for each of these layers from just 20 photographs recorded in a few seconds from a single viewpoint. We extract spatially-varying specular reflectance and singlescattering parameters from polarization-difference images under spherical and point source illumination. Next, we employ direct-indirect separation to decompose the remaining multiple scattering observed under cross-polarization into shallow and deep scattering components to model the light transport through multiple layers of skin. Finally, we match appropriate diffusion models to the extracted shallow and deep scattering components for different regions on the face. We validate our technique by comparing renderings of subjects to reference photographs recorded from novel viewpoints and under novel illumination conditions.

Introduction:

This paper aims to develop a practical appearance model which is in addition easy to incorporate in existing rendering systems. The detail in the facial appearance model should be such that full-screen close-ups can be faithfully reproduced. Additionally, working with live subjects requires fast acquisition to avoid registration problems, temporal changes in the appearance (e.g., due to sweat or blood flow), and to enable capture of facial appearance of natural expressions which are difficult to hold for more than a few seconds.

(Fig. 2) Modeled skin reflectance components.

To achieve these goals, we model facial skin reflectance as a combination of different layers: specular reflectance, single scattering, and shallow and deep multiple scattering (Fig. 1). A suitable reflectance or scattering model is selected for each layer, and parameters are obtained using a single high-resolution still camera to capture a small set of 20 photographs under environmental and projected lighting conditions. For each reflectance component, we estimate or infer high-frequency details such as albedo and normals per pixel based on the environmental illumination patterns, while modeling lower-frequency BRDF and scattering behavior per region based on the projected patterns. This allows for fast acquisition and straightforward processing, while achieving a high level of realism in the resulting models.

Measured Reflectance Components:

Figure 3: (a) Polarization difference image under spherical illumination, used for estimating specular albedo. (b) Cross-polarized image under spherical illumination, used to measure total scattered albedo. (c) Polarization difference image under directional illumination, used for estimating the specular lobe shape per region. The image also includes some polarization preserving non-specular backscattering (which we model as mostly single-scattering), which can be seen to pick up color from the melanin in the epidermis. (d) Cross-polarized image under directional illumination, showing multiple scattering. (e) "direct" component of (d), showing shallow scattering. (f) "indirect" component of (d), showing deeply scattered light. Note that (d) = (e) + (f) and that (c) + (d) produces a typical front-lit photograph.

 

Material:

ACM SIGGRAPH Asia 2008 Paper:

SIGGRAPH 2008 Talk:

Related Projects:
  • Surface Reflectometry:

  • Light Stage 1:

  • Light Stage 2:

  • Light Stage 3:

  • Light Stage 5:

    • © 2008 - 2013 USC Institute for Creative Technologies

    		Best Viewed in Firefox, Opera, Chrome and IE6 +