We present a practical method for modeling layered facial reflectance consisting of specular reflectance, single scattering, and shallow and deep subsurface scattering. We estimate parameters of appropriate reflectance models for each of these layers from just 20 photographs recorded in a few seconds from a single viewpoint. We extract spatially-varying specular reflectance and singlescattering parameters from polarization-difference images under spherical and point source illumination. Next, we employ direct-indirect separation to decompose the remaining multiple scattering observed under cross-polarization into shallow and deep scattering components to model the light transport through multiple layers of skin. Finally, we match appropriate diffusion models to the extracted shallow and deep scattering components for different regions on the face. We validate our technique by comparing renderings of subjects to reference photographs recorded from novel viewpoints and under novel illumination conditions.
This paper aims to develop a practical appearance model which is in addition easy
to incorporate in existing rendering systems. The detail in the facial appearance
model should be such that full-screen close-ups can be faithfully reproduced.
Additionally, working with live subjects requires fast acquisition to avoid
registration problems, temporal changes in the appearance (e.g., due to sweat or
blood flow), and to enable capture of facial appearance of natural expressions
which are difficult to hold for more than a few seconds.
To achieve these goals, we model facial skin reflectance as a combination of different layers: specular reflectance, single scattering, and shallow and deep multiple scattering (Fig. 1). A suitable reflectance or scattering model is selected for each layer, and parameters are obtained using a single high-resolution still camera to capture a small set of 20 photographs under environmental and projected lighting conditions. For each reflectance component, we estimate or infer high-frequency details such as albedo and normals per pixel based on the environmental illumination patterns, while modeling lower-frequency BRDF and scattering behavior per region based on the projected patterns. This allows for fast acquisition and straightforward processing, while achieving a high level of realism in the resulting models.
Figure 2: Modeled skin reflectance components.
Figure 3:(a) Polarization difference image under spherical illumination, used for estimating specular albedo. (b) Cross-polarized image under spherical illumination, used to measure total scattered albedo. (c) Polarization difference image under directional illumination, used for estimating the specular lobe shape per region. The image also includes some polarization preserving non-specular backscattering (which we model as mostly single-scattering), which can be seen to pick up color from the melanin in the epidermis. (d) Cross-polarized image under directional illumination, showing multiple scattering. (e) "direct" component of (d), showing shallow scattering. (f) "indirect" component of (d), showing deeply scattered light. Note that (d) = (e) + (f) and that (c) + (d) produces a typical front-lit photograph.