Efficient Multispectral Reflectance Function Capture for Image-Based Relighting
Efficient Multispectral Reflectance Function Capture for Image-Based Relighting
IS&T Color Imaging Conference 2016
Chloe LeGendre     Xueming Yu     Paul Debevec  
USC Institute for Creative Technologies

Figure 1: Row 1. Comparison among three different methods of image-based relighting (IBRL) for a still life scene illuminated by a complex multispectral lighting environment and the real photographed appearance of the scene under the actual illumination. Left: IBRL result using white LEDs only for capturing the reflectance field. Second from the left: photograph of the scene lit by the real-world illumination. Second from the right: Multispectral IBRL using six spectral LED channels to capture the multispectral reflectance field, as recorded by a color camera. Right: Multispectral IBRL again using multispectral LEDs, as observed by a monochrome camera. Row 2. Close-up insets of the images of Row 1, demonstrating superior color rendition with Multispectral IBRL with a color camera. Row 3. After sampling the pixel values from the color charts centered in the scene, we scale the charts to the same overall brightness with a single factor. The foreground dots represent sampled values from the images of the same column, while the background squares represent the target colors from the color chart in the real mixed-illumination lighting environment. Good color rendition is indicated when the foreground dots visually fade into the background squares. (For the real lighting environment color chart, the foreground and background colors would be the same.).



Image-based relighting (IBRL) renders the appearance of a subject in a novel lighting environment as a linear combination of the images of its reflectance field, the appearance of the subject lit by each incident lighting direction. Traditionally, a tristimulus color camera records the reflectance field as the subject is sequentially illuminated by broad-spectrum white light sources from each direction. Using a multispectral LED sphere and either a tristimulus (RGB) or monochrome camera, we photograph a still life scene to acquire its multispectral reflectance field -- its appearance for every lighting direction for multiple incident illumination spectra. For the tristimulus camera, we demonstrate improved color rendition for IBRL when using the multispectral reflectance field, producing a closer match to the scene's actual appearance in a real-world illumination environment. For the monochrome camera, we also show close visual matches. We additionally propose an efficient method for acquiring such multispectral reflectance fields, augmenting the traditional broad-spectrum lighting basis capture with only a few additional images equal to the desired number of spectral channels. In these additional images, we illuminate the subject by a complete sphere of each available narrow-band LED light source, in our case: red, amber, green, cyan, and blue. From the full-sphere illumination images, we promote the white-light reflectance functions for every direction to multispectral, effectively hallucinating the appearance of the subject under each LED spectrum for each lighting direction. We also use polarization imaging to separate the diffuse and specular components of the reflectance functions, spectrally promoting these components according to different models. We validate that the approximated multispectral reflectance functions closely match those generated by a fully multispectral omnidirectional lighting basis, suggesting a rapid multispectral reflectance field capture method which could be applied for live subjects.


Sample Multispectral Reflectance Function Basis Images:

Figure 2: Row A. Full, even sphere of illumination of the scene with each spectral channel of the LED sphere. Row B. Head light illumination of the scene with each spectral channel. Row C. Example measured multispectral reflectance field basis lighting condition images for each spectral channel. Row D. Example full sphere white light promoted multispectral reflectance field basis lighting condition images. White - actual image; Red, Amber, Green, Cyan, and Blue - Virtual images.


Full Paper:

CIC 2016 Slides:

Related Projects: