[Japanese | Thesis | Researches in Minoh Lab | Minoh Lab]

A Dynamic Lighting Control Method based on Foreground Extraction with Background Image Synthesis

When we take video images, they should reflect a scene accurately. A unit of charge coupled device (CCD) arrays is a widely used imaging device. One CCD element receives irradiance which is derived from scene radiance, the irradiance and its exposure time determine its exposure, and the exposure determines against intensity of one pixel on image. The CCD element can convert the exposure into the intensity accurately if only the exposure is mesured within valid range of the CCD element. We call this range the dynamic range and is finite and limited. When the exposure comes over or under the dynamic range, the intensity of the pixel can not be determined and the image becomes white out or black out.

Usually, intelligent CCD cameras control its exposure time automatically in order to keep the exposure of CCD elements within the dynamic range. However, this approach fails in some cases. For example, one typical case arises when the scene radiance is so low or high that the corresponding exposure can not be set within the dynamic range no matter how the exposure time is changed. Another case is that they can not control the exposure locally in the CCD arrays, because the exposure time can not prevent from making influence on all the CCD elements uniformly. Therefore, a new approach that can control the exposure be within the dynamic range against these cases is needed.

In this paper, we propose a novel dynamic lighting control method that changes the scene radiance instead of exposure time. The goal of this method is to keep the exposure of all the CCD elements on which a foreground object is projected within the dynamic range by changing the strength of lights in the scene. We call the exposure of CCD elements that correspond to the foreground object the foreground exposure. We solve two issues to accomplish this goal.

The first issue is to know which CCD element corresponds to the foreground object. In other words, it is to extract a foreground image region. It is necessary to measure the foreground exposure and control lights according to it. We introduce a method that can synthesize background image according to the strength of lights. Even if the strength of the lights is changed dynamically as we control, the foreground image region is extracted through conventional background subtraction method by using the synthesized background image.

The second issue is to measure the response of the foreground exposure against the strength change of the lights. The response is essential to determine the strength of lights so as to keep the foreground exposure within the dynamic range. We measure the response by comparing two images taken at different strength of the lights.

The procedure to control the strength of the lights is as following. First, the foreground image region is extracted and the foreground exposure is measured. When the foreground exposure goes over or under the dynamic range, the strength of lights is changed in order to measure the response of the foreground exposure. If it comes into the dynamic range according to the change, the procedure ends. Otherwise, the strength of the lights is changed according to the measured response. These steps are repeated until the foreground exposure comes into the dynamic range.

We experimented and evaluated the validity of our method. We confirmed that the synthesized background images are precise enough under any strength of the lights. We extracted the foreground image region with the synthesized background image. In the final experiment, we implemented a dynamic light control system that succeeded in keeping the foreground exposure within the dynamic range.

Go back to Thesis Page