Abstract
[Japanese | Thesis | Researches in Minoh Lab | Minoh Lab]


3D Shape Acquisition Based on Consistency between Object Region and Projections of the Visual Hull


In this paper, we describe an approach to acquire 3D(three-dimensional) shapes of various objects. Acquiring 3D exact shapes automatically is significant for the application requiring various 3D objects, such as virtual studio.

The techniques using laser range finder, the multi-baseline stereo method, and the volume intersection are widely used for 3D shape acquisition. The volume intersection technique is more suitable than the others, when we consider the application requiring various types of 3D shapes.

A visual hull is constructed as the intersection of two or more conic volumes created from the object regions in the camera images by the volume intersection technique. The visual hull can be constructed with higher accuracy in principle, as we use more camera images. However, the constructed visual hull will be deteriorated if the object regions are extracted incorrectly. The deterioration increases with the number of camera images, and is a critical problem in the volume intersection technique.

In this paper, we propose a new approach which is effective when the object regions may not be extracted correctly. If all of the object regions are extracted correctly, the projected region of the visual hull to each camera coincides with the object region, or the projected region may not coincide with the object region. That means that we can evaluate whether the visual hull is constructed properly through considering the coincidence, which we call projective consistency, among the projections and the object regions. In the volume intersection technique, the object feature, which is the information extracted from background subtracted images, is used to obtain the visual hull. In this paper, we obtain the 3D shape by considering the object feature and the projective consistency integratively. The object feature shows us whether a pixel is inside the object region or not. The object feature can be calculated from the histogram of the background subtracted image. We estimate a fitness (we call it a shape fitness) for each pixel inside the object region, and treat it as the object feature.

In this paper, we construct a regular distribution model to express the probability distribution of background subtracted values (we call it a diff value later) of each pixel in a background region. We calculate the probability distribution of diff values of each pixel in the corresponding object region, using the probability distribution of pixel values of the corresponding region of both the background images and camera images. The shape fitness corresponding to a diff value can be calculated as the ratio of number of pixels which have the same value as the diff inside the object region to the number of the pixels which have the same value as the diff inside the background subtracted image. We consider the projective consistency among the shape fitness (which is 2D) and the information based on the shape fitness (which is 3D), because we need to consider object feature and projective consistency integratively. In the volume intersection technique, a voxel does not belong to the visual hull if one of the projections of the center of the voxel to each camera is outside the object region. Accordingly, we estimate the fitness of each pixel to express whether the voxel is inside the 3D shape as the information based on the shape fitness. We construct a voxel fitness space by giving each voxel a fitness.

The 3D shape can be extracted from the voxel fitness space by setting a threshold. The projections of the 3D shape can be calculated. The projective consistency increases, as the sum of shape fitness values of each pixel inside the projection increases, and as the sum of shape fitness values of each pixel outside the projection decreases. We extract the 3D shape from the voxel fitness space by maximizing the projective consistency value.

We made a simulation to demonstrate the effectiveness of our approach. As a result, our approach shows to be more effective when the object regions may not be extracted correctly using standard procedures.


Go back to Thesis Page