[Japanese | Thesis | Researches in Minoh Lab | Minoh Lab]

Model Based Refinement for Thin Part of Visual Hull

Accurate 3D shapes of various objects are needed for exhibition of virtual museums. 3D shapes of objects with thin parts such as insects are also needed.

There are many methods for 3D shape acquisition. Using laser rangefinders is one of generally used methods for 3D shape acquisition. However, this method is unable to acquire shapes of objects that absorb the laser. The stereo vision is also a generally used method for 3D shape acquisition. Though, it is difficult to calculate shapes of objects which do not have their textures with the stereo vision. Another generally used method for 3D shape acquisition is the Volume Intersection Method(VIM). Apart from the methods explained above, the VIM can acquire 3D shapes of objects if only the objects' region on images can be required. An object's region on an image is called as a silhouette. The 3D shape acquired with the VIM is called as a visual hull. This VIM suits the purpose of acquisition of 3D shapes for exhibition of virtual museum. Based on this advantage, we use the VIM to acquire 3D shapes.

Although the VIM has the advantage as described above, if there is any defect in any silhouette, the defect will also occur in a visual hull created by the silhouette. A silhouette is generally acquired by calculating substraction values between intensity values of pixels of a foreground image and that of a background image. Therefore, a defect will occur on the silhouette on a region that the color of the foreground image resembles that of the background image. According to this, the use of only the silhouettes makes it difficult to acquire an accurate visual hull. Especially when there is any defect on the visual hull at thin parts of the object, the defect becomes strongly apparent. This type of loss is undesirable to be occurred on a 3D shape used as an exhibit.

In this paper, we propose a method of refinement for defects in a visual hull at thin parts of an object. Based on an idea that human can detect some defects on a visual hull with priori knowledge about object's shape properties, we apply these object's shape properties to shape refinement. For an object with thin parts such as an insect, we can explain two object's shape properties. The first property is that the object is made up of one torso, multiple limbs and antennas. These torso, limbs and antennas can be segmentalized to primitive segments. The second property is that all primitive segments are connected. We then make a model from these properties and use the model for visual hull refinement.

However, the model does not give the object's accurate 3D shape directly. We use the model only as constraints for shape refinement. Since information on real object's shape is given only from the foreground and the background images, the visual hull is refined to be maintaining consistency with the foreground and the background images.

First, an articulated object model of the insect is built according to the first object's shape property. This articulated object model is used for limiting region to be detected whether it should be restored as a region for refining the visual hull. A region limited by one primitive segment of the articulated object model is called a restoration domain.

Next, for each primitive segment, defects on the visual hull inside the restoration domain are detected. Due to the second object's shape property, some defects occur on the visual hull can be detected by examining the number of connected region of visual hull inside the restoration domain. If there are plural connected regions, it shows that there is a defect on the visual hull.

Finally, the detected defects on the visual hull are refined by addition of 3D regions. In calculation of these additional 3D regions, 3D space is termed by voxel space. The voxel which is projected to the pixel that has greater background substraction value, has more probability to be included in the real object shape's region. Based on this fact, for shape refinement, the greater the voxel's background substraction value is, the more preferentially the voxel is added to the region for refining the visual hull. At last, the refined hull is to be a single connected region.

In our experiment, we acquired a 3D shape of a real insect from images with the VIM. We applied our method to refine the acquired 3D shape. The result of the experiment shows that the acquired 3D shape is refined to a single connected region. Due to the result, the effectiveness of our method is proven.

Go back to Thesis Page