[Japanese | Thesis | Researches in Minoh Lab | Minoh Lab]

Interactive Correction of the Error in Image Superimposed by Virtual Studio with Augmented Reality

Recently there are many TV programs produced by Virtual Studio. By using Virtual Studio, it is possible to overlay an image from a camera on an image created by Computer Graphics (CG). On the other hand, in the field of Virtual Reality (VR), it is proposed to present a stereoscopic display including three dimensional (3D) virtual space together with real space using a see-through Head Mounted Display(HMD) and a stereoscopic projector. This technique is called gAugmented Reality (AR)h. This paper considers a the situation in which a human manipulating a virtual object using AR is shot by Virtual Studio.

In this paper, the human who manipulates a virtual object in the studio is called gcasterh, and those who watch the real video images from a camera overlaid on CG with a floor monitor are called gviewerh. In order to obtain video images for the scene in which the caster manipulates a virtual object using AR, the following two kinds of consistencies are required. One is the consistency in position between the real space and the virtual space for using AR. The other is that between the real space and the virtual space to superimpose the images of the two spaces using Virtual Studio. The former occurs due to the difference in the casterfs perception of the real space and the virtual space. In order to realize these consistencies, it is necessary to cope with the following two errors. One is the error due to a difference between the casterfs perception of the real space and the virtual space through a stereoscopic display. The other is the error caused by incomplete camera calibration. In previous work, it has been tried to correct each of these errors individually in advance. However, complete correction of the errors often requires special facilities and skills. On the other hand, when the purpose of the error correction is to synthesize video images for the situation in which a caster manipulates a virtual object, the acceptable range of the errors depends on the virtual object manipulation shown by the video images. In this situation, even if the errors are corrected up to some extent in advance following the previous approach, there is no guarantee that the correction of the errors is sufficient for synthesizing video images of virtual object manipulation. In this paper, it is proposed to determine the acceptable range of the error and to correct the error so that they are within the acceptable range at the same time through the process of virtual objects manipulation by the caster in the studio. The errors that are caused by a stereoscopic perception and a camera calibration are counted as a single error for the difference in position between the real space and the virtual space. In this work, the goal is not to correct the error completely, but to make it within the acceptable range as the result of the error correction. This acceptable range depends on virtual objects and the task of object manipulation. In virtual object manipulation, it is often considered a physical constraint for the positions of virtual object similar to the object manipulation in real space for the convenience of object manipulation in virtual space with AR. Examples of the physical constraint are, geach object can be handled on its surfaceh, geach object does not collide with other objectsh, and so on.Since the caster tries to satisfy the constraint in manipulating virtual objects, the object manipulation has to be realize under the constraint in order for the errors to be within the acceptable range. In this paper, the positions of virtual objects are corrected so that the caster's object manipulation for pointing satisfies the physical constraint. This error correction is executed interactively through the process of virtual object manipulation when the position of a virtual object to be manipulated by a caster does not satisfy the constraint.

In order to evaluate the effectiveness of the method, the following experimental results will be presented. It is assumed that a caster points static objects with a real pointer. Different size of objects are presented to the caster for this pointing. The resultant error corrected through the pointing is evaluated. As the result, it is shown that the amount of the error decreases as the pointing manipulation is repeated, and that the resultant error becomes smaller when the size of virtual object becomes smaller. It means that the error correction attains the acceptable range dependent on the object size.

Go back to Thesis Page