[Japanese | Thesis | Researches in Minoh Lab | Minoh Lab]
Many people become to have an opportunity to shoot an object due to social spread of video cameras. If we can take a video image that is useful to understand a fine action of a moving object, we can use it in various ways. For example, in the field of plays or sports, we can use the video of ourselves to check our action and the video of other people to elaborate their actions.
The video that is useful to understand a fine action of a moving object is defined as follows. The location change of the moving object in the video image is small, the size in the video image is as large as possible and fixed, so that we can easily estimate the velocity of the object in the space. If the location change of the moving object in the video image is small and the size is large, it is easy to understand the fine action of the moving object. The image shows the movement of the object clearly so as to make it possible to infer the velocity of the object in the 3D space. The problem is that the object tends to go out of the image when we want to shoot the moving object at large size in the image. In this paper, we propose an automatic shooting method to get the video by controlling pan/tilt/zoom camera parameters. It is useful to understand an action of the moving object.We suppose that a target object moves on a plane and its size is known in advance.
We describe our goals in other words to let them clear. ``The location change of the moving object in the image is small'' is considered as that its apparent location in the image should be fixed and its apparent velocity be small. ``The size of the moving object in the image is as large as possible and fixed'' is considered as that the apparent size of it should be as large as possible and be fixed. ``The image shows the movement of the object clearly'' is considered as that we should limit the apparent velocity of background regions in the image.
We control the apparent location and velocity of the moving object by handling pan/tilt speed of camera and control, the apparent size by handling zoom value. We have to formulate relations between the apparent location and velocity and pan/tilt speed of camera and at the same time we have to formulate a relation between the apparent size and zoom value. In addition to that, we have to find a method to control the apparent velocity of the background regions in the image with pan/tilt speed and zoom in real time.
We use two kinds of cameras to shoot a moving object. One is an observing camera, and the other is a shooting camera. The camera control is achieved in the following way. First, we capture an image taken by an observing camera and estimate the spatial location of the moving object, and then predict its future spatial location and velocity. Then, we convert these two values into camera parameter representations so as to determine the most suitable camera parameters(pan/tilt speed and zoom value). Finally, we send the control command with the determined parameters to the camera.
We focus on two problems that should be overcome to achieve our purpose. The first problem is that the time gap between the time when observing cameras gets an image of the moving object and the time to send the commands to the shooting cameras. It implies that it is impossible to get the accurate spatial location and velocity during observing time interval. The second problem is that pan/tilt speed and zoom value that we can control are limited and quantized due to the hardware equipment we use.
In this paper, we describe our approach which solves these problems.
We evaluate validity of our method by showing experimental result of shooting a moving person. We succeeded in getting the video image in which the apparent location of the person is close to the center of the image, the apparent velocity is small, the apparent size is as large as possible and fixed, and the apparent velocity of the background regions is below the specified speed.