[Japanese | Thesis | Researches in Minoh Lab | Minoh Lab]
The purpose of this paper is to realize real-time stereo vision with DSP boards. We propose the method of real-time stereo vision appropriate for DSP boards and show its efficiency by experiment.
The problem of real-time stereo vision has two aspects, ``accuracy'' and ``speed''. In the aspect of ``accuracy'', a problem of occlusion needs to be solved to reduce wrong correspondence. In the aspect of ``speed'', a configuration of cameras, especially the length of their base line (the length between the two cameras), needs to be considered because longer processing time is required for searching correspondence between points in two images when the length is longer. In addition, when DSP boards are employed for high-speed processing, a configuration of DSP boards needs to be considered. In this paper, solutions to the three above-mentioned problems are proposed and their efficiency is shown by experiment.
First, there are two kinds of occlusion in stereo vision. One is ``occlusion of projected point'', that a point projected on one image may not be projected on the other image. Such a point on one image corresponds to a wrong point on the other image. As a solution, we propose to reject the correspondence of two points when similarity between them is lower than a threshold, based on the fact that wrong correspondence of points in many cases causes low similarity between them. The other is ``occlusion of boundary'', that an object projected on a side of an edge in one images differs from that projected on the same side of the corresponding edge in the other image. As a solution, we propose to calculate the intensity similarities on the same side of each edge on two images and adapt the higher similarity in both sides.
Second, two cameras are arranged so that their image planes are on the same plane. This arrangement reduces the uncertainty of measurement and realizes a fast process on DSP board because the scanning line corresponds with the epi-polar line. In this configuration, when the length of the base line becomes longer, more accurate measurement can be realized, although it also takes more processing time because the range of searching corresponding points becomes wider. To make accuracy compatible with speed, we limit the range of stereo measurement and the range of searching correspondence between points, and calculate the ideal length of the base line by their limitations and the camera's focal length.
Finally, this paper shows the most effective configuration of DSP boards for real-time processing is parallel connection. Especially, the parallel processing by splitting the image is effective. Thus, two methods of parallel processing is introduced, one by splitting image horizontally and the other by splitting image vertically. As a result, it is known, in edge extraction process, splitting images vertically realizes faster process than splitting those horizontally in case the width of image is longer than the height. In the opposite case, splitting images horizontally realizes faster process. The process of searching correspondence between points becomes faster when characteristic points are splitted equally.
The efficiency of the two parallel processing methods, by splitting images horizontally or vertically, are demonstrated. The processing times for two methods are measured. The image size is 250 pixels in width and 200 pixels in height. In the edge extraction process, it was shown that processing time in splitting image vertically is a little shorter. In the process for searching correspondence between points, it was shown that the processing time becomes shorter when characteristic points are splitted equally.
We realized real-time stereo vision by this method. From the result, the efficiency of parallel processing method proposed in this paper is shown.