[Japanese | Thesis | Researches in Minoh Lab | Minoh Lab]

The Recognition of Documents Handling Process for Indexing of Video Taken during the Graphic Design Meeting

This paper propose a method to recognize the process of handling papers in order to index videos taken during graphic design meeting. By using this method, it becomes possible to search the scene which is related with each paper appearing in the meeting.

When a brochure is produced by designers, they often have a meeting with their clients in order to exchange their opinions about the design of the brochure. Designers present their plans, which are printed on papers so that designers and clients share the same understanding for what they are producing. The clients give their opinions on that plans, and designers present new plans in the next meeting. They make the brochure better through such meetings. In this paper, meetings by the designer and their clients for producing a brochure in the process above is called {\it graphic design meeting}. It is useful to record a video of graphic design meetings, because the participants in the meeting can check their discussions after the meetings.

Participants in a graphic design meeting discuss the plans presented by the designers. It is often observed that they `move' the paper to the place where it can be seen by all the participants in the meeting. The part of the brochure that is currently under discussion is often pointed by the participants. Participants sometimes `pass' some papers to someone when they ask him to confirm the contents of those papers. We suppose that the paper which is handled like this is concerned with the paper which is under discussion. It is useful to realize that handling of papers described above can be used for an index of video taken during graphic design meeting.

It is difficult to recognize the process of handling papers described above, because papers do not keep its flat shape in the process of handling and no assumption is possible for their color. This article proposes a method to recognize the process of handling papers. This method assumes that the participants in a graphic design meeting have a discussion, surrounding the same table. In order to recognize the process of handling papers, first papers have to be discriminated from the other objects and the process of handling the paper have to be recognized. Papers are discriminated from the other objects by using fact that a paper is the thin object that coincides with the surface of the table when it is placed on the table. The process of handling papers is considered to be the relations of papers and hands, because papers are handled by hands. This article propose a method to recognize the process of handling papers by using those relations.

This paper describes a method to discriminate papers used in a graphic design meeting from the other objects. First it is assumed that the object that is on the same surface of the table is a paper. The image region of each camera corresponding to the table plane is called a {\it camera projection regin}. Based on the assumption given above, athe projection regions of each camera must coincide with each other. So, the object is classified as a paper when the lap region of the projection regions is sufficiently large. On the other hand, we classify the object into the others when it is small. In order to adopt this method, cameras must be placed low against the table plane.

In order to recognize the handling process, we define `grasp', which is a relation between papers and hands as the paper and the hand being near and same velocity. We also define `on the table', which is a state of papers as the paper being on the table. We define `moving', which is a state of papers as the paper moving. We can define the handling process by using `grasp', `on the table' and `moving'. For exmple, `point' is defined as a hand and a paper are in 'grasp' and nither of them are `moving'.

In the experiment to evaluate the method for discriminating papers from the other objects, the proportion of the lap region to the each projection region is evaluated for many objects. From the result of this experiment, it was shown that papers can be distinguished from others based on that proportion. From the result of the experiment to evaluate for recognizing process of handling papers, it was shown that a method we propose is useful for the indexing described above.

Go back to Thesis Page