TOP  >  Thesis/Dissertation  >  Human Posture Estimation from Students' Image with Different Part Detector Depending on Degree of Self Occlusion - Minoh Laboratory

Human Posture Estimation from Students' Image with Different Part Detector Depending on Degree of Self Occlusion

The purpose of this paper is to obtain students' posture sequences from lecture movies. Posture sequence is the time series of postures for a student observed by a camera. Posture is the tuple of the positions of the human body parts, such as a head, a neck, elbows and wrists in a given image. Behavior is a specific part of posture sequence which is attached a meaning.

We use students' posture sequences for behavior analysis. We can know the degree of participation and intensive degree for each student by analyzing the students' behavior in a lecture. In previous works, observers have notated the students' behavior in lecture movies manually. However, it is difficult to enforce the uniform classification criteria with a manual method. Therefore the system which automatically obtains the students' behavior is necessary.

We perform student posture estimation from lecture movies observed by a camera at the front upper part of a lecture room. In lecture movies, overlap of the body parts inside the same student frequently occurs. It is called self-occlusion. Self-occlusion often makes posture estimation difficult, and the posture estimation result is greatly different from the correct answer. The purpose of this paper is to obtain students' posture information useful for data mining. We use students' posture information statistically. Hence, we should avoid posture estimation results to be greatly different from the correct answer.

In this paper, we classify student images into predefined classes depending on the degree of self-occlusion. Then, we perform posture estimation with a part detector suitable for the class. If we try to perform posture estimation of student images with self-occlusion by using one part detector which can support the posture of large variations, the part detector may apply a posture greatly different from the correct answer. The self-occlusion occurring in lecture rooms has a few patterns. When self-occlusion of a certain pattern occurs, the position of the body parts where self-occlusion occurred is limited. Therefore we can limit posture variation and prevent the posture estimation to be greatly different from the correct answer; this leads the improvement of the precision of the posture estimation.

In order to change part detectors in the proposed method, we need judge the degree of self-occlusion for students in given images. Part detection is necessary to detect self-occlusion by judging the overlap of body parts. %However, in order to know how the self-occlusion occurs, it is necessary to perform appropriate part detection. However, in order to perform appropriate part detection, it is necessary to know how the self-occlusion occurs. In this paper, we classify student images by a method based on the knowledge obtained by observing a large number of student images. We define three patterns of self-occlusion occurred in lecture rooms: a head covers the front of the neck; a forearm covers another forearm in front of the body; and a head covers the forearms. We classify student images in three classes with two criteria: whether a student puts up the face; and whether a student puts the forearms in front of the body.

In the experiment, we applied the proposed method to real lecture movies. By the proposed method, the average error was about 2/3 compared with the one by a previous method, and the ratio that the error was less than 20 pixel improved at least 20% for images with 240 ~ 300 pixel size. These results showed the effectiveness of our proposed method. Moreover, we visualized the distribution of the body part positions from posture estimation results of a student. Thus, we showed that they can express difference of student behavior and it reveal the possibility to be applied to behavior analysis.

The proposed method cannot deal with overlap caused by objects or other students. If we deal with such overlap, we can apply the posture estimation method to more students. Furthermore, the proposed method does not use time sequence information. If we combine proposed method with the posture estimation method considering a continuity of the time sequence of postures, posture estimation performance will be improved. They are left as future works.