[Japanese | Thesis | Researches in Minoh Lab | Minoh Lab]
In this paper, we propose a method to relate multimedia teaching materials to each other for medical education based upon the intentions of authors of them that seem to be closely related to their differences in their appearance.
Multimedia teaching materials attract attention in the field of education because they are useful for us to see and learn better. They not only consist of documents, but also consist of figures, photographs, audios, videos, and many other kinds of materials so that each topic can be described from various aspects.
In order to develop teaching materials, we have to arrange some materials based on our intentions. If those materials are assorted by their meaning in a database, computers can automatically find materials that are most suitable to our intentions. However, it is very difficult for computers to understand the meaning of multimedia data, especially that of an image, which consists of an array of pixel data.
Some methods of image retrieval have been proposed so far. One of them is to add the meaning of each image as one of its attributes including the name, the type, etc., and another is to use information of the shape and the color of each area, which can be obtained by analyzing pixel data of the image.
These two methods only consider inherent attributes of each image, while not only those attributes of the image but also its contextual relations to other materials are important to determine its meaning. Furthermore, when we place some materials side by side to explain something, we usually express their difference in the meaning as their difference in the appearance in order to show our intention more clearer. In this paper, we employ the difference between a pair of materials in the appearance in order to estimate hte intention of authors of them, and to relate teaching materials based on it.
In the medical education, many figures give detailed explanation of a whole human body and its organs, and not a few figures can be shared.
Explanatory words are attached on most figures. These words indicate the attributes including the role and the name of the part of figures. When the same words are written on different figures, the words characterize their relations.
We propose a method of relating figures for the medical education to each other based on their meaning implied by their authors, which is expected to be characterized as the difference in appearance of explanatory words. The difference between a pair of figures is characterized with explanatory words shared by the pair, using the positions and the color of the pixel pointed by the explanatory word as well as the ratio of the image size and the number of the explanatory words between the pair.
A ``difference feature vector'' of each pair of figures is defined to represent the relation between the pair of figures. Then, we place each pair of figures in 3D space at the position according to its different feature vector, so that the relation between two pairs is represented as the distance between the positions of the pairs in the space.
In order to verify that the difference feature vector is valid to relate pairs of figures each other, we evaluated correspondence between the vector and the implied meaning of each teaching material through experiments with figures for the medical education.
As the result, we could see the correspondence between the difference feature vector and the implied meaning, but the variation of the vectors corresponding to the same meaning is not small enough to show the correspondence among the pairs of figures very clear. It shows the difficuly to express relation among the pairs of figures based on the difference feature vector visually in the 3D space.