**6. Application to seismic data analysis**

In this study, in order to evaluate the effectiveness of the developed system, the platform of immersive visual data mining was applied to the seismic data analysis. In Japan, a lot of earthquakes occur every year. Then, it is important to analyze large amount of past data to predict next earthquake and estimate damage (Furumura, 2005). Particularly, this problem has been very important after Great East Japan Earthquake on March 11, 2011. As original data for the immersive visual data mining, the seismic hypocenter data in last five years that is stored in Hi-net system (Okada, 2004) constructed by National Research Institute for Earth Science and Disaster Prevention was used.

#### **6.1 Visualization of hypocenter data**

In the visualization using the conventional two-dimensional display, it is difficult to observe both the individual location and the overall distribution of the hypocenter data and to understand the feature of three-dimensional distribution of the earthquake. Therefore, first, the seismic hypocenter data was visualized using the platform of representing point cloud data in the super high-definition immersive visual data mining environment. In this system, about 600,000 seismic hypocenter data with magnitude and occurrence time were stored in the database table and they were visualized as point clouds. From the hypocenter data, statistical data of b-value in Gutenberg-Richter (GR) relation were calculated. GR relation is a typical model to represent the distribution of magnitude of the earthquakes. In this case, more than 2,000,000 b-value data were calculated in the area specified by time and place, and they were stored in another database table to visualize them in the platform. In this application, these data were visualized overlapped with the coastal line of Japan using the plug-in function of Open CABIN Library.

Immersive Visual Data Mining Based on Super High Definition Image 165

value data, because too many point clouds were visualized simultaneously. Figure16 shows the visualization image of b-value data filtered by time parameter interactively. With the reduction of the number of displayed point data, the users can recognize the feature of spatially distributed data. These visualization images were rendered in real time through the user's interactive operation using the physical MIDI controller. Thus, in this application, the user could effectively examine the feature of the distribution of the earthquakes

interactively.

Fig. 15. Visualization of b-value data

Fig. 16. Visualization of b-value data filtered by time

Figure 13 shows the visualization image of the seismic hypocenter data that is colored according to the magnitude value. From the high resolution stereo image, the user can recognize the tendency of the earthquake distribution. Figure 14 shows another example of the visualization image in which the occurrence time of the earthquake is mapped to the zaxis. From this image, the user can see how long a lot of small aftershocks continued after big earthquake.

Fig. 13. Visualization of seismic hypocenter data

Fig. 14. Visualization of seismic hypocenter data by mapping occurrence time to z-axis

Figure 15 shows the visualization of b-values calculated from the seismic hypocenter data. In this case, it was difficult for the user to recognize the distribution of the visualized b-

Figure 13 shows the visualization image of the seismic hypocenter data that is colored according to the magnitude value. From the high resolution stereo image, the user can recognize the tendency of the earthquake distribution. Figure 14 shows another example of the visualization image in which the occurrence time of the earthquake is mapped to the zaxis. From this image, the user can see how long a lot of small aftershocks continued after

Fig. 14. Visualization of seismic hypocenter data by mapping occurrence time to z-axis

Figure 15 shows the visualization of b-values calculated from the seismic hypocenter data. In this case, it was difficult for the user to recognize the distribution of the visualized b-

big earthquake.

Fig. 13. Visualization of seismic hypocenter data

value data, because too many point clouds were visualized simultaneously. Figure16 shows the visualization image of b-value data filtered by time parameter interactively. With the reduction of the number of displayed point data, the users can recognize the feature of spatially distributed data. These visualization images were rendered in real time through the user's interactive operation using the physical MIDI controller. Thus, in this application, the user could effectively examine the feature of the distribution of the earthquakes interactively.

Fig. 15. Visualization of b-value data

Fig. 16. Visualization of b-value data filtered by time

Immersive Visual Data Mining Based on Super High Definition Image 167

36.6051 139.8046 36.5895 139.8202 N36.6051\_E139.8046.dat

36.6051 139.8202 36.5895 139.8358 N36.6051\_E139.8202.dat

36.6051 139.8358 36.5895 139.8514 N36.6051\_E139.8358.dat

In this system, the user can access these databases from the virtual environment and visualize the retrieved data by specifying the condition. Thus, this system enables the user to understand the relationship among the hypocenter, terrain, basement depth, and plate structure data intuitively, and to analyze the feature of the earthquake phenomenon, by

As for the mechanism of integrating the visualized data in the virtual space, the plug-in function of the OpenCABIN library was used. In this system, each data is visualized by different application programs and they are integrated in the three-dimensional space in the runtime. Figure 17 and figure 18 show that several visualization data are integrated in the same space. In these examples, visualization programs for hypocenter data, terrain data, basement depth data and plate structure data are plugged-in to represent the relation among the data. When the hypocenter data were visualized using the sphere, the

….. *…..* ….. ….. …..

overlapping these data in the three-dimensional visualization environment.

Fig. 17. Visualization of relationship between terrain data and hypocenter data

east edge in

latitude filename

south edge in latitude

north edge in latitude

west edge in latitude

Table 4. Database table of terrain data

#### **6.2 Visualization of integrated data**

Next, this system was applied to analyze the relationship between several kinds of data by visualizing integrated image. In this case, several application programs that visualize hypocenter data, terrain data, basement depth data and plate structure data were integrated in the super high-definition immersive visual data mining environment. Though these data are stored in different database, every data has location information. Therefore, they can be related with each other using the location information as a reference key. Table 1 to Table 4 show the database table used in this system. Though the terrain data originally consist of altitude information corresponding to the latitude and longitude on the lattice, they are stored as data sets for the divided areas in the database.


Table 1. Database table of hypocenter data.


Table 2. Database table of basement structure data.


Table 3. Database table of plate depth data.


Table 4. Database table of terrain data

166 Applications of Virtual Reality

Next, this system was applied to analyze the relationship between several kinds of data by visualizing integrated image. In this case, several application programs that visualize hypocenter data, terrain data, basement depth data and plate structure data were integrated in the super high-definition immersive visual data mining environment. Though these data are stored in different database, every data has location information. Therefore, they can be related with each other using the location information as a reference key. Table 1 to Table 4 show the database table used in this system. Though the terrain data originally consist of altitude information corresponding to the latitude and longitude on the lattice, they are

latitude longitude depth magnitude date

36.3005 139.9837 40.68 0.8 2003-01-01

36.0927 138.7390 153.97 1.7 2003-01-01

36.2901 139.6655 121.42 1.3 2003-01-01

….. *…..* ….. ….. …..

latitude longitude depth

36. 505 138.510 3.057

36.505 138.515 2.801

36.505 138.520 2.661

….. ….. …..

latitude longitude depth

36.510 138.460 151.827

36.510 138.465 151.103

36.510 138.470 151.371

….. ….. …..

**6.2 Visualization of integrated data** 

stored as data sets for the divided areas in the database.

Table 1. Database table of hypocenter data.

Table 2. Database table of basement structure data.

Table 3. Database table of plate depth data.

In this system, the user can access these databases from the virtual environment and visualize the retrieved data by specifying the condition. Thus, this system enables the user to understand the relationship among the hypocenter, terrain, basement depth, and plate structure data intuitively, and to analyze the feature of the earthquake phenomenon, by overlapping these data in the three-dimensional visualization environment.

As for the mechanism of integrating the visualized data in the virtual space, the plug-in function of the OpenCABIN library was used. In this system, each data is visualized by different application programs and they are integrated in the three-dimensional space in the runtime. Figure 17 and figure 18 show that several visualization data are integrated in the same space. In these examples, visualization programs for hypocenter data, terrain data, basement depth data and plate structure data are plugged-in to represent the relation among the data. When the hypocenter data were visualized using the sphere, the

Fig. 17. Visualization of relationship between terrain data and hypocenter data

Immersive Visual Data Mining Based on Super High Definition Image 169

so that the user can easily visualize the data by changing the visualization method and visualization parameter in trials and errors. This platform was applied to the seismic data analysis, and several kinds of data such as map, terrain model, depth of basement and plate structure were visualized overlapped with the hypocenter data. Then, the effectiveness and possibility of the intuitive and accurate analysis in the immersive visual data mining

Future research will include developing more effective visual data mining method through the collaboration with the earthquake experts and applying this technology to other

This study was partially supported by Keio University Global COE program (Center for Education and Research of Symbiotic, Safe and Secure System Design). And we would like to thank Takashi Furumura, Shoji Itoh, Kengo Nakajima, Takahiro Katagiri (The University of Tokyo), Hanxiong Chen, Osamu Tatebe, Atsuyuki Morishima, Hiroto Tadano (University

Ammoura, A., Zaiane, O.R., Ji, Y. (2001). Immersed Visual Data Mining: Walking the Walk,

Ferey, N., Gros, P.E., Herisson, J., Gherbi, R. (2005). Visual Data Mining of Genomic

Furumura, T., Kennett, B.L.N. (2005). Subduction Zone Guided Waves and the

Kantardzie M. (2002). *Data Mining: Concepts, Models, Methods, and Algorithms*, Wiley-IEEE

Keim, D.A. (2002). Information Visualization and Visual Data Mining, *IEEE Transactions on* 

Nagel, H.R., Granum, E., Bovbjerg, S., Vittrup, M. (2008). Immersive Visual Data Mining:

Ogi, T., Daigo, H., Sato, S., Tateyama, Y., Nishida, Y. (2008). Super High Definition Stereo

Okada, Y., Kasahara, K., Hori, S., Obara, K., Sekiguchi, S., Fujiwara, H. Yamamoto, A. (2004).

Renambot, L., Jeong, B., Jagodic, R., Johnson, A., Leigh, J., Aguilera, J. (2006). Collaborative

Japan, *Journal of Geophysical Research*, Vol.110, B10302.1-B10302.27

*Visualization and Computer Graphics*, Vol.7, No.1, pp.100-107

*Conference on Artificial Reality and Telexistence)*, pp.365-366

Lecture Notes in Computer Science, Proc. of BNCOD 2001 (18th British National

Databases by Immersive Graph-Based Exploration, *Proc. of 3rd International Conference on Computer Graphics and Interactive Techniques in Australasia and South* 

Heterogeneity Structure of the Subducted Plate: Intensity anomalies in northern

The 3DVDM Approach, *Visual Data Mining: Theory, Techniques and Tools for Visual* 

Image Using 4K Projection System, *ICAT 2008 (Proceedings of 18th International* 

Recent progress of seismic observation networks in Japan –Hi-net, F-net, K-NET

Visualization using High-Resolution Tiled Displays, *CHI 06 Workshop on Information Visualization and Interaction Techniques for Collaboration Across Multiple Displays*

environment were confirmed through the interaction with the visualized data.

application fields.

**9. References** 

Press

**8. Acknowledgment** 

of Thsukuba) for their supports.

*East Asia*, pp.143-146

Conference on Databases), pp.202-218

*Analytics*, Springer-Verlag, pp.281-311

and KiK-net, *Earth Planets Space*, 56, xv-xxviii

size of it indicates the magnitude of the earthquake. The terrain data is created by mapping the texture image captured from the satellite onto the shape model. And the basement depth and the plate structure data are represented using the colors that indicate the depth values.

Fig. 18. Visualization of relationship among hypocenter data, basement depth data and plate structure data.

In this system, when the several visualization programs are plugged-in, the toggle buttons that show the conditions of each data are displayed. The user can switch the visible condition of each data by using the toggle button in the virtual space. For example, the user can change the visualization data from the combination of hypocenter data and basement depth data to the combination of hypocenter data and plate structure data, while running the application programs in the visualization environment. By using this method, the user could intuitively understand the feature of the distribution of hypocenter and the relation with other data. For example, the user could see whether the attention data of the earthquake occurred on the plate or in the plate structure. Thus, this system could be effectively used to represent the relationship among several kinds of data in the threedimensional space and to analyze the earthquake phenomenon.

## **7. Conclusion**

In this study, the super high-definition immersive visual data mining environment that uses 4K stereo projectors was constructed. In this system, it is expected that the effect of visual data mining is greatly improved, since the super high-definition stereo image transmits a large amount of information from the computer to the user and the interactive interface enables the user to explore the data space. However, the result of the experiments also suggested the limitations of representation ability of the super high-definition image. Therefore, the platform of immersive visual data mining for point cloud data was developed so that the user can easily visualize the data by changing the visualization method and visualization parameter in trials and errors. This platform was applied to the seismic data analysis, and several kinds of data such as map, terrain model, depth of basement and plate structure were visualized overlapped with the hypocenter data. Then, the effectiveness and possibility of the intuitive and accurate analysis in the immersive visual data mining environment were confirmed through the interaction with the visualized data.

Future research will include developing more effective visual data mining method through the collaboration with the earthquake experts and applying this technology to other application fields.

#### **8. Acknowledgment**

168 Applications of Virtual Reality

size of it indicates the magnitude of the earthquake. The terrain data is created by mapping the texture image captured from the satellite onto the shape model. And the basement depth and the plate structure data are represented using the colors that indicate the depth values.

Fig. 18. Visualization of relationship among hypocenter data, basement depth data and plate

In this system, when the several visualization programs are plugged-in, the toggle buttons that show the conditions of each data are displayed. The user can switch the visible condition of each data by using the toggle button in the virtual space. For example, the user can change the visualization data from the combination of hypocenter data and basement depth data to the combination of hypocenter data and plate structure data, while running the application programs in the visualization environment. By using this method, the user could intuitively understand the feature of the distribution of hypocenter and the relation with other data. For example, the user could see whether the attention data of the earthquake occurred on the plate or in the plate structure. Thus, this system could be effectively used to represent the relationship among several kinds of data in the three-

In this study, the super high-definition immersive visual data mining environment that uses 4K stereo projectors was constructed. In this system, it is expected that the effect of visual data mining is greatly improved, since the super high-definition stereo image transmits a large amount of information from the computer to the user and the interactive interface enables the user to explore the data space. However, the result of the experiments also suggested the limitations of representation ability of the super high-definition image. Therefore, the platform of immersive visual data mining for point cloud data was developed

dimensional space and to analyze the earthquake phenomenon.

structure data.

**7. Conclusion** 

This study was partially supported by Keio University Global COE program (Center for Education and Research of Symbiotic, Safe and Secure System Design). And we would like to thank Takashi Furumura, Shoji Itoh, Kengo Nakajima, Takahiro Katagiri (The University of Tokyo), Hanxiong Chen, Osamu Tatebe, Atsuyuki Morishima, Hiroto Tadano (University of Thsukuba) for their supports.

#### **9. References**


**9** 

**Realizing Semantic Virtual Environments with** 

Multi-User Virtual Environment (MUVE) has attracted much attention recently due to the increasing number of users and potential applications. Fig. 1 shows the common components that a MUVE system may provide. Generally speaking, a MUVE refers to a virtual world that allows multiple users to log in concurrently and interact with each other by texts or graphics provided by the system. On-line games can be considered as a special kind of virtual environment with specific characters, episode, and ways of interactions. Other MUVE systems such as SecondLife provide a general framework for users to design their own 3D contents and interact with other users through their avatars in a more general way. Although the users are allowed to build their own world, the animations that can be displayed are limited to those that have been prepared by the system. In addition, due to the lack of semantic information, it is not feasible to design virtual avatars that are controlled by

Under the concept of web 2.0, we think future virtual environments will also depend on how easily the users can share their own designs of procedures for customized animations and high-level behaviours. However, it is a great challenge to design an extensible virtual environment system that allows the users to write their own customized procedures that can dynamically acquire the information of the virtual environment and other users. In our previous work, we have succeeded in extending a MUVE system developed by ourselves, called IMNET (Li et al., 2005), to allow user-defined animation procedures to be specified, downloaded, and executed on the fly (Chu et al., 2008). However, in order to enable these user-defined procedures to create richer animations for interactions, we must be able to describe the semantics of the objects in the world in a standard way accessible to all

In this paper, we aim to make use of ontology to describe the semantics of the objects in the virtual environment such that users can design their own animation procedures based on the information. For examples, if we can acquire object information such as 3D geometry, height, and 2D approximation, we can design a motion planning procedure that can generate a collision-free path for the avatar to walk to a given destination. In addition, we have also designed the ontology for information exchange between avatars and added a new information query mechanism to facilitate the communication between avatars. These

**1. Introduction** 

the computer to interact with other avatars.

potential animation/behaviour designers.

**Ontology and Pluggable Procedures** 

*Department of Computer Science, National Chengchi University, Taipei* 

Yu-Lin Chu and Tsai-Yen Li

 *Taiwan, ROC* 

