**Immersive Visual Data Mining Based on Super High Definition Image**

Tetsuro Ogi1, Yoshisuke Tateyama1 and So Sato2 *1Graduate School of System Design and Management, Keio University 2System Design and Management Research Institute, Keio University Japan* 

#### **1. Introduction**

152 Applications of Virtual Reality

De Almeida Silva, Tadeu Augusto; Farias, O. L. M. (2009). A Virtual Environment for

Elmasri, R.; Navathe, S. B. (1994). - Fundamentals of Database Systems, Redwood City:

Farias, O. L. M; Santos N. (2005). Agent-Based Geographical Information System. International

http://code.google.com/intl/en/apis/maps/documentation/javascript/tutorial.h

Lim, K. ; Deadman, P. J.; Moran, E. ; Brondízio, E. ; McCracken, S.(2002).Agent-based

Longley, Paul A.; Batty, Michael (2003). Advanced Spatial Analysis: The Casa Book of GIS.,

Maguire, David; BATTY, Michael; Goodchild, Michael (2005). GIS, Spatial Analysis, and

North, Michael J.; Collier , Nicholson T.; Vos, Jerry R. (2006). Experience Creating Three

Soares Filho, B. S.; Cerqueira, G. C.; (2002). DINAMICA: A new model to simulate and study

Winner, N.(1948/1961). Cybernetics, or Control and Communication in the Animal and

Wooldridge, M.J. (2002). An Introduction to Multiagent Systems, John Wiley & Sons,

Wooldridge, M. J., Jennings, N. R. (1995). Agents Theories, Architectures, and Languages: a

Veldkamp A.; Fresco, L. O (1996). CLUE: a conceptual model to study the conversion of land

Verburg, P. H.; Soepboer, W.; Veldkamp, A.; Limpiada, R.; Espaldon, V.; Mastura, S. S. A.

Survey. Proceedings of ECAI'94 Workshop on Agent Theories, Architectures and

(2002). Modeling the spatial dynamics of regional land use: the CLUE-S model. In

Oracle (2011). MySQL Enterprise Edition Trial from http://www.mysql.com/trials/

IAWTIC'2005, v. 1. p. 998-1005. , Edited by M. Mohammadian, Vienna. Gardner, M.(1971).Mathematical games: On cellular automata, self-reproduction, the Garden of Eden, and the game "life". Scientific American 224(2):112-117. Gimblett, H. Randy (2002). Integrating Geographic Information Systems and Agent-Based

Education (CONTE). American Nuclear Society. Florida, USA.

Benjamin. Cummings Publishing Company, Inc.

Google (October, 2011). Google Maps Javascript API V3 Tutorial from

Levy, S. (1992). Artificial Life. The Quest for a New Creation. Penguin.

York: Oxford University Press, pp. 277–310.

Modelling and Computer, vol. 16, no. 1.

landscape dynamics. Ecological Modeling.

Wolfram, S.(2002). A new kind of science. Wolfram Media, Incorporated.

use and its effects. Ecological Modeling 85, pp.253-270.

Environmental Management, Vol. 30, No. 3, pp. 391-405.

Machine. Cambridge, MA, MIT Press.

University Press, USA.

tml#Intro

ESRI Press.

Modeling. ESRI Press.

London, England.

Languages, pp.1-22.

Simulation of Radiological Accidents. Conference on Nuclear Training and

Conference on Intelligent Agents, Web Technologies and Internet Commerce -

Modeling Techniques for Simulating Social and Ecological Processes. Oxford,

simulations of household decision making and land use change near Altamira, Brazil. In Integrating Geographic Information Systems and Agent-Based Modeling: Techniques for Simulating Social and Ecological Processes, ed. R. Gimblett, New

Implementations of the Repast Agent Modelling Toolkit. ACM Transaction

In recent years, a large amount of data has been generated and recorded in various fields according to the advances of computer simulation, sensor network, and database technologies. However, it is generally difficult to find valuable information or knowledge from the huge data. Therefore, the establishment of a methodology to utilize these data effectively is an important issue. Though data mining is one of the methods to solve such a problem, it is difficult for the user to understand the process of mining data and to evaluate the result intuitively, because the data is processed in the computer (Kantardzie, 2002). On the other hand, the visualization technique can be used effectively to represent the data so that the user can understand it intuitively. This study focuses on the method of visual data mining in which the user performs the data mining by visualizing the data (Wong, 1999) (Keim, 2002).

In the visual data mining, the user can analyze the data interactively with the computer by visualizing the process and the result of the data mining. Then, it is expected that a new information or knowledge can be found by combining the ability of high-speed calculation of the computer with the human's ability such as intuition, common sense and creativity. Particularly, this study aims at improving the effect of visual data mining by enhancing the ability of expressing data and interaction function by using the immersive virtual environment based on super high-definition stereo images. In this system, it is expected that the user can perform accurate and intuitive data mining by using the high resolution stereo images.

This paper discusses the system architecture and the visualization ability of the super highdefinition immersive visual data mining environment that was developed in this study, and the effectiveness of this method is evaluated by applying the platform of super highdefinition immersive visual data mining to the seismic data analysis.

#### **2. Concept of super high-definition immersive visual data mining**

Visual data mining supports the user to analyze a large amount of data and to find new information by combining the abilities of computer and human. The computer has large memory and high speed calculation capability. On the other hand, the human is superior in intuition, commonsense and creativity based on the visual information. By combining the

Immersive Visual Data Mining Based on Super High Definition Image 155

4,096 x 2,160 pixels, and its image quality is more than four times of the usual highdefinition image. In this system, two stacked projectors (SONY SRX-S110) are used, and the images seen from the right eye and left eye positions of the user are projected. The images output from the projectors are rear projected onto the 180-inch acrylic rear projection screen (Nippura, Blue Ocean Screen) through the polarizing filter. In this condition, 1 pixel in the projected image has approximately 0.97 mm width on the screen. Then, the user can see the high resolution passive stereo image based on the binocular parallax by wearing the polarized 3D glasses. In this system, since two 108-inch LCD monitors are placed at both sides of the 4K screen, various kinds of information can be displayed in the multi-display environment. Figure 3 shows the system configuration of the super high-definition

immersive visual data mining environment.

4K projector Sony SRX-S110

Sharp 108-inch LCD monitor

> polarized 3D glasses

Fig. 2. Super high-definition immersive visual data mining environment.

Sharp 108-inch LCD monitor

Fig. 3. System configuration of immersive visual data mining environment.

switch Polarizing filter

Dell Precision T7400 Nippura Blue Ocean

matrix

USB game controller

GUI interface

physical MIDI controller

video server

graphics workstation

Dell Precision T7400 + Nvidia Quadro Plex 1000

(master)

graphics workstation

(renderer)

information processes performed by computer and human, a large amount of data could be analyzed effectively to find new information.

In the researches by Wong (Wong, 1999) or Keim et al. (Keim, 2002), data mining is effectively performed in the visualization environment. In these cases, visualization technology is used to transmit the information from the computer to the human and interactive interface technology is used to support the collaboration between human and computer. If the expression abilities of information visualization and the interaction function between human and computer were increased, the performance of visual data mining would be improved based on the improvement of information processing in both human and computer.

As for the increase of the expression ability in the information visualization, super highdefinition image would enables the transmission of the detailed information from the computer to the human, and three-dimensional stereo image would be used effectively to represent the relationship among several kinds of data in the three-dimensional space. In the researches by Renambot (Renambot, 2006), high-resolution image is effectively used in the data visualization. And as for the interaction between human and computer, immersive virtual environment would be used to enable the user to operate the visualized data directly and to explore data space as first person experience. Ammoura (Ammoura, 2001), Wegman (Wegman, 2002) and Ferey (Ferey, 2005) discuss the effectiveness of immersive virtual environment in visual data mining.

In this study, super high-definition immersive visual data mining environment was constructed to utilize the effect of advanced visualization and interaction technologies. Figure 1 shows the concept of super high-definition immersive visual data mining that is proposed in this study. In this figure, it is shown that the performance of visual data mining can greatly be improved by increasing the expression ability and the interaction function between computer and human.

#### **3. Immersive visual data mining environment**

#### **3.1 4K stereo image**

In this study, immersive visual data mining environment using 4K stereo projection system was constructed as shown in figure 2 (Ogi, 2008). 4K means super high resolution image of

information processes performed by computer and human, a large amount of data could be

In the researches by Wong (Wong, 1999) or Keim et al. (Keim, 2002), data mining is effectively performed in the visualization environment. In these cases, visualization technology is used to transmit the information from the computer to the human and interactive interface technology is used to support the collaboration between human and computer. If the expression abilities of information visualization and the interaction function between human and computer were increased, the performance of visual data mining would be improved based on the improvement of information processing in both human

As for the increase of the expression ability in the information visualization, super highdefinition image would enables the transmission of the detailed information from the computer to the human, and three-dimensional stereo image would be used effectively to represent the relationship among several kinds of data in the three-dimensional space. In the researches by Renambot (Renambot, 2006), high-resolution image is effectively used in the data visualization. And as for the interaction between human and computer, immersive virtual environment would be used to enable the user to operate the visualized data directly and to explore data space as first person experience. Ammoura (Ammoura, 2001), Wegman (Wegman, 2002) and Ferey (Ferey, 2005) discuss the effectiveness of immersive virtual

In this study, super high-definition immersive visual data mining environment was constructed to utilize the effect of advanced visualization and interaction technologies. Figure 1 shows the concept of super high-definition immersive visual data mining that is proposed in this study. In this figure, it is shown that the performance of visual data mining can greatly be improved by increasing the expression ability and the interaction function

> improvement of information expression

visual data mining

In this study, immersive visual data mining environment using 4K stereo projection system was constructed as shown in figure 2 (Ogi, 2008). 4K means super high resolution image of

Fig. 1. Concept of super high-definition immersive visual data mining.

**3. Immersive visual data mining environment** 

improvement of interaction function user

analysis new knowledge

analyzed effectively to find new information.

environment in visual data mining.

between computer and human.

super high-definition stereo image

immersive display

database

**3.1 4K stereo image** 

broad-band network

workstation

and computer.

4,096 x 2,160 pixels, and its image quality is more than four times of the usual highdefinition image. In this system, two stacked projectors (SONY SRX-S110) are used, and the images seen from the right eye and left eye positions of the user are projected. The images output from the projectors are rear projected onto the 180-inch acrylic rear projection screen (Nippura, Blue Ocean Screen) through the polarizing filter. In this condition, 1 pixel in the projected image has approximately 0.97 mm width on the screen. Then, the user can see the high resolution passive stereo image based on the binocular parallax by wearing the polarized 3D glasses. In this system, since two 108-inch LCD monitors are placed at both sides of the 4K screen, various kinds of information can be displayed in the multi-display environment. Figure 3 shows the system configuration of the super high-definition immersive visual data mining environment.

Fig. 2. Super high-definition immersive visual data mining environment.

Fig. 3. System configuration of immersive visual data mining environment.

Immersive Visual Data Mining Based on Super High Definition Image 157

besides the visualization program for each data. However, the plug-in mechanism can add a visualization function for necessary data to the application program that is being executed. Namely, the integrated visualization environment for several kinds of data can be constructed, by executing several visualization programs for each data simultaneously by

In the data visualization using the 4K image, detailed information can be represented based on the high resolution image compared with the conventional visualization using the SXGA or WUXGA resolution image. Particularly, when the 4K stereo image is used, it is expected that the detailed three-dimensional information can be displayed with high accuracy in the three-dimensional virtual space. In this study, the representation ability of the 4K stereo

First, the spatial resolution with which the visualized data can be perceived in the threedimensional space was experimentally measured. In this experiment, the subjects sat at a position 3m away from the screen, and two vertical parallel lines with the gaps of 0.5mm, 1mm, 2.5mm, 5mm, or 10mm were displayed randomly in the direction from the front to left as shown in figure 5. The subjects were asked to move the parallel lines in the depth direction and to stop them at the boundary position where they can recognize the displayed image as two lines by using the USB game controller. The boundary positions of the parallel

gap

3m

subject

Figure 6 shows the result of the experiment for five subjects with visual acuity of more than 1.0. In this graph, the contour lines are drawn by connecting the average values of the

screen

0.5, 1, 2.5, 5 or 10mm

using the plug-in mechanism.

**4.1 Perception of spatial resolution** 

**4. Data representation ability of 4K stereo image** 

image in the immersive visual data mining was examined.

lines with various gaps and in various directions were recorded.

parallel lines

field of view

Fig. 5. Condition of experiment on the perception of spatial resolution.

move

In order to generate and render the 4K stereo image, two high-end graphics workstations (Dell Precision T7400, 2xQuad Core Xeon 3.2GHz) with the graphics engine (NVIDIA Quadro Plex 1000 Model IV) that has a genlock function are used for right eye image and the left eye image, respectively. In this system, interface devices such as a USB game controller and a physical MIDI controller are connected to the graphics workstation. The USB game controller is used to walk through the visualized data space or to change the visualization method. On the other hand, the physical MIDI controller is used to change the visualization parameter minutely. By using these interface devices, the user can perform accurate and intuitive interaction with the visualized data, and the immersive visual data mining using 4K resolution stereo image can be realized.

### **3.2 Software structure**

Figure 4 shows the software structure of the immersive visual data mining environment. As for the software library to develop the visual data mining application in the immersive virtual environment, Open CABIN Library was used (Tateyama, 2008). Open CABIN Library is a software platform to develop the immersive virtual reality application. This library has features of master-renderer programming and plug-in mechanism.

Fig. 4. Software structure of immersive visual data mining application.

The master-renderer programming is a style that constructs an application program consisting of a master part that calculates the state of the virtual world and renderer parts that render the image of the virtual world. This software configuration is effective to construct the network application that needs to access to the database or needs to transmit data to other system through the network, because master process has only to communicate with the remote processes.

On the other hand, the plug-in mechanism is a method to construct an application program in the form of dynamically loaded library. When API of the usual library is used, it is necessary to rewrite the source code of the program to add a certain function to improve it. For example, in order to construct an application program that visualizes several kinds of data simultaneously, the integrated visualization program must be developed specially besides the visualization program for each data. However, the plug-in mechanism can add a visualization function for necessary data to the application program that is being executed. Namely, the integrated visualization environment for several kinds of data can be constructed, by executing several visualization programs for each data simultaneously by using the plug-in mechanism.
