**Realizing Semantic Virtual Environments with Ontology and Pluggable Procedures**

Yu-Lin Chu and Tsai-Yen Li

*Department of Computer Science, National Chengchi University, Taipei Taiwan, ROC* 

#### **1. Introduction**

170 Applications of Virtual Reality

Tateyama, Y., Oonuki, S., Ogi, T. (2008). K-Cave Demonstration: Seismic Information

Wong, P.C. (1999). Visual Data Mining, *Computer Graphics and Applications*, Vol.19, No.5,

*International Conference on Artificial Reality and Telexistence)*, pp.363-364 Wegman, E.J., Symanzik, J. (2002). Immersive Projection Technology for Visual Data Mining,

*Journal of Computational and Graphical Statistics*, Vol.11, pp.163-188

pp.20-21

Visualization System Using the OpenCABIN Library, *ICAT 2008 (Proceedings of 18th* 

Multi-User Virtual Environment (MUVE) has attracted much attention recently due to the increasing number of users and potential applications. Fig. 1 shows the common components that a MUVE system may provide. Generally speaking, a MUVE refers to a virtual world that allows multiple users to log in concurrently and interact with each other by texts or graphics provided by the system. On-line games can be considered as a special kind of virtual environment with specific characters, episode, and ways of interactions. Other MUVE systems such as SecondLife provide a general framework for users to design their own 3D contents and interact with other users through their avatars in a more general way. Although the users are allowed to build their own world, the animations that can be displayed are limited to those that have been prepared by the system. In addition, due to the lack of semantic information, it is not feasible to design virtual avatars that are controlled by the computer to interact with other avatars.

Under the concept of web 2.0, we think future virtual environments will also depend on how easily the users can share their own designs of procedures for customized animations and high-level behaviours. However, it is a great challenge to design an extensible virtual environment system that allows the users to write their own customized procedures that can dynamically acquire the information of the virtual environment and other users. In our previous work, we have succeeded in extending a MUVE system developed by ourselves, called IMNET (Li et al., 2005), to allow user-defined animation procedures to be specified, downloaded, and executed on the fly (Chu et al., 2008). However, in order to enable these user-defined procedures to create richer animations for interactions, we must be able to describe the semantics of the objects in the world in a standard way accessible to all potential animation/behaviour designers.

In this paper, we aim to make use of ontology to describe the semantics of the objects in the virtual environment such that users can design their own animation procedures based on the information. For examples, if we can acquire object information such as 3D geometry, height, and 2D approximation, we can design a motion planning procedure that can generate a collision-free path for the avatar to walk to a given destination. In addition, we have also designed the ontology for information exchange between avatars and added a new information query mechanism to facilitate the communication between avatars. These

Realizing Semantic Virtual Environments with Ontology and Pluggable Procedures 173

as configuration, scheduling, and interaction. These systems constructed knowledge-level representation of the virtual scene and supported high-level processing with a natural language interface. These systems also used causal behaviours to replace the simulation of

The concept of semantic virtual environment was proposed to facilitate advanced applications that allow better communications between agents and the environment (Otto, 2005; Kleinermann et al., 2007). Unlike traditional virtual environments that were designed mainly for visual effects, semantic virtual environments are designed to contain richer structural semantic information adequate for a computer to process (Otto, 2005). For example, a plate on top of four sticks may be easily interpreted as a table by a human but it will be more difficult for a machine to infer its function from low-level geometry reasoning. An analogy of this unstructured information for visual interpretation is the vast home pages on the web. This is also why semantic web was proposed to facilitate automatic processing and communication among web servers. Similarly, in order to facilitate the reuse of 3D design and animation procedures, it is crucial to annotate the objects in a virtual world with semantic information in a standard format such that the same contents can be reused in

The idea of SVE has been realized from various aspects by much work in the literature. For example, the SEVEN system was proposed by Otto (2005) as an example of SVE with the concept of software components. It focuses more on the reusability of system components on different MUVE systems instead of components designed by the users. Gutierrez et al. (2005) also have proposed an ontology for virtual human by incorporating semantics into human shapes. They also regarded that the design of ontology is a continuous process where richer semantic information about human attributes should be added in a collaborative fashion. Instead of dealing with human shapes, Abaci et al. (2005) focused on adding action semantics in smart objects to facilitate the interaction between the avatars and with the objects in the virtual environment. Garcia-Rojas et al. (2006) also proposed to add semantic information, such as emotion and expressiveness, to animations in order to facilitate the selection of appropriate animations for a specific

For developing and sharing animation procedures in the IMNET system, we will describe our ontology design for virtual objects and avatars in IMNET in this section. The topology described here is to serve as a demonstration example for potential applications; therefore,

The objective of ontology design for virtual environment in this work is two-fold. First, we would like to keep the information that exists in the original IMNET such as object geometry and transformation. Second, we hope to use an example to show that more semantic information about the virtual objects can facilitate the computation of advanced reasoning

procedures such as a path planner that may be designed by the users.

physical features.

scenario.

**3. Ontology design in IMNET** 

the design is by no means complete.

**3.1 Ontology design of virtual environment** 

different worlds or in different MUVE systems.

new functions will be demonstrated through several examples where the user-designed programs acquire application-specific semantics in the standard ontology format after the programs have been deployed dynamically to other clients' machines.

Fig. 1. Common components in a multi-user virtual environment: (a) login; (c) choose an avatar; (b) interact with virtual world; (d) a scripting interface (optional)

The remaining of the paper is organized as follows. In the next section, we will review the research related to our work. In the third section, we will describe the example design of ontology for the objects in the virtual worlds and for the avatars. In Section 4, we will describe the improved communication protocol allowing on-demand query of information among avatars. Then, in the following two sections, we will give examples of how to design animation components that can take advantage of the semantic information to generate richer user behaviours. Finally, we will conclude the paper with future directions.

#### **2. Related work**

In this work, we aim to provide a smart virtual environment that can enable richer contents and interactions. Before the concept of semantic virtual environment emerges, there has been much research about how to integrate AI technologies into a virtual environment system. R. Aylett et al. (2001) found that this type of virtual environments have several common features. For example, these systems added components for solving problems such

new functions will be demonstrated through several examples where the user-designed programs acquire application-specific semantics in the standard ontology format after the

Fig. 1. Common components in a multi-user virtual environment: (a) login; (c) choose an

(c) (d)

(a) (b)

The remaining of the paper is organized as follows. In the next section, we will review the research related to our work. In the third section, we will describe the example design of ontology for the objects in the virtual worlds and for the avatars. In Section 4, we will describe the improved communication protocol allowing on-demand query of information among avatars. Then, in the following two sections, we will give examples of how to design animation components that can take advantage of the semantic information to generate richer user behaviours. Finally, we will conclude the paper with future

In this work, we aim to provide a smart virtual environment that can enable richer contents and interactions. Before the concept of semantic virtual environment emerges, there has been much research about how to integrate AI technologies into a virtual environment system. R. Aylett et al. (2001) found that this type of virtual environments have several common features. For example, these systems added components for solving problems such

avatar; (b) interact with virtual world; (d) a scripting interface (optional)

directions.

**2. Related work** 

programs have been deployed dynamically to other clients' machines.

as configuration, scheduling, and interaction. These systems constructed knowledge-level representation of the virtual scene and supported high-level processing with a natural language interface. These systems also used causal behaviours to replace the simulation of physical features.

The concept of semantic virtual environment was proposed to facilitate advanced applications that allow better communications between agents and the environment (Otto, 2005; Kleinermann et al., 2007). Unlike traditional virtual environments that were designed mainly for visual effects, semantic virtual environments are designed to contain richer structural semantic information adequate for a computer to process (Otto, 2005). For example, a plate on top of four sticks may be easily interpreted as a table by a human but it will be more difficult for a machine to infer its function from low-level geometry reasoning. An analogy of this unstructured information for visual interpretation is the vast home pages on the web. This is also why semantic web was proposed to facilitate automatic processing and communication among web servers. Similarly, in order to facilitate the reuse of 3D design and animation procedures, it is crucial to annotate the objects in a virtual world with semantic information in a standard format such that the same contents can be reused in different worlds or in different MUVE systems.

The idea of SVE has been realized from various aspects by much work in the literature. For example, the SEVEN system was proposed by Otto (2005) as an example of SVE with the concept of software components. It focuses more on the reusability of system components on different MUVE systems instead of components designed by the users. Gutierrez et al. (2005) also have proposed an ontology for virtual human by incorporating semantics into human shapes. They also regarded that the design of ontology is a continuous process where richer semantic information about human attributes should be added in a collaborative fashion. Instead of dealing with human shapes, Abaci et al. (2005) focused on adding action semantics in smart objects to facilitate the interaction between the avatars and with the objects in the virtual environment. Garcia-Rojas et al. (2006) also proposed to add semantic information, such as emotion and expressiveness, to animations in order to facilitate the selection of appropriate animations for a specific scenario.
