**The Design and Implementation of On-Line Multi-User Augmented Reality Integrated System**

Hsiao-shen Wang and Chih-Wei Chiu *National Taichung University of Education/ Hyweb Technology Co., Ltd. Taiwan* 

## **1. Introduction**

226 Augmented Reality – Some Emerging Application Areas

Zhong, X., Boulanger, P. (2002). Collaborative augmented reality: A prototype for industrial

Today's technological advancements, many innovative applications continue to emerge, and in supporting education and learning has brought many changes. With these changes, the application of using virtual reality technology has greatly different on the educational learning way compared to the traditional computer-assisted instruction. Such as abstract concepts simulation, virtual object manipulation, and interactive 3D gaming system, etc.

Through innovative technology-based learning, many learners do produce effective learning, and based on this learning effectiveness, more and more different kinds of technology-transfer medium system were requested to support learners more realistic environment in their computer-based learning system. Therefore, Augmented Reality (AR) technology gains attention in educational use because of its feature of combing real-life situation and the characteristics of virtual objects.

AR technology, through the heavy helmet display, until using the webcam and markers directly to display the result, the combing of real-life scenes and the virtual objects, on the screen, is a mature technology can be applied to assist students with learning. However, as the fixed correspondence of each maker and virtual object in AR learning system, the development of each learning courseware based on AR technology is time-consuming. Thus, how to apply software engineering methods, combined with the AR technology in education to promote more effective use of AR-based learning programs, is a subject can't be ignored.

Cooperative learning is an important way of learning in a modern educational environment. Remote cooperative learning is also an inevitable way of learning in today's internet era. Collaborative learning, from the past face to face discussions to the current Internet remote distance learning, is constantly changing, but the key is how learners can really communicate with each other between the meaning of sharing and the achievement of interaction. Based on this critical factor, many Internet-based collaborative learning systems were proposed. These systems are common, mainly through the use of the real environment among learners or the share of the virtual world generated by the system, to allow learners to engage in a dialogue between the real scenes and common virtual objects for discussion.

The Design and Implementation of On-Line Multi-User Augmented Reality Integrated System 229

(retrieved from http://studierstube.icg.tu-graz.ac.at)

Fig. 2.1 Co-located collaborative AR.

(Billinghurst et al. , 2003)

Fig. 2.2 Co-located collaborative AR2.

However, in VR-based collaborative environments, each of the client location includes a real scene and each virtual objects generated by its VR system. How to make virtual objects of each AR system could be able shown between learners in a collaborative environment is a key problem of Cooperative learning. Thus, the purpose of the paper is to propose an online multi-user augmented reality integrated system (OMARIS) to enhance students learning in AR collaborative environment.

## **2. Related work**

Using technology-based application to overcome student learning difficulties is one of important and innovative ways in education environment. Because of its virtual interactive and simulative features, the applications of using Virtual Reality (VR) technology have been widely to use and to enhance several subject domains learning effect in school, such as computer assembly, art appreciation, emotional expression, medical training and geological science (Wang, 1999, 2000,; Basori et al., 2009; Deng & Zhou, 2009; Ni et al., 2009). With the same application of technology benefits, Augment Reality (AR) also plays a significant substantial advantage in education.

By seamlessly combining the real world with the various virtual materials, AR technology provides intuitive interaction experience to the learners. The various virtual materials, including graph, image, text, or animation, are superimposed on the reality scene based on the instructional design. With the characteristic of AR tehnology, Shelton and Hedley (2002) successfully applied the intuitive features in the nine planets learning activities which allowed students to construct and manipulate virtual objects through reality scene, and to establish their abstract scientific concepts. In recent years, AR technology has been applied in medical procedures (Rosenblum & Julier, 2007; Samset et al, 2008), assembly design and planning (Ong, Pang, Nee, 2007), mathematical education (Lee & Lee, 2008), Physics (Beaney & Namee, 2008), PCDIY (Chiang et al., 2011), city maps(Jiang et al., 2011) etc. Kaufman(2006) indicated that whether teachers or students are very interested in the AR-based geometry learning activities and have positive attitude to use AR-related software in the future curriculum. However, the creation of each AR application is very time-consuming, and therefore how to reuse developed materials, including markers and virtual objects, to create a new suitable AR courseware is one of must be addressed problems for the substance of the school curriculum needs.

In AR environment, students create their understanding of the learning content through the fusion of reality scene and virtual objects, and share and discuss with other students to strengthen the significance of domain knowledge. AR simply can provide a collaborative interactive AR environment for school setting, where students can interact naturally and intuitively. AR collaboration approach can be effectively used to develop face to face interfaces. There are two kinds of AR collaborative environments which are co-located collaborative AR and remote collaborative AR (Silva, Giraldi & Oliveira, 2003). The characteristic of co-located collaborative AR is that multiple users manipulate virtual objects within an augmented reality space in the same physical environment, and through the speech, gestures, eye contact and other means of communication of each other to share their meaning and reach the purpose of discussion.(Fig. 2.1, Fig.2.2)

(retrieved from http://studierstube.icg.tu-graz.ac.at) Fig. 2.1 Co-located collaborative AR.

228 Augmented Reality – Some Emerging Application Areas

However, in VR-based collaborative environments, each of the client location includes a real scene and each virtual objects generated by its VR system. How to make virtual objects of each AR system could be able shown between learners in a collaborative environment is a key problem of Cooperative learning. Thus, the purpose of the paper is to propose an online multi-user augmented reality integrated system (OMARIS) to enhance students learning

Using technology-based application to overcome student learning difficulties is one of important and innovative ways in education environment. Because of its virtual interactive and simulative features, the applications of using Virtual Reality (VR) technology have been widely to use and to enhance several subject domains learning effect in school, such as computer assembly, art appreciation, emotional expression, medical training and geological science (Wang, 1999, 2000,; Basori et al., 2009; Deng & Zhou, 2009; Ni et al., 2009). With the same application of technology benefits, Augment Reality (AR) also plays a significant

By seamlessly combining the real world with the various virtual materials, AR technology provides intuitive interaction experience to the learners. The various virtual materials, including graph, image, text, or animation, are superimposed on the reality scene based on the instructional design. With the characteristic of AR tehnology, Shelton and Hedley (2002) successfully applied the intuitive features in the nine planets learning activities which allowed students to construct and manipulate virtual objects through reality scene, and to establish their abstract scientific concepts. In recent years, AR technology has been applied in medical procedures (Rosenblum & Julier, 2007; Samset et al, 2008), assembly design and planning (Ong, Pang, Nee, 2007), mathematical education (Lee & Lee, 2008), Physics (Beaney & Namee, 2008), PCDIY (Chiang et al., 2011), city maps(Jiang et al., 2011) etc. Kaufman(2006) indicated that whether teachers or students are very interested in the AR-based geometry learning activities and have positive attitude to use AR-related software in the future curriculum. However, the creation of each AR application is very time-consuming, and therefore how to reuse developed materials, including markers and virtual objects, to create a new suitable AR courseware is one of must be addressed problems for the substance of the

In AR environment, students create their understanding of the learning content through the fusion of reality scene and virtual objects, and share and discuss with other students to strengthen the significance of domain knowledge. AR simply can provide a collaborative interactive AR environment for school setting, where students can interact naturally and intuitively. AR collaboration approach can be effectively used to develop face to face interfaces. There are two kinds of AR collaborative environments which are co-located collaborative AR and remote collaborative AR (Silva, Giraldi & Oliveira, 2003). The characteristic of co-located collaborative AR is that multiple users manipulate virtual objects within an augmented reality space in the same physical environment, and through the speech, gestures, eye contact and other means of communication of each other to share their

meaning and reach the purpose of discussion.(Fig. 2.1, Fig.2.2)

in AR collaborative environment.

substantial advantage in education.

school curriculum needs.

**2. Related work** 

(Billinghurst et al. , 2003) Fig. 2.2 Co-located collaborative AR2.

The Design and Implementation of On-Line Multi-User Augmented Reality Integrated System 231

instructors to set up learning scenarios, and students to explore individual AR learning activities. The last multi-user cooperative AR learning system is to provide students cooperative learning activities in AR environment. Detailed system design description, as

In general AR system, the link between the marker and the virtual object are fixed, so that the materials of the AR system are difficult to reuse. Thereby, how to manage makers, objects, and re-setup the link relationship between them is the main idea of the system. Based on the idea, the marker and object database system includes three subsystems: marker database system, object database system and marker and object matching system. The first two material database systems basically manage the uploaded materials, including marker symbols and virtual objects by authorized users. The latter one application system is based on the needs of instructional designers or general users, to set up the link between marker symbols and virtual objects in the database systems and to output as a single XML learning script file for future instructional use. After the linking relationship processed by the system, it then will be provided as the necessary materials

Since the innovation development of AR technology in education, the AR application makes the learner more motivation and substance of learning, thus providing individualized and flexible AR learning system for the learners is a key factor of the system development. Based on the idea, the personal AR learning system is designed for the instructors and learners. The instructors can select the predefine AR learning scenario, defined from previously material database system, or can customize the current AR learning script file for specific instructional needs in the personal AR learning system. The learner then can choose related topic to explore the combinational effect of the reality scenes and virtual objects in personal

This collaborative learning environment is especially to provide multiple users to be able to sharing mixed AR contents from their computer system, including reality scene and virtual objects. As the virtual objects are virtual feature in the client computer system, they can't be display directly through video devices to remote collaborative computers. Therefore, in this study, Flash Actionscript 3.0 with flartoolkit AR library and PaperVision three-dimensional display library, and the point to point transmission technology of Adobe stratus are used to

The principle of the design methodology is when a computer via a webcam captures the marker and the reality scene, and analyses the image data of the marker by the flartoolkit library to identify the number, location, size, tilt and rotation angle and other information of the virtual object, and then hands over those data to PaperVision library to calculate the corresponding three-dimensional model presentation, and finally merges the reality scene

described below.

**3.1 Marker and object database system** 

for the further AR learning system.

**3.2 Personal AR learning system** 

solve the virtual display problem.

**3.3 Multi-user cooperative AR learning system** 

and virtual objects as a single image. (Figure 3.1 & Figure 3.2)

AR learning system.

While remote collaborative AR allows people in different spaces to using AR technology to share their idea for some specific virtual objects. The teachers, for instance, can use augmented reality in the remote system to guide students learning or solving problems, the students can also in different places at the same time on a specific topic of virtual objects to discuss and share their idea in the use of AR systems (Fig. 2.3).

(Kato and Billinghurst, 1999)

Fig. 2.3 Remote collaborative AR.

AR has been successfully used in many learning environments, especially in cooperative interactive learning is to play its effectiveness (Lievonen et al.,2009; Shen,Ong & Nee,2008; Quy et al.,2009; Nilsson,Johansson & Jonsson, 2009; Godet-Bar,Rieu & Dupuy-Chessa, 2010; Li, 2010; Dierker, Pitsch & Hermann, 2011; Morrison et al.,2011). However, in the applications of AR collaborative environment, the learners are mostly to look from the same virtual view with an AR environment, or remote in video-conference or a network to watch and interact on a same reality scenee. How to really effectively convey the message, including image, text, and related animation, between the learners is still a major issue to enhance the learning effect in cooperation AR environment. Moreover, If both partners are in remote AR environment, the virtual objects, which are produced by the trigger of each maker of their original AR environment, do not really exist. How then to make original content that does not exist, can be delivered to distant learners to share is the key success factor in the cooperative AR environment.

Therefore, based on the concept of reuse learning materials, easy on-line AR application for students, as well as effective learning in remote collaboration AR environment, the OMARIS is developed primarily to provide teachers and students to enhance their teaching and learning effect by using AR technology in educational environment.

## **3. System design of OMARIS**

The OMARIS system is mainly divided into three parts, namely, maker and object database system, personal AR learning system, as well as multi-user cooperative AR learning system. The marker and object database system is to manage markers, objects, and the links between the markerS and objects relationship. The personal AR learning system is to provide instructors to set up learning scenarios, and students to explore individual AR learning activities. The last multi-user cooperative AR learning system is to provide students cooperative learning activities in AR environment. Detailed system design description, as described below.

#### **3.1 Marker and object database system**

230 Augmented Reality – Some Emerging Application Areas

While remote collaborative AR allows people in different spaces to using AR technology to share their idea for some specific virtual objects. The teachers, for instance, can use augmented reality in the remote system to guide students learning or solving problems, the students can also in different places at the same time on a specific topic of virtual objects to

AR has been successfully used in many learning environments, especially in cooperative interactive learning is to play its effectiveness (Lievonen et al.,2009; Shen,Ong & Nee,2008; Quy et al.,2009; Nilsson,Johansson & Jonsson, 2009; Godet-Bar,Rieu & Dupuy-Chessa, 2010; Li, 2010; Dierker, Pitsch & Hermann, 2011; Morrison et al.,2011). However, in the applications of AR collaborative environment, the learners are mostly to look from the same virtual view with an AR environment, or remote in video-conference or a network to watch and interact on a same reality scenee. How to really effectively convey the message, including image, text, and related animation, between the learners is still a major issue to enhance the learning effect in cooperation AR environment. Moreover, If both partners are in remote AR environment, the virtual objects, which are produced by the trigger of each maker of their original AR environment, do not really exist. How then to make original content that does not exist, can be delivered to distant learners to share is the key success

Therefore, based on the concept of reuse learning materials, easy on-line AR application for students, as well as effective learning in remote collaboration AR environment, the OMARIS is developed primarily to provide teachers and students to enhance their teaching and

The OMARIS system is mainly divided into three parts, namely, maker and object database system, personal AR learning system, as well as multi-user cooperative AR learning system. The marker and object database system is to manage markers, objects, and the links between the markerS and objects relationship. The personal AR learning system is to provide

learning effect by using AR technology in educational environment.

discuss and share their idea in the use of AR systems (Fig. 2.3).

(Kato and Billinghurst, 1999)

Fig. 2.3 Remote collaborative AR.

factor in the cooperative AR environment.

**3. System design of OMARIS** 

In general AR system, the link between the marker and the virtual object are fixed, so that the materials of the AR system are difficult to reuse. Thereby, how to manage makers, objects, and re-setup the link relationship between them is the main idea of the system. Based on the idea, the marker and object database system includes three subsystems: marker database system, object database system and marker and object matching system. The first two material database systems basically manage the uploaded materials, including marker symbols and virtual objects by authorized users. The latter one application system is based on the needs of instructional designers or general users, to set up the link between marker symbols and virtual objects in the database systems and to output as a single XML learning script file for future instructional use. After the linking relationship processed by the system, it then will be provided as the necessary materials for the further AR learning system.

#### **3.2 Personal AR learning system**

Since the innovation development of AR technology in education, the AR application makes the learner more motivation and substance of learning, thus providing individualized and flexible AR learning system for the learners is a key factor of the system development. Based on the idea, the personal AR learning system is designed for the instructors and learners. The instructors can select the predefine AR learning scenario, defined from previously material database system, or can customize the current AR learning script file for specific instructional needs in the personal AR learning system. The learner then can choose related topic to explore the combinational effect of the reality scenes and virtual objects in personal AR learning system.

#### **3.3 Multi-user cooperative AR learning system**

This collaborative learning environment is especially to provide multiple users to be able to sharing mixed AR contents from their computer system, including reality scene and virtual objects. As the virtual objects are virtual feature in the client computer system, they can't be display directly through video devices to remote collaborative computers. Therefore, in this study, Flash Actionscript 3.0 with flartoolkit AR library and PaperVision three-dimensional display library, and the point to point transmission technology of Adobe stratus are used to solve the virtual display problem.

The principle of the design methodology is when a computer via a webcam captures the marker and the reality scene, and analyses the image data of the marker by the flartoolkit library to identify the number, location, size, tilt and rotation angle and other information of the virtual object, and then hands over those data to PaperVision library to calculate the corresponding three-dimensional model presentation, and finally merges the reality scene and virtual objects as a single image. (Figure 3.1 & Figure 3.2)

The Design and Implementation of On-Line Multi-User Augmented Reality Integrated System 233

As both of individual computers have synthesized their reality scene and virtual objects image, they then can, through Adobe Status point to point technology, transfer those merged data to each other. (Figure 3.3 & Figure 3.4) Thus, the learners of OMARIS could process their

cooperative learning activities to share ideas with each reality and virtual materials.

Fig. 3.3 The point to point transmission technology diagram.

Fig. 3.4 The reality scene and virtual object encoding processing.

Fig. 3.1 The principle of reality scene and virtual object synthesis.

Fig. 3.2 The logic design of reality scene and virtual object synthetic process.

232 Augmented Reality – Some Emerging Application Areas

Fig. 3.1 The principle of reality scene and virtual object synthesis.

Fig. 3.2 The logic design of reality scene and virtual object synthetic process.

As both of individual computers have synthesized their reality scene and virtual objects image, they then can, through Adobe Status point to point technology, transfer those merged data to each other. (Figure 3.3 & Figure 3.4) Thus, the learners of OMARIS could process their cooperative learning activities to share ideas with each reality and virtual materials.

Fig. 3.3 The point to point transmission technology diagram.

Fig. 3.4 The reality scene and virtual object encoding processing.

The Design and Implementation of On-Line Multi-User Augmented Reality Integrated System 235

During the personal instructional session, the instructor can first enter the OMARIS system to choose the predefine AR learning module, or to edit the additional learning materials for the specific learning purpose, then guides the students to login into the OMARIS system for

When learners enter the collaborative learning mode, each of them can choose same or different topic in their AR learning system. They then can send meesage to Adobe stratus to

Fig. 4.2 Maker / model mapping interface.

**4.2 The personal instructional session** 

AR-based learning activities.(Figure 4.3)

Fig. 4.3 Personal AR learning system.

**4.3 The collaborative learning session** 

## **4. System implementation of the OMARIS**

The OMARIS working processes can be divided into three main parts. These parts are AR instructional database building session, which is the basis of the OMARIS, second the personal instructional session, which is used for the single user AR learning system, and finally the collaborative learning session, which is used for the multiple users AR learning system. The working processes are discussed briefly as follows.

#### **4.1 AR instructional database building session**

First, the system users prepare their instructional materials which include marker pictures and responding learning materials, comprising static data and 2D or 3D animation. They then enter into the OMARIS system to build the AR instructional database. Every authorized user can view the whole items of the database and manage and match their own VR instructional materials for future instructional use as shown in Fig. 4.1 and Fig. 4.2

Fig. 4.1 AR instructional database diagram.

Fig. 4.2 Maker / model mapping interface.

234 Augmented Reality – Some Emerging Application Areas

The OMARIS working processes can be divided into three main parts. These parts are AR instructional database building session, which is the basis of the OMARIS, second the personal instructional session, which is used for the single user AR learning system, and finally the collaborative learning session, which is used for the multiple users AR learning

First, the system users prepare their instructional materials which include marker pictures and responding learning materials, comprising static data and 2D or 3D animation. They then enter into the OMARIS system to build the AR instructional database. Every authorized user can view the whole items of the database and manage and match their own VR instructional materials for future instructional use as shown in Fig. 4.1 and Fig. 4.2

**4. System implementation of the OMARIS** 

**4.1 AR instructional database building session** 

Fig. 4.1 AR instructional database diagram.

system. The working processes are discussed briefly as follows.

## **4.2 The personal instructional session**

During the personal instructional session, the instructor can first enter the OMARIS system to choose the predefine AR learning module, or to edit the additional learning materials for the specific learning purpose, then guides the students to login into the OMARIS system for AR-based learning activities.(Figure 4.3)

Fig. 4.3 Personal AR learning system.

#### **4.3 The collaborative learning session**

When learners enter the collaborative learning mode, each of them can choose same or different topic in their AR learning system. They then can send meesage to Adobe stratus to

The Design and Implementation of On-Line Multi-User Augmented Reality Integrated System 237

real scenes but only a same virtual situation. Thus the OMARIS system tries to offer the instructional material system, personal AR learning system and collaborative AR learning system to be more flexible reuse AR materials, the better ease use of AR application and the more completely conveying and sharing integrated message of each other during cooperative learning. With the OMARIS model, how can the creation of innovative teaching strategies to achieve greater learning effectiveness, will be more in-depth issue of the future

Amaoka,T.; Laga,H.; Yoshie, M. & Nakajima, M.(2011). Personal space-based simulation of

Argelague, F.; Kulik A.; Kunert A.; Andujar, C. and Froehlich, B. See-through techniques for

Beaney, D. & Namee, B. M.(2008). Forked! A Demonstration of Physics Realism in

Bideau, B.; Kulpa, R.; Vignais, Nicolas.; Brault, S.; Multon, F. & Craig, C.(2010). Using

Billinghurst, M. , Belcher, D., Gupta, A., Kiyokawa K. (2003). Communication Behaviors in Colocated Collaborative AR Interfaces. Computer Interaction 16(3), pp.395-423. Botella, C.; Bretón-López, J.; Quero, Soledad.; Baños, Rosa.; García-Palacios, A. (2010).

Cáceres, R. & Ebling, M. (2010). Gaming and Augmented Reality Come to Location- Based

Castle, R.O. & Murray, D.W.(2011). Keyframe-based recognition and localization during

doi:10.1016/j.imavis.2011.05.002. (Manuscript submitted for publication) Castle, R.O.; Klein, G. & Murray, D.W. (2011). Wide-area augmented reality using camera

Chastine, J. W.; Nagel, K.; Zhu, Y & Yearsovich, L. (2007). Understanding the Design Space

Chiang, H.; Chou, Y.; Chang, L.; Huang, H.; Kuo, F. & Chen, H. (2011). An Augmented

Angus Antley and Mel Slater.(2010). The Effect on Lower Spine Muscle Activation

Azuma, R.; Billinghurst, M.& Klinker, G.(2011). Special Section on Mobile

Services. Ieee Pervasive Computing (2010), 9, 1, 5-6.

Graphics Interface 2007 on GI 07, 2007, 207-214.

Human International Conference, 12.

Camera tracking and mapping in multiple regions. Computer Vision and Image

Proof (Manuscript submitted for publication).

Human-Computer Studies, 69, 6, 387-400. Augmented Reality. Computers & Graphics, 35, 4, vii-viii.

Augmented Reality.171-172.

Applications, 30, 2, 14-21.

Understanding, 115, 854–867.

401–413.

non-verbal communications. Entertainment Computing, Article in Press, Corrected

referential awareness in collaborative virtual reality. International Journal of

Augmented Reality. 2009 8th IEEE International Symposium on Mixed and

Virtual Reality to Analyze Sports Performance. IEEE Computer Graphics and

Treating Cockroach Phobia With Augmented Reality. Behavior Therapy, 41 (2010),

video-rate parallel tracking and mapping. Image and Vision Computing,

tracking and mapping in multiple regions. Computer Vision and Image

of Referencing in Collaborative Augmented Reality Environments. Proceedings of

Reality Learning Space for PC DIY. AH '11 Proceedings of the 2nd Augmented

study.

**6. References** 

get their peerID for network connection. Once they take the individual's peerID, they can call each other peerID to synchronize video and audio for the cooperative activities. Through mutual learning and sharing activities, they can spread virtual objects with real-life situation in order to achieve the purpose of cooperative learning.(Figure 4.4)

Fig. 4.4 Collaborative AR learning system.

## **5. Conculsion**

As technology continues to progress and innovation, how to more effectively use in education to enhance students better productive learning is being constantly raised. Because of its creative feature of combining reality scene and virtual objects, AR technology has great attention in the use of education. Such AR based educational use, if the design of the applications considers the flexible material usability, facilitate use of equipment and the characteristics of cooperative learning, the effectiveness of the application will be more improved. The feature of the OMARIS is mainly from the educational view to use software engineering integrating technology components and learning materials to make an innovative combination in educational environment and to increase flexibly and valueadded the AR learning system in the use of school setting. In particular, the system uses multi-user remote integrating technology to on-line combine and transfer each other's real scene and virtual learning objects, which originally do not exist in a client learning environment, to both cooperative learning sides. This is different with past collaborative AR learning systems which their cooperative learning environments are processing in multiuser real scenes but only a same virtual situation. Thus the OMARIS system tries to offer the instructional material system, personal AR learning system and collaborative AR learning system to be more flexible reuse AR materials, the better ease use of AR application and the more completely conveying and sharing integrated message of each other during cooperative learning. With the OMARIS model, how can the creation of innovative teaching strategies to achieve greater learning effectiveness, will be more in-depth issue of the future study.

## **6. References**

236 Augmented Reality – Some Emerging Application Areas

get their peerID for network connection. Once they take the individual's peerID, they can call each other peerID to synchronize video and audio for the cooperative activities. Through mutual learning and sharing activities, they can spread virtual objects with real-life

As technology continues to progress and innovation, how to more effectively use in education to enhance students better productive learning is being constantly raised. Because of its creative feature of combining reality scene and virtual objects, AR technology has great attention in the use of education. Such AR based educational use, if the design of the applications considers the flexible material usability, facilitate use of equipment and the characteristics of cooperative learning, the effectiveness of the application will be more improved. The feature of the OMARIS is mainly from the educational view to use software engineering integrating technology components and learning materials to make an innovative combination in educational environment and to increase flexibly and valueadded the AR learning system in the use of school setting. In particular, the system uses multi-user remote integrating technology to on-line combine and transfer each other's real scene and virtual learning objects, which originally do not exist in a client learning environment, to both cooperative learning sides. This is different with past collaborative AR learning systems which their cooperative learning environments are processing in multiuser

situation in order to achieve the purpose of cooperative learning.(Figure 4.4)

Fig. 4.4 Collaborative AR learning system.

**5. Conculsion** 


The Design and Implementation of On-Line Multi-User Augmented Reality Integrated System 239

Ismail, A. W. & Sunar, M. S.(2009). Collaborative Augmented Reality Approach For Multi-

Jeferson R. Silva, Thiago T. Santos, Carlos H. Morimoto.(2011). Automatic camera control in

Jiang, X.; Broelemann, K.; Wachenfeld, S. & Kruger, A.(2011). Graph-based markerless

Jo, H.; Hwang, S.; Park, H. & Ryu, J-H.(2011). Aroundplot: Focus + context interface for offscreen objects in 3Denvironments. Computers & Graphics, 35 , 841–853. Juan, M. C. & Pérez, D.(2010). Using augmented and virtual reality for the development of

Kansaku, K.; Hata, N. & Takano, K.(2010). My thoughts through a robot's eyes: An augmented reality-brain–machine interface. Neuroscience Research, 66, 219-222. Kato, H. and Billinghurst, M. (1999). Marker tracking and HMD calibration for a videobased augmented reality conferencing system. In IWAR '99, pp. 85-94. Klein, G. and Murray, D. W.(2010). Simulating Low-Cost Cameras for Augmented Reality

Kroeker, K. L.(2010). Mainstreaming Augmented Reality. communications of the acm, 53 ,7,

Langlotz, T.; Degendorfer, C.; Mulloni, A.; Schall, G.; Reitmayr, G.; Schmalstieg,

Leblanc F, Senagore AJ, Ellis CN, Champagne BJ, Augestad KM, Neary PC, Delaney CP &

Lee, J, Han, S, Yang, J. (2011). Construction of a computer-simulated mixed reality

Li, S.(2010). Research on Network Cooperative Learning System Based on Virtual Reality.

Lievonen, M.; Rosenberg, D.; Dörner, R.; Kühn, G. & Walkowski, S.(2009). Augmented

training models. Journal Of Surgical Education (2010), 67, 4, 200-204. Lee, H. S & Lee, J. W.(2008). Mathematical Education Game Based on Augmented Reality.

reality browsing , Computers & Graphics , 35 , 831–840.

Li, S. & Xu, C. (2011). Efficient lookup table based camera pose estimation

Business, e-Management and e-Learning, 43-46.

Janssen, R .(2011).Augmented Reality -The Ethical Importance of a Shared Context.

Computers and Electrical Engineering, 36, 169-179.

Graphics , 35 ,412–421.

Understanding , 115 , 1032-1043.

& Graphics, 34, 6, 756-766.

GRAPHICS,16, 3, 369-380.

98.

DOI: :10.1145/1785414.1785422

450, DOI: 10.1007/978-3-540-69736-7\_48.

user Interaction in Urban Simulation. ICIMT '09 Proceedings of the 2009 International Conference on Information and Multimedia Technology. 19-23. Ivetic, D.; Mihic, S.& Markoski, B.(2010). Augmented AVI video file for road surveying.

virtual environments augmented using multiple sparse videos. Computers &

registration of city maps using geometric hashing. Computer Vision and Image

acrophobic scenarios. Comparison of the levels of presence and anxiety. Computers

Compositing. IEEE TRANSACTIONS ON VISUALIZATION AND COMPUTER

D.(2011).Robust detection and tracking of annotations for outdoor augmented

Colorectal Surgery Training Group.(2010). Hand-assisted laparoscopic sigmoid colectomy skills acquisition: augmented reality simulator versus human cadaver

Lecture Notes in Computer Science, 2008, Volume 5093/2008, 442-

environment for virtual factory layout planning. Computers in Industry , 62 , 86–

IC4E '10 Proceedings of the 2010 International Conference on e-Education, e-

Reality as Means for Creating Shared Understanding. ECCE '09 European


238 Augmented Reality – Some Emerging Application Areas

Deng, C.; Xue, L. & Zhou, Z.(2009). Integration of Web2.0, Panorama Virtual Reality and

Dierker, Angelika; Pitsch, Karola; Hermann, Thomas.(2011). An augmented-reality-based

Doru Talaba, Imre Horváth, Kwan H Lee.(2010).Special issue of Computer-Aided Design on

Dougherty, M. A.; Mann, S. A.; Bronder, M. L.; Bertolami, J. & Craig, R. M.(2010) Patent

Fernández, C.; Baiget, P.; Roca, F. X. & Gonzàlez, J.(2011). Augmenting video surveillance

GAO, X.; HU, H.; JIA, Q-X.; SUN, H-X. & SONG, J-Z.(2011). 3D augmented reality

Gee, A. P.; Webb, M.; Escamilla-Ambrosio, J.; Mayol-Cuevas, W. & Calway, A. (2011). A

Godet-Bar, G.; Rieu, D. & Dupuy-Chessa, S.(2010). HCI and business practices in a

Grasset, R.; Dünser, A. & Billinghurst, M.(2008). Edutainment with a Mixed Reality Book: A

Guan1, T.; Duan, L.; Yu, J.; Chen, Y. and Zhang, X.(2010). Real Time Camera Pose Estimation

Hachet, M and Kruijff, E.(2010). Guest Editor's Introduction: Special Section on the ACM

Hilsenrat, M. & Reiner, M.(2010). Hapto-visual Virtual Reality as a Tool in Psychophysical

Ishiguro, Y. & Rekimoto, J.(2011). Peripheral Vision Annotation: Noninterference

Proceedings of the 2nd Augmented Human International Conference, 8.

Holmes, N.(2010). The Varieties of Reality. *The varieties of reality.* Computer, 43,1,96,94-95. Holz, T.; Campbell, A. G.; O'Hare, G.M.P.; Stafford, J. W.; Martin, A.; Dragone, M.(2011).

ON VISUALIZATION AND COMPUTER GRAPHICS, 16, 1, 2-3.

Technology. 1625-1628.

2010/0257252 A1.

Letters, 32, 878–889.

Technology (2010), 52, 492-505.

Technology (2008).292-295.

Applications, 31, 3, 56-68.

854-868.

268.

Repository, urn:nbn:de:0070-bipr-49260.

Design, 42, 5, 361-363, DOI: 10.1016/j.cad.2010.01.001.

of Posts and Telecommunications, 18 , 1, 105-112.

Geological Information System. ICCIT '09 Proceedings of the 2009 Fourth International Conference on Computer Sciences and Convergence Information

scenario for the collaborative construction of an interactive museum. BiPrints

virtual and augmented reality technologies in product design. Computer-Aided

application title: Augmented Reality Cloud Computing. Microsoft Corporation,

footage with virtual agents for incremental event evaluation. Pattern Recognition

teleoperated robot system based on dual vision. The Journal of China Universities

topometric system for wide area augmented reality. Computers & Graphics, 35,

collaborative method for augmented reality systems. Information and Software

visually augmented illustrative childrens' book. Proceedings of the 2008 International Conference on Advances in Computer Entertainment

for Wide Area Augmented Reality Applications. Computer Graphics and

Symposium on Virtual Reality Software and Technology. IEEE TRANSACTIONS

Research on Roughness Sensitivity. ACHI '10 Proceedings of the 2010 Third International Conference on Advances in Computer-Human Interactions, 139-142.

MiRA—Mixed Reality Agents. J. Journal of Human-Computer Studies, 69 , 251–

Information Presentation Method for Mobile Augmented Reality. AH '11


The Design and Implementation of On-Line Multi-User Augmented Reality Integrated System 241

Rosenblum, L. & Julier, S. (2007). Action- and Workflow-Driven Augmented Reality for

Samset, E.; Schmalstieg, D.; Vander Sloten, J.; Freudenthal, A.; Declerck, J.; Casciaro, S.;

Sayed, N. A.M. El.; Zayed, H. H.; Sharawy, M. I.(2011). ARSC: Augmented reality student

Sayed, N.A.M., El, Zayed, H. H., Sharawy, M.I.(2011). ARSC: Augmented reality student

Shen, Y.; Ong, S. K. & Nee, A. Y. C.(2008). Augmented reality for collaborative product

Shen, Y.; Ong, S. K. & Nee, A. Y. C. (2011).Vision-Based Hand Interaction in Augmented

Shin, D. H. & Dunston, P. S.(2010). Technology development needs for advancing

Silva, R.; Giraldi, G. & Oliveira, Jauvane C.(2003). Introduction to augmented reality.

Taketomi, T.; Sato T. & Yokoya, N. (2011). Real-time and accurate extrinsic camera

Uva, A.E.; Cristiano, S.; Fiorentino, M. & Monno, G.(2010). Distributed design review using tangible augmented technical drawings. Computer-Aided Design, 42, 364-372. Walsh, J. A. & Thomas, B. H.(2011). Visualising Environmental Corrosion in Outdoor

Wang, H. S. (1999). The system design of on-line virtual reality courseware in computer

Wang, H. S. (2000). The study of using virtual reality software in art curriculum, Journal of

Wang, Y.; Shen, Y.; Liu, D.; Wei, S. & Zhu, C.(2010). Key Technique of Assembly System in

Wrzesien, Maja.; Burkhardt,J-M.; Raya,M.A.& Botella, C.(2011). Mixing Psychology and HCI

Yabuki, N.; Miyashita, K. & Fukuda, T.(2011). An invisible height evaluation system for

computing systems, 2119-2124, doi:10.1145/1979742.1979898.

design and development. Design Studies (2010), 31, 2, 118-145.

Technical Report: 25/2003, LNCC, Brazil, 2003.

assembly, Journal of Secondary Education, 6, 355-386

Computers & Graphics, 35, 768–777.

Secondary Education, 7, 173-200

Automation in Construction, 20, 228–235.

(2007), 10-14.

523-544.

South Australia.

35, 810–822

182.

Vision and Electronic Imaging XIII.

Education, 56 ,1045–1061.

Education, 56, 1045–1061.

Computer-Aided Medical Procedures. IEEE Computer Society, September/October

Rideng, O. & Gersak, B.(2008). Augmented Reality in Surgical Procedures. Human

card An augmented reality solution for the education field. Computers &

card An augmented reality solution for the education field. Computers &

Reality Environment. International Journal of Human-Computer Interaction, 27 ,6,

Augmented Reality-based inspection. Automation in Construction, 19 (2010), 169–

parameter estimation using feature landmark database for augmented reality.

Augmented Reality School of Computer and Information Science University of

an Augmented Reality Environment. ICCMS '10 Proceedings of the 2010 Second International Conference on Computer Modeling and Simulation, 1, 133-137. Wither, J.; Tsai, Y-T. &Azuma, R. (2011). Indirect augmented reality. Computers & Graphics,

in Evaluation of Augmented Reality Mental Health Technology. CHI EA '11 Proceedings of the 2011 annual conference extended abstracts on Human factors in

building height regulation to preserve good landscapes using augmented reality.

Conference on Cognitive Ergonomics: Designing beyond the Product --- Understanding Activity and User Experience in Ubiquitous Environments, 17.


240 Augmented Reality – Some Emerging Application Areas

Martin, S.; Diaz, Gabriel; Sancristobal, E.; Gil, R.; Castro, M. & Peire, J.(2011). New

McCall, R.; Wetzel, R.; Lo¨schner, h.; Braun, A-K.(2011). Using presence to evaluate an

Morrison, A.; Mulloni, A.; Lemmelä, S.; Oulasvirta, A.; Jacucci, G.; Peltonen, P.; Schmalstieg,

Nilsson, S.; Johansson, B. & Jonsson, A.(2009). Using AR to support cross-organisational

Ong, S.K. & Wang, Z.B.(2011). Augmented assembly technologies based on 3D bare-hand

Paelke, V. & Sester, Monika.(2010). Augmented paper maps: Exploring the design space of a

Pecchioli, L.; Carrozzino, M.; Mohamed, F.; Bergamasco, M & Kolbe, T. H.(2011).ISEE:

Peter Meier, CTO, metaio GmbH.(2010). Mobile Augmented Reality 2010. metaio

Phan, V. T. & Choo, S. Y.(2009). A Combination of Augmented Reality and Google Earth's

Pintaric ,T. & Kaufmann, H.(2007). Affordable Infrared-Optical Pose-Tracking for Virtual

Polys, N. F.; Bowman, Doug A. & North, Chris.(2011). The role of Depth and Gestalt cues in

Portalés, C.; Lerma , J. L. & Navarro, S.(2010). Augmented reality and photogrammetry: A

Quy, PS.; Lee, J.; Kim, JH.; Kim, JI. & Kim, HS.(2009). Collaborative Experiment and

International Symposium on Mixed and Augmented Reality, 3-12.

of Cultural Heritage, 2011/02/25, doi:10.1016/j.culher.2010.11.001.

Virtual Environments. 2007/03/14 - 2007/03/17 ,44 - 51.

Photogrammetry and Remote Sensing (2010) ,65, 1, 134-142.

Sciences and Convergence Information Technology, 80- 85.

interaction. CIRP Annals - Manufacturing Technology, 60 ,1-4.

GRAPHICS, 16, 3, 353 – 354.

1, 25-35.

256-265.

30–51.

Computers & Education, 57, 1893–1906.

AUGMENTED SOLUTIONS, 01/2010.

Applications, 4, 3, 26-34.

paper maps. Computers & Graphics , 35 , 789–799.

Conference on Cognitive Ergonomics: Designing beyond the Product --- Understanding Activity and User Experience in Ubiquitous Environments, 17. Livingston, M. A.; Azuma, R. T.; Bimber, O. & Saito, H.(2010). Guest Editors' Introduction:

Special Section on The International Symposium on Mixed and Augmented Reality (ISMAR). IEEE TRANSACTIONS ON VISUALIZATION AND COMPUTER

technology trends in education: Seven years of forecasts and convergence.

augmented reality location aware game. Personal and Ubiquitous Computing, 15,

D. & Regenbrecht, H.(2011).Collaborative use of mobile augmented reality with

collaboration in dynamic tasks. ISMAR '09 Proceedings of the 2009 8th IEEE

mixed reality system. ISPRS Journal of Photogrammetry and Remote Sensing, 65,

Information access through the navigation of a 3D interactive environment. Journal

facilities for urban planning in idea stage. International Journal of Computer

and Augmented Reality. IEEE VR Workshop on Trends and Issues in Tracking for

information-rich virtual environments. Journal of Human-Computer Studies, 69 ,

synergy to visualize physical and virtual city environments. ISPRS Journal of

Education based on Networked Virtual Reality. 2009 Fourth International Conference on Computer Sciences and Convergence Information Technology. ICCIT '09 Proceedings of the 2009 Fourth International Conference on Computer


**13** 

J. Alejandro Betancur

*EAFIT University* 

*Colombia* 

**Physical Variable Analysis Involved in Head-Up** 

The command dashboard of an automobile is the instrument where most of the information related to the current state of the vehicle is displayed, and visually it is the way the driver can have access to that information. Nowadays, it is a great risk not only for the driver, but also for the passengers that the driver has to focus his attention off the road to focus on the dashboard information; on situations like this, are when Head-Up Display (HUD) systems

The kind of HUDs here analyzed are the Optical See-through Augmented Reality systems, where the objects of an outer environment are combined with previously structured additional information, and the result is visualized by mean of a translucent display (combiner), which is generally the windshield of an automobile. On this type of system, what we want to accomplish

This chapter focuses on the approach of the functional design requirements that must include an HUD system applied to current automobiles, in which the fast acquisition of the vehicle's available functional information in its panel would imply a positive impact in terms of the vehicle's simplicity to be driven, security, handling information, communication, among others. The analysis of the functional aspects of this technology will be covered by the theoretical analysis and the instrumental implementation, so that we can understand the behavior of optic phenomena that are involved in these systems, in order to be able to determine which parameters are the most critical in the possible construction of an HUD. The appreciations and considerations expressed in this chapter are defined after several years of applied and exploratory research that gathers documental fields and laboratory aspects, using the characteristics from the scientific method as aid, determining

The general objective of this chapter is to evaluate the scientific and engineering phenomena from an HUD, in order to set main characteristics and crucial parameters in terms of physics, thus, determining what considerations are fundamental for the proper functioning of these types of augmented reality systems, and identifying the functional and physical characteristics from the basic optical elements of HUD's visualization systems adapted to automobiles.

This chapter is addressed in four phases: the first phase approaches the analysis of the conceptual information about what is currently structured like HUDs applied to

is an interaction with the user through a series of visual stimuli and artificial scenes.

represent a significant breakthrough in terms of automotive safety.

what is transcendent and what is possible from the facts.

**1. Introduction** 

**Display Systems Applied to Automobiles** 

