Section 1 Mixed Reality

**3**

**1. Introduction**

**Chapter 1**

*I-Jui Lee*

**Abstract**

Using Augmented Reality

Technology to Construct a Wood

Furniture Sampling Platform for

AbstractsThe production and design of wood furniture manufacturing includes

manufacturing furniture parts and their assembly with appropriate finishing operations; the process requires repeated communication and discussions, as well as furniture sampling and trials, which are indispensable. However, in the sampling process, due to the different understandings of the designer and the sample maker in regard to the size of 2D drawings and the modeling of 3D furniture, the sampling results often differ greatly from the designer's original concept; such errors appear mostly in the prototyping of wooden furniture. In this study, we focus on the wooden chair to explore whether augmented reality (AR) can contribute to the comparison between the virtual and physical shapes in the furniture prototyping process. We hope that by employing AR, the gap between the prototype and the finished furniture will be narrowed. By researching actual furniture prototyping with three furniture designers and two sample makers, this study has defined three furniture prototyping methods in the industry. Based on the basic principles, we recruited 38 designers to participate in the comparison experiments employing the above three different furniture prototypes. The results confirmed that applying the AR technology can effectively narrow the gap between judgment and prototype.

**Keywords:** augmented reality, product design and manufacturing, wood furniture sampling, virtual and physical comparison, virtual and physical prototyping

Taiwan's furniture industry has transformed from mass production to smallscale self-owned furniture brands with better design and styling characteristics; in the past, most of the large-scale furniture manufacturers were transferred to Vietnam or the Chinese mainland [1]. As a result of the outward furniture manufacturing and production plants, Taiwanese furniture designers began to produce

Designers and Sample Makers

to Narrow the Gap between

Judgment and Prototype

#### **Chapter 1**

Using Augmented Reality Technology to Construct a Wood Furniture Sampling Platform for Designers and Sample Makers to Narrow the Gap between Judgment and Prototype

*I-Jui Lee*

### **Abstract**

AbstractsThe production and design of wood furniture manufacturing includes manufacturing furniture parts and their assembly with appropriate finishing operations; the process requires repeated communication and discussions, as well as furniture sampling and trials, which are indispensable. However, in the sampling process, due to the different understandings of the designer and the sample maker in regard to the size of 2D drawings and the modeling of 3D furniture, the sampling results often differ greatly from the designer's original concept; such errors appear mostly in the prototyping of wooden furniture. In this study, we focus on the wooden chair to explore whether augmented reality (AR) can contribute to the comparison between the virtual and physical shapes in the furniture prototyping process. We hope that by employing AR, the gap between the prototype and the finished furniture will be narrowed. By researching actual furniture prototyping with three furniture designers and two sample makers, this study has defined three furniture prototyping methods in the industry. Based on the basic principles, we recruited 38 designers to participate in the comparison experiments employing the above three different furniture prototypes. The results confirmed that applying the AR technology can effectively narrow the gap between judgment and prototype.

**Keywords:** augmented reality, product design and manufacturing, wood furniture sampling, virtual and physical comparison, virtual and physical prototyping

#### **1. Introduction**

Taiwan's furniture industry has transformed from mass production to smallscale self-owned furniture brands with better design and styling characteristics; in the past, most of the large-scale furniture manufacturers were transferred to Vietnam or the Chinese mainland [1]. As a result of the outward furniture manufacturing and production plants, Taiwanese furniture designers began to produce

#### **Figure 1.**

*In the sampling process, the designers and sample makers need repeated discussions and corrections to understand the exact proportion and shape of furniture.*

**Figure 2.** *Due to the lack of an effective discussion tool, the sampling needs repeated detail correction.*

"small-volume but diversified" design manufacturing; in cooperation with the furniture factories in the Chinese mainland and Vietnam. This production model has become the main cooperation design mode for Taiwanese brand furniture [2]. In the past, designers and sample makers conducted discussions and trials on furniture sampling based on 2D drawings [3] (see **Figure 1**), which was their main method of communication [4, 5]. Currently, with the mature 3D drawing software, today's furniture manufacturing technology has been greatly improved, and the application of Computer Numerical Control (CNC) has brought more styling changes and possibilities to furniture design and mass production [6–8]; even the most experienced sample makers need to view the 3D furniture simulation to understand the shape and style of the designer's furniture [5].

However, most of these 3D simulations can only present the shape and structure of 3D furniture via 2D paper despite the fact that the furniture has more changes in the curvature of the composite space, especially the curved lines and shapes, which are difficult to present on 2D paper, resulting in the deviation from the 2D blueprint during the real 3D sampling process [9] (**Figure 2**). For example, furniture in the Ming dynasty, such as the Ming-style round-back armchair; when it comes to multiple visual viewpoints to compound one curve, the curvature of the armrest and the backrest cannot be judged in a non-frontal view or a side view [10]. These multi-changing curve shapes and spatial angles present a difficult problem for designers and sample makers in understanding and communication because the maker cannot fully understand the shape and spatial size the designer wants, or achieve the accurate curvature from the 2D surface or the 3D simulation [11]. The only method is visual observation and repeated sampling to create the designed furniture; as a result, there will be significant difference between the initially sampled furniture and the designer's prototype. Meanwhile, the furniture has its own requirements for esthetic quality so a slight difference in curvature will lead to obvious deviation [12]. Moreover, wood furniture, unlike metal steel pipe or plastic injection furniture, cannot be directly formed or extruded by machine [13].

**5**

**Figure 3.**

*judgments.*

**sampling**

*Using Augmented Reality Technology to Construct a Wood Furniture Sampling Platform…*

Instead, it still relies on the sample maker's handwork to perform the preliminary sampling, so the spatial cognitive difference in the three dimensions still cannot be

At present, in the furniture sampling work of the sampling factory, 2D proportional blueprints will be obtained from the designer to print the paper or cut the cardboard [15] (**Figure 3**), and then based on the scale model (ratio 1:1 in the size and appearance) of the physical appearance, the sample makers will do their job according to their rich experience, this process will be corrected after many times of discussion and confirmation [16]. A sample of furniture that is closest to the designer's concept will be supplied to the furniture manufacturer for mass production. However, the sampling of wooden furniture is a 3D manual operation production process by the sample maker; the traditional 2D drawing is converted into the 3D hand-made sampling operation, but the spatial modeling cognitive difference between the 2D drawing and 3D spatial structure is still unavoidable [12, 17, 18]. Any slight deviation will affect the proportion and beauty of the furniture production, so repeated correction on sampling is necessary; discussions and communicate waste a

Now that multinational design and manufacturing procedures have become an inevitable trend in the current furniture sampling production [20], furniture designers are facing the need for cross-country or off-site cooperation to discuss furniture sampling with the sampling factory [21]. However, due to the absence of actual space comparison, discussions on the 2D drawing will be more difficult because the sample maker cannot precisely grasp what the designer wants to present [9], and only oral dictation or repeated styling corrections are used in the furniture sampling [18]. Furniture sampling is a time-consuming, costly process as the designer has to go abroad for discussions with the sample maker, so an efficient communication platform for the designers and the sampler makers is highly demanded [19].

**2.2 Advantages of AR technology when applied to traditional furniture** 

There are considerable advantages in applying AR technology to traditional furniture sampling. Furniture sampling involves the translation from 2D "planar engineering drawing" into 3D "physical objects" [22]. The application of the AR technology to product development can provide designers with styling and

*In furniture sampling, 2D cardboard cutting or various drawings are needed by the sample maker to make* 

*DOI: http://dx.doi.org/10.5772/intechopen.90471*

**2.1 Current furniture sampling methods**

lot of the designer and sample maker's time (**Figure 4**) [19].

overcome [14].

**2. Literature review**

*Using Augmented Reality Technology to Construct a Wood Furniture Sampling Platform… DOI: http://dx.doi.org/10.5772/intechopen.90471*

Instead, it still relies on the sample maker's handwork to perform the preliminary sampling, so the spatial cognitive difference in the three dimensions still cannot be overcome [14].

#### **2. Literature review**

*Mixed Reality and Three-Dimensional Computer Graphics*

*understand the exact proportion and shape of furniture.*

and style of the designer's furniture [5].

"small-volume but diversified" design manufacturing; in cooperation with the furniture factories in the Chinese mainland and Vietnam. This production model has become the main cooperation design mode for Taiwanese brand furniture [2]. In the past, designers and sample makers conducted discussions and trials on furniture sampling based on 2D drawings [3] (see **Figure 1**), which was their main method of communication [4, 5]. Currently, with the mature 3D drawing software, today's furniture manufacturing technology has been greatly improved, and the application of Computer Numerical Control (CNC) has brought more styling changes and possibilities to furniture design and mass production [6–8]; even the most experienced sample makers need to view the 3D furniture simulation to understand the shape

*Due to the lack of an effective discussion tool, the sampling needs repeated detail correction.*

*In the sampling process, the designers and sample makers need repeated discussions and corrections to* 

However, most of these 3D simulations can only present the shape and structure of 3D furniture via 2D paper despite the fact that the furniture has more changes in the curvature of the composite space, especially the curved lines and shapes, which are difficult to present on 2D paper, resulting in the deviation from the 2D blueprint during the real 3D sampling process [9] (**Figure 2**). For example, furniture in the Ming dynasty, such as the Ming-style round-back armchair; when it comes to multiple visual viewpoints to compound one curve, the curvature of the armrest and the backrest cannot be judged in a non-frontal view or a side view [10]. These multi-changing curve shapes and spatial angles present a difficult problem for designers and sample makers in understanding and communication because the maker cannot fully understand the shape and spatial size the designer wants, or achieve the accurate curvature from the 2D surface or the 3D simulation [11]. The only method is visual observation and repeated sampling to create the designed furniture; as a result, there will be significant difference between the initially sampled furniture and the designer's prototype. Meanwhile, the furniture has its own requirements for esthetic quality so a slight difference in curvature will lead to obvious deviation [12]. Moreover, wood furniture, unlike metal steel pipe or plastic injection furniture, cannot be directly formed or extruded by machine [13].

**4**

**Figure 2.**

**Figure 1.**

#### **2.1 Current furniture sampling methods**

At present, in the furniture sampling work of the sampling factory, 2D proportional blueprints will be obtained from the designer to print the paper or cut the cardboard [15] (**Figure 3**), and then based on the scale model (ratio 1:1 in the size and appearance) of the physical appearance, the sample makers will do their job according to their rich experience, this process will be corrected after many times of discussion and confirmation [16]. A sample of furniture that is closest to the designer's concept will be supplied to the furniture manufacturer for mass production. However, the sampling of wooden furniture is a 3D manual operation production process by the sample maker; the traditional 2D drawing is converted into the 3D hand-made sampling operation, but the spatial modeling cognitive difference between the 2D drawing and 3D spatial structure is still unavoidable [12, 17, 18]. Any slight deviation will affect the proportion and beauty of the furniture production, so repeated correction on sampling is necessary; discussions and communicate waste a lot of the designer and sample maker's time (**Figure 4**) [19].

Now that multinational design and manufacturing procedures have become an inevitable trend in the current furniture sampling production [20], furniture designers are facing the need for cross-country or off-site cooperation to discuss furniture sampling with the sampling factory [21]. However, due to the absence of actual space comparison, discussions on the 2D drawing will be more difficult because the sample maker cannot precisely grasp what the designer wants to present [9], and only oral dictation or repeated styling corrections are used in the furniture sampling [18]. Furniture sampling is a time-consuming, costly process as the designer has to go abroad for discussions with the sample maker, so an efficient communication platform for the designers and the sampler makers is highly demanded [19].

#### **2.2 Advantages of AR technology when applied to traditional furniture sampling**

There are considerable advantages in applying AR technology to traditional furniture sampling. Furniture sampling involves the translation from 2D "planar engineering drawing" into 3D "physical objects" [22]. The application of the AR technology to product development can provide designers with styling and

#### **Figure 3.**

*In furniture sampling, 2D cardboard cutting or various drawings are needed by the sample maker to make judgments.*

#### **Figure 4.**

*The slight deviation in the furniture sampling is difficult to express and convey on a 2D drawing. Repeated sampling is time-consuming and labor-intensive (the top of the figure is the product status of the actual sampling of the furniture design company (STIMLIG); below is the 3D simulation to illustrate the details of the correction).*

structural judgments by combining different virtual shapes with existing product models [23]. In the past, relevant research applied AR technology to the spatial layout of furniture [24, 25]. The AR technology can correctly represent the quantitative information and material performance on the shape of furniture [18]. Researchers have even found that AR technology can help designers accurately understand the furniture layout plan in the pre-sales phase [26, 27]. It can effectively save the cost of furniture handling and placement in real space; with AR technology, different furniture items can be quickly replaced to present real-time visual effects corresponding to indoor space [28]. In addition, AR technology can quickly present 3D visual images so the designers are able to get more diverse ideas and spatial discussions [29]; for example, designers can quickly change furniture shapes or components (such as chair legs, chair backs or armrests) [18]. The details of furniture parts can be changed through AR technology to help designers communicate with the sample maker. The relevant literature has confirmed that AR technology can effectively help the furniture maker to interpret the structural state of the furniture, which contributes to the work efficiency and correctness in furniture production, as well as presents the 3D furniture assembly [18, 30]. The animation explains the state of the different furniture components, so that the sample maker can clearly understand the characteristics and key points of the furniture structure [18]. In addition, AR technology can help the maker quickly convert the 2D and 3D drawings and understand the furniture; these visual and spatial advantages can be applied to solve the furniture appearance and structural problems encountered by the sample maker in the process of making furniture [18].

#### **2.3 Application of AR technology in the virtual and physical comparison of the shape**

In a recent study, Fernandes explores the user's judgment on virtual and physical objects in a spatial AR environment, and decides through experiments whether inaccurate judgment will occur between virtual and physical objects. The experimental results indicate that inaccurate judgments occur more when the real object cannot be seen, but if the object can be moved and rotated through the

**7**

**3. Methods**

*Using Augmented Reality Technology to Construct a Wood Furniture Sampling Platform…*

AR operation, the spatial judgment will be more accurate [31]. In addition, some researchers have pointed out that if virtual objects are put into a real environment, people can judge the virtual objects as physical based on past visual experience and realistic 3D images [32], which will help the user understand and innovate in modeling [12, 29, 33, 34]. For example, Ford Motor and Microsoft used the visual features of AR technology to jointly develop a service system for car modeling [35]. By using Microsoft's newly developed head-mounted display, the new product model was combined with the existing developed model [29]. Using this system, the new car design team continuously developed the car's modeling and saved time and cost in development and production, whereas in the traditional car modeling design, several car samples are required [29]. Moreover, when developing XBOX game consoles, Microsoft also applied AR technology to its pre-production test, the internal parts of the game consoles and the circuit board were combined in the console prototype to test whether there were problems such as protruding parts or insufficient internal space [36]. AR also can help engineers and designers discuss and test products together [37, 38] to modify and improve the style and structure of

The purpose of this study is to apply the advantages of AR technology in space and vision to reduce the visual spatial difference between the 2D drawing and the 3D modeling during the sampling process of the furniture, when the AR sampling system can help the sample maker directly compare the semi-finished samples with the spatial virtual furniture to get the correct size and curvature of the furniture. The sample maker can use this system to understand and compare the furniture styling before and after the sampling. Based on the 3D virtual furniture model generated by AR, the sample maker can compare and review whether the hand-made sample meets the accuracy of the shape and size designed by the furniture designer. More importantly, it can fully present the correct proportion and shape of the furniture, so that the maker can see the shape and spatial structure of the designer before production, and accurately determine whether the size and shape of the sample are accurate. This study also focused on the comparison of shapes, carried out three different sampling experiments on the shape and structure, to find whether AR technology contributes to confirming the appearance and proportion of sampling furniture.

This study applied AR technology to furniture sampling (**Figure 5**) to determine whether it could effectively assist furniture designers and sample makers in the comparison and discussion of the modeling. The sample maker used AR technology to verify whether the sample was in line with the designer's design and modeling accuracy. The study carried out three different experiments on furniture sampling with 38 designers who had more than 3 years of furniture design and production experience. The aim was to understand whether AR technology is helpful for furniture sampling and physical comparison. The experimental design and data verification of the virtual

and physical characteristics of the furniture's basic structure was conducted. Based on the experiments and subject tests, researchers can use the "visual interface assistance" and "space virtual and reality comparison" features provided by AR technology to understand how to solve the problems in the actual situation of furniture sampling production. They can also understand what problems will occur in the development of furniture sampling, as well as the advantages and visual

*DOI: http://dx.doi.org/10.5772/intechopen.90471*

the product [39, 40].

**2.4 Purpose of the study**

*Using Augmented Reality Technology to Construct a Wood Furniture Sampling Platform… DOI: http://dx.doi.org/10.5772/intechopen.90471*

AR operation, the spatial judgment will be more accurate [31]. In addition, some researchers have pointed out that if virtual objects are put into a real environment, people can judge the virtual objects as physical based on past visual experience and realistic 3D images [32], which will help the user understand and innovate in modeling [12, 29, 33, 34]. For example, Ford Motor and Microsoft used the visual features of AR technology to jointly develop a service system for car modeling [35]. By using Microsoft's newly developed head-mounted display, the new product model was combined with the existing developed model [29]. Using this system, the new car design team continuously developed the car's modeling and saved time and cost in development and production, whereas in the traditional car modeling design, several car samples are required [29]. Moreover, when developing XBOX game consoles, Microsoft also applied AR technology to its pre-production test, the internal parts of the game consoles and the circuit board were combined in the console prototype to test whether there were problems such as protruding parts or insufficient internal space [36]. AR also can help engineers and designers discuss and test products together [37, 38] to modify and improve the style and structure of the product [39, 40].

#### **2.4 Purpose of the study**

*Mixed Reality and Three-Dimensional Computer Graphics*

structural judgments by combining different virtual shapes with existing product models [23]. In the past, relevant research applied AR technology to the spatial layout of furniture [24, 25]. The AR technology can correctly represent the quantitative information and material performance on the shape of furniture [18]. Researchers have even found that AR technology can help designers accurately understand the furniture layout plan in the pre-sales phase [26, 27]. It can effectively save the cost of furniture handling and placement in real space; with AR technology, different furniture items can be quickly replaced to present real-time visual effects corresponding to indoor space [28]. In addition, AR technology can quickly present 3D visual images so the designers are able to get more diverse ideas and spatial discussions [29]; for example, designers can quickly change furniture shapes or components (such as chair legs, chair backs or armrests) [18]. The details of furniture parts can be changed through AR technology to help designers communicate with the sample maker. The relevant literature has confirmed that AR technology can effectively help the furniture maker to interpret the structural state of the furniture, which contributes to the work efficiency and correctness in furniture production, as well as presents the 3D furniture assembly [18, 30]. The animation explains the state of the different furniture components, so that the sample maker can clearly understand the characteristics and key points of the furniture structure [18]. In addition, AR technology can help the maker quickly convert the 2D and 3D drawings and understand the furniture; these visual and spatial advantages can be applied to solve the furniture appearance and structural problems encountered by

*The slight deviation in the furniture sampling is difficult to express and convey on a 2D drawing. Repeated sampling is time-consuming and labor-intensive (the top of the figure is the product status of the actual sampling of the furniture design company (STIMLIG); below is the 3D simulation to illustrate the details of* 

the sample maker in the process of making furniture [18].

**2.3 Application of AR technology in the virtual and physical comparison** 

In a recent study, Fernandes explores the user's judgment on virtual and physical objects in a spatial AR environment, and decides through experiments whether inaccurate judgment will occur between virtual and physical objects. The experimental results indicate that inaccurate judgments occur more when the real object cannot be seen, but if the object can be moved and rotated through the

**6**

**of the shape**

**Figure 4.**

*the correction).*

The purpose of this study is to apply the advantages of AR technology in space and vision to reduce the visual spatial difference between the 2D drawing and the 3D modeling during the sampling process of the furniture, when the AR sampling system can help the sample maker directly compare the semi-finished samples with the spatial virtual furniture to get the correct size and curvature of the furniture. The sample maker can use this system to understand and compare the furniture styling before and after the sampling. Based on the 3D virtual furniture model generated by AR, the sample maker can compare and review whether the hand-made sample meets the accuracy of the shape and size designed by the furniture designer. More importantly, it can fully present the correct proportion and shape of the furniture, so that the maker can see the shape and spatial structure of the designer before production, and accurately determine whether the size and shape of the sample are accurate. This study also focused on the comparison of shapes, carried out three different sampling experiments on the shape and structure, to find whether AR technology contributes to confirming the appearance and proportion of sampling furniture.

#### **3. Methods**

This study applied AR technology to furniture sampling (**Figure 5**) to determine whether it could effectively assist furniture designers and sample makers in the comparison and discussion of the modeling. The sample maker used AR technology to verify whether the sample was in line with the designer's design and modeling accuracy. The study carried out three different experiments on furniture sampling with 38 designers who had more than 3 years of furniture design and production experience. The aim was to understand whether AR technology is helpful for furniture sampling and physical comparison. The experimental design and data verification of the virtual and physical characteristics of the furniture's basic structure was conducted.

Based on the experiments and subject tests, researchers can use the "visual interface assistance" and "space virtual and reality comparison" features provided by AR technology to understand how to solve the problems in the actual situation of furniture sampling production. They can also understand what problems will occur in the development of furniture sampling, as well as the advantages and visual

#### **Figure 5.**

*AR can contribute to the comparison between the physical and the virtual shapes in the furniture prototyping process.*

characteristics of the visual aid in an AR environment. This technology is also quite simple and convenient in research practice. The system interface of the AR can be presented using a tablet computer (**Figure 5**). Since the carrier is simple and can be used for the furniture sampling process to test the furniture body, this technology will be very helpful for the furniture sampling and design development.

#### **3.1 Participants**

In this study, 38 designers with more than 3 years of experience in furniture design and production were asked to serve as participants. Three different sampling experiments were conducted to compare the shapes of the furniture. When designing test questions, the researcher provided two pictures of chairs with the same shape, but the structural elements of the chair in one picture were a scaled 3D drawing (a part of the chair is fine-tuned, for example, tuning 10% of the back of the chair to simulate the error range of sampling). Then the subjects were asked to determine whether the shape was different. The question group was divided according to the three different sampling methods, respectively, comparing the traditional paper 2D furniture drawing with: (1) Sampling Method I: paper 3D furniture blueprint; (2) Sampling Method II: physical 3D furniture (3) Sampling Method III: physical 3D furniture using AR technology. This study explored the error in the judgment of shape during the sampling process based on the three different comparisons.

**9**

*Using Augmented Reality Technology to Construct a Wood Furniture Sampling Platform…*

Studying the "differences in the comparison of furniture sampling" and taking

In this study, the chair was used for furniture sampling and modeling test. The reason was that the shape change of the chair is more diverse and complicated than that of the table and the cabinet [4]. In addition, the structural units of the chair have a rich modeling appearance and surface changes on each component (such as seat, armrest and seat back). Also, the probability of styling errors in the actual sampling program is higher than with other furniture, so this study mainly focuses on the structural decomposition of the chair to understand its structure. The chair was decomposed into different basic structures according to the related literature classification: (1) seat back, (2) armrest, (3) seat surface, (4) chair foot and (5) chair rail. The researchers then separately carried out the experimental simulation on the furniture sampling through the three different sampling methods previously defined. By 3D modeling software pro-e, the modeling parameters were designed to accurately control the shape change of the chair. AR was then used via 3D printing to make the correct virtual shape on the physical furniture structure to simulate the difference in the furniture shapes, in addition to understanding whether the AR technology could effectively reduce the sampling mistakes by the sample makers in the modeling.

In the present study, the following two questions were used:

Method III: physical 3D furniture using AR technology.

(1) Seat back, (2) armrest, (3) seat surface, (4) chair foot, (5) chair rail.

Which part of the basic structure of furniture is the most difficult to identify?

In comparing the visual display interfaces in the three sampling methods, which

This study focuses on the shape recognition of the chair. In the experiment, the shapes of the basic structure of the chair: (1) seat back, (2) armrest, (3) seat surface, (4) chair foot, and (5) chair rail were compared. Two drawings were provided in the test to compare the pairs, one of which is the correct proportion of the furniture surface, and the other is the comparison of the three-dimensional furniture drawings with the partially adjusted components. A slightly different drawing of furniture allows the subject to judge the simulation error of the furniture sampling. The drawing only emphasizes the fine-tuned part of the furniture structure; when testing the seat back, it was presented with a three-dimensional color model, and the overall appearance of the rest was in a dotted line (**Figure 6**), in order to retain the main appearance of the overall shape of the furniture outside the main test component. As the individual parts of the furniture structure did not conform to the real state, the difficulty of the shape recognition was increased. The size, proportion and perspective of the furniture in the various test drawings were the same to facilitate the subsequent judgment of "which part of the basic structure of furniture is the most difficult to identify" as well as the consistency of the analysis and the experimental reliability and validity. Test questions on the modeling difference were used when analyzing: (1) the difficulty of judging the chair's structural shape, and (2) the proportion of error in sampling, to explore the structural components that led to

is the easiest to distinguish the error on the shape? Respectively comparing the traditional paper 2D furniture drawing with: (1) Sampling Method I: paper 3D furniture blueprint, (2) Sampling Method II: physical 3D furniture, (3) Sampling

**3.4 Evaluating test material: comparing the sampling on the chair's shape**

*DOI: http://dx.doi.org/10.5772/intechopen.90471*

**3.2 Experimental design**

the chair as the test object.

**3.3 Research questions**

*Using Augmented Reality Technology to Construct a Wood Furniture Sampling Platform… DOI: http://dx.doi.org/10.5772/intechopen.90471*

#### **3.2 Experimental design**

*Mixed Reality and Three-Dimensional Computer Graphics*

characteristics of the visual aid in an AR environment. This technology is also quite simple and convenient in research practice. The system interface of the AR can be presented using a tablet computer (**Figure 5**). Since the carrier is simple and can be used for the furniture sampling process to test the furniture body, this technology

*AR can contribute to the comparison between the physical and the virtual shapes in the furniture prototyping* 

In this study, 38 designers with more than 3 years of experience in furniture design and production were asked to serve as participants. Three different sampling experiments were conducted to compare the shapes of the furniture. When designing test questions, the researcher provided two pictures of chairs with the same shape, but the structural elements of the chair in one picture were a scaled 3D drawing (a part of the chair is fine-tuned, for example, tuning 10% of the back of the chair to simulate the error range of sampling). Then the subjects were asked to determine whether the shape was different. The question group was divided according to the three different sampling methods, respectively, comparing the traditional paper 2D furniture drawing with: (1) Sampling Method I: paper 3D furniture blueprint; (2) Sampling Method II: physical 3D furniture (3) Sampling Method III: physical 3D furniture using AR technology. This study explored the error in the judgment of shape during the sampling process based on the three different comparisons.

will be very helpful for the furniture sampling and design development.

**8**

**3.1 Participants**

**Figure 5.**

*process.*

Studying the "differences in the comparison of furniture sampling" and taking the chair as the test object.

In this study, the chair was used for furniture sampling and modeling test. The reason was that the shape change of the chair is more diverse and complicated than that of the table and the cabinet [4]. In addition, the structural units of the chair have a rich modeling appearance and surface changes on each component (such as seat, armrest and seat back). Also, the probability of styling errors in the actual sampling program is higher than with other furniture, so this study mainly focuses on the structural decomposition of the chair to understand its structure. The chair was decomposed into different basic structures according to the related literature classification: (1) seat back, (2) armrest, (3) seat surface, (4) chair foot and (5) chair rail. The researchers then separately carried out the experimental simulation on the furniture sampling through the three different sampling methods previously defined. By 3D modeling software pro-e, the modeling parameters were designed to accurately control the shape change of the chair. AR was then used via 3D printing to make the correct virtual shape on the physical furniture structure to simulate the difference in the furniture shapes, in addition to understanding whether the AR technology could effectively reduce the sampling mistakes by the sample makers in the modeling.

#### **3.3 Research questions**

In the present study, the following two questions were used:

Which part of the basic structure of furniture is the most difficult to identify? (1) Seat back, (2) armrest, (3) seat surface, (4) chair foot, (5) chair rail.

In comparing the visual display interfaces in the three sampling methods, which is the easiest to distinguish the error on the shape? Respectively comparing the traditional paper 2D furniture drawing with: (1) Sampling Method I: paper 3D furniture blueprint, (2) Sampling Method II: physical 3D furniture, (3) Sampling Method III: physical 3D furniture using AR technology.

#### **3.4 Evaluating test material: comparing the sampling on the chair's shape**

This study focuses on the shape recognition of the chair. In the experiment, the shapes of the basic structure of the chair: (1) seat back, (2) armrest, (3) seat surface, (4) chair foot, and (5) chair rail were compared. Two drawings were provided in the test to compare the pairs, one of which is the correct proportion of the furniture surface, and the other is the comparison of the three-dimensional furniture drawings with the partially adjusted components. A slightly different drawing of furniture allows the subject to judge the simulation error of the furniture sampling. The drawing only emphasizes the fine-tuned part of the furniture structure; when testing the seat back, it was presented with a three-dimensional color model, and the overall appearance of the rest was in a dotted line (**Figure 6**), in order to retain the main appearance of the overall shape of the furniture outside the main test component. As the individual parts of the furniture structure did not conform to the real state, the difficulty of the shape recognition was increased. The size, proportion and perspective of the furniture in the various test drawings were the same to facilitate the subsequent judgment of "which part of the basic structure of furniture is the most difficult to identify" as well as the consistency of the analysis and the experimental reliability and validity. Test questions on the modeling difference were used when analyzing: (1) the difficulty of judging the chair's structural shape, and (2) the proportion of error in sampling, to explore the structural components that led to

#### **Figure 6.**

*The overall and parts of the chair structure to be tested are presented in a dotted line (for example, the figure is to test the chair seat).*

sampling errors in the chair seat. In addition, the researchers also explored whether the visual conversion from 2D to 3D caused the difference when determining the shape, and how to apply AR technology to the design and sampling of furniture.

#### **3.5 Measurement materials**

#### *3.5.1 Five basic structures of furniture chairs*

This study used the design chair of the STIMLIG Furniture Company (https:// www.stimlig.com/) that was entering the sampling stage to design the test questions. A total of 10 gradients are created according to the five basic structures of the chair, and a new shape was created for each 10% of the chair with microadjustment, while the rest remained unchanged. There were five basic structures of furniture chairs: (1) seat back, (2) armrest, (3) seat surface, (4) chair foot, and (5) chair rail. They respectively constructed 10 gradient units and a single drawing was randomly taken to test the subject. Only one model was supplied for comparison when sampling. There were 50 test questions in total.

#### *3.5.2 Test method for sampling*

The five basic structures in the furniture chair were tested by the following three different sampling methods to analyze the differences in the shape judgment and the problems that might occur with different furniture components in sampling, as well as the potential possibilities of their technical application. The scale of the difference in shape represents the rate of the sampling error. For example, when the subject could tell the fine-tuning was more than 30% in shape, it meant that the sampling method had only 30% error rate in the shape deviation because the difference below 30% could not be distinguished, so the visual difference and limitation between the 2D drawing and the 3D shape could be determined. After the test was

**11**

**Table 1.**

*Three furniture sampling methods.*

*Using Augmented Reality Technology to Construct a Wood Furniture Sampling Platform…*

over, the researcher could determine the proportion of the subjects' shape judgment for different sampling comparison methods, as well as the benefit of applying AR to

Sampling Method I: comparing the traditional paper 2D furniture drawing with

At this stage, the researcher would understand whether shape deviation would occur when comparing the traditional paper 2D furniture drawing with paper 3D furniture blueprint (printed on the 2D paper); the purpose was to find out whether difference between the 2D and the 3D would occur during the sampling process. With the paper-based method, the subjects were asked to judge the shape difference between the left and right objects printed on paper, and the testing method and data were used to analyze which basic structure of the chair was the most difficult

Sampling Method II: comparing the traditional paper 2D furniture drawing with

At this stage, when comparing the traditional paper 2D furniture drawing with physical 3D furniture (2D vs. 3D), through 3D printing, the accurate physical furniture shape (10% gradual difference each time) was completed and compared with the 2D paper drawing to determine the shape difference between the traditional paper 2D furniture drawing and the physical 3D furniture. This method is closest to the actual sampling state and consistent to the conclusion of a real furniture sampling factory when determining the physical 3D furniture based on the 2D drawing. Sampling Method III: comparing the traditional paper 2D furniture drawing

In this stage, the researchers amplified the virtual objects based on AR technology applied to the physical 3D furniture printed, so that the subject could determine whether AR contributed to the comparison of the furniture shape. Based on the different ratios between the virtual and the real, the proper ratio of the AR furniture sampling comparison system (10% each time) that needed to be adjusted could be found. Therefore, the subjects could quickly and correctly distinguish the obvious differences in the shape, clarify the virtual and real interface design and make a

Based on Unity3D [41], this study developed the AR Furniture sampling system that can be installed on an Android tablet computer. Via tablet computer or other display devices, designers can put their designed 3D furniture model into the system, and make comparisons and discussions through the replacement of different furniture components. In the system, through the back-end server and remote synchronization, the researchers updated the version of the sampling program on

**Sampling method type Original proofing material Comparison proofing material**

Paper 3D furniture blueprint

printing)

technology

Physical 3D furniture (created by 3D

Physical 3D furniture using AR

*DOI: http://dx.doi.org/10.5772/intechopen.90471*

the actual sampling comparison (**Table 1**).

to identify or the most likely to cause judgment error.

with physical 3D furniture using AR technology.

**3.6 Constructing the AR furniture sampling system**

1.Sampling Method I Traditional paper 2D furniture drawing

2.Sampling Method II Traditional paper 2D furniture drawing

3.Sampling Method III Traditional paper 2D furniture drawing

the paper 3D furniture blueprint.

physical 3D furniture.

detailed discussion.

*Using Augmented Reality Technology to Construct a Wood Furniture Sampling Platform… DOI: http://dx.doi.org/10.5772/intechopen.90471*

over, the researcher could determine the proportion of the subjects' shape judgment for different sampling comparison methods, as well as the benefit of applying AR to the actual sampling comparison (**Table 1**).

Sampling Method I: comparing the traditional paper 2D furniture drawing with the paper 3D furniture blueprint.

At this stage, the researcher would understand whether shape deviation would occur when comparing the traditional paper 2D furniture drawing with paper 3D furniture blueprint (printed on the 2D paper); the purpose was to find out whether difference between the 2D and the 3D would occur during the sampling process. With the paper-based method, the subjects were asked to judge the shape difference between the left and right objects printed on paper, and the testing method and data were used to analyze which basic structure of the chair was the most difficult to identify or the most likely to cause judgment error.

Sampling Method II: comparing the traditional paper 2D furniture drawing with physical 3D furniture.

At this stage, when comparing the traditional paper 2D furniture drawing with physical 3D furniture (2D vs. 3D), through 3D printing, the accurate physical furniture shape (10% gradual difference each time) was completed and compared with the 2D paper drawing to determine the shape difference between the traditional paper 2D furniture drawing and the physical 3D furniture. This method is closest to the actual sampling state and consistent to the conclusion of a real furniture sampling factory when determining the physical 3D furniture based on the 2D drawing.

Sampling Method III: comparing the traditional paper 2D furniture drawing with physical 3D furniture using AR technology.

In this stage, the researchers amplified the virtual objects based on AR technology applied to the physical 3D furniture printed, so that the subject could determine whether AR contributed to the comparison of the furniture shape. Based on the different ratios between the virtual and the real, the proper ratio of the AR furniture sampling comparison system (10% each time) that needed to be adjusted could be found. Therefore, the subjects could quickly and correctly distinguish the obvious differences in the shape, clarify the virtual and real interface design and make a detailed discussion.

#### **3.6 Constructing the AR furniture sampling system**

Based on Unity3D [41], this study developed the AR Furniture sampling system that can be installed on an Android tablet computer. Via tablet computer or other display devices, designers can put their designed 3D furniture model into the system, and make comparisons and discussions through the replacement of different furniture components. In the system, through the back-end server and remote synchronization, the researchers updated the version of the sampling program on


**Table 1.** *Three furniture sampling methods.*

*Mixed Reality and Three-Dimensional Computer Graphics*

**3.5 Measurement materials**

**Figure 6.**

*to test the chair seat).*

*3.5.2 Test method for sampling*

*3.5.1 Five basic structures of furniture chairs*

when sampling. There were 50 test questions in total.

sampling errors in the chair seat. In addition, the researchers also explored whether the visual conversion from 2D to 3D caused the difference when determining the shape, and how to apply AR technology to the design and sampling of furniture.

*The overall and parts of the chair structure to be tested are presented in a dotted line (for example, the figure is* 

This study used the design chair of the STIMLIG Furniture Company (https:// www.stimlig.com/) that was entering the sampling stage to design the test questions. A total of 10 gradients are created according to the five basic structures of the chair, and a new shape was created for each 10% of the chair with microadjustment, while the rest remained unchanged. There were five basic structures of furniture chairs: (1) seat back, (2) armrest, (3) seat surface, (4) chair foot, and (5) chair rail. They respectively constructed 10 gradient units and a single drawing was randomly taken to test the subject. Only one model was supplied for comparison

The five basic structures in the furniture chair were tested by the following three different sampling methods to analyze the differences in the shape judgment and the problems that might occur with different furniture components in sampling, as well as the potential possibilities of their technical application. The scale of the difference in shape represents the rate of the sampling error. For example, when the subject could tell the fine-tuning was more than 30% in shape, it meant that the sampling method had only 30% error rate in the shape deviation because the difference below 30% could not be distinguished, so the visual difference and limitation between the 2D drawing and the 3D shape could be determined. After the test was

**10**

different tablets. When the designer updates the designed 3D furniture model, the remote sample maker can also simultaneously update the models and functions in the app. The "designer" and "sample maker" can simulate the functionality that this system should be equipped with in different settings.

#### **3.7 Setting**

The AR Furniture Sampling System was set up in the laboratory space (about 3 × 5 m<sup>2</sup> ), and a 65-inch large display screen was placed in front of the experimental field to present the furniture in 1:1 ratio for the test image of furniture sampling. By the video camera and image recognition technology, the state of the furniture in the AR could be presented. The subject sat in front of the screen with the table and the chair placed in front of the screen; the subject could make the comparison on the furniture identification card on the table with the 3D printing model. After capturing the card, the camera projected the virtual 3D furniture shape onto the 3D printed furniture model for the subject to compare the furniture. Thus, the subject could simulate the furniture-sampling environment; the researchers gave them different sampling tasks in sequence to perform the three different sampling methods to determine the five basic structural units of the chair (**Figure 7**).

**13**

*Using Augmented Reality Technology to Construct a Wood Furniture Sampling Platform…*

The results of the experiments have undergone comprehensive recording and subsequent data analysis. The experimental data were used to find out whether the three different sampling methods would cause judgment errors. By the rigorous styling gradient and the small-scale styling ratio change, the researchers controlled the relationships among the three different sampling methods (independent variable) and the judgment errors in 10 modeling scales (dependent variable). In this study, the researchers used different proportions of modeling changes to represent the degree of judgment errors in the sampling; by simulating the deviation of the sampling bodies, the shape judgments of different shapes made visually between 2D and 3D were quantified. In this way, the operation was relatively objective and the experimental limitations on actual sampling were

The experimental results are shown in **Table 2**. The data show different amplitude changes for the overall shape structure of the chair by the three different sampling methods, indicating that they have an influence on the modeling judg-

The shapes of the chair reflected in the three sampling methods differ. When the proportion of the difference in the shape that can be recognized is lower (the difference in the shape is more subtle), the more helpful the sampling method is, as it signifies that the differences of the furniture shape are subtle and the subject can immediately distinguish the difference, and in the sampling process, it is easier to

In the Sampling Method I (**Table 2**), the chair foot (71%) and the chair rail (65%) have the highest recognition level. When the modeling difference reaches an average of 68% or more, the shape difference can be correctly found; the armrest is the second. Only when the modeling difference reaches an average of 75%, can the difference in the shape change be evident between the 2D drawing and the 3D shape. The most difficult to recognize are the seat back (78%) and the seat surface (82%), which require an average of 80% or more for the subject to feel the

In the Sampling Method II (**Table 2**), the order of recognition level has changed somewhat. The chair foot (58%) and the chair rail (53%) have the highest recognition level on the modeling difference. When it reaches an average of 55.5%, the difference can be noticed. The armrest (62%) and seat surface (68%) are second; the difference must reach an average of 65% before the obvious difference in the shape change can be felt. The most difficult to identify is the seat back, which requires an

**4.1 Judgmental differences of the basic chair structure among the three** 

find the sampling difference between the 2D drawing and the 3D shape.

*DOI: http://dx.doi.org/10.5772/intechopen.90471*

ment under different visual media displays.

**sampling methods**

*4.1.1 Sampling Method I*

*4.1.2 Sampling Method II*

average of 72% for the subjects to tell the difference.

difference.

**3.8 Data collection and analysis**

simplified.

**4. Results**

**Figure 7.** *The AR furniture sampling system setup.*

*Using Augmented Reality Technology to Construct a Wood Furniture Sampling Platform… DOI: http://dx.doi.org/10.5772/intechopen.90471*

#### **3.8 Data collection and analysis**

The results of the experiments have undergone comprehensive recording and subsequent data analysis. The experimental data were used to find out whether the three different sampling methods would cause judgment errors. By the rigorous styling gradient and the small-scale styling ratio change, the researchers controlled the relationships among the three different sampling methods (independent variable) and the judgment errors in 10 modeling scales (dependent variable). In this study, the researchers used different proportions of modeling changes to represent the degree of judgment errors in the sampling; by simulating the deviation of the sampling bodies, the shape judgments of different shapes made visually between 2D and 3D were quantified. In this way, the operation was relatively objective and the experimental limitations on actual sampling were simplified.

#### **4. Results**

*Mixed Reality and Three-Dimensional Computer Graphics*

system should be equipped with in different settings.

**3.7 Setting**

3 × 5 m<sup>2</sup>

chair (**Figure 7**).

different tablets. When the designer updates the designed 3D furniture model, the remote sample maker can also simultaneously update the models and functions in the app. The "designer" and "sample maker" can simulate the functionality that this

The AR Furniture Sampling System was set up in the laboratory space (about

), and a 65-inch large display screen was placed in front of the experimental field to present the furniture in 1:1 ratio for the test image of furniture sampling. By the video camera and image recognition technology, the state of the furniture in the AR could be presented. The subject sat in front of the screen with the table and the chair placed in front of the screen; the subject could make the comparison on the furniture identification card on the table with the 3D printing model. After capturing the card, the camera projected the virtual 3D furniture shape onto the 3D printed furniture model for the subject to compare the furniture. Thus, the subject could simulate the furniture-sampling environment; the researchers gave them different sampling tasks in sequence to perform the three different sampling methods to determine the five basic structural units of the

**12**

**Figure 7.**

*The AR furniture sampling system setup.*

The experimental results are shown in **Table 2**. The data show different amplitude changes for the overall shape structure of the chair by the three different sampling methods, indicating that they have an influence on the modeling judgment under different visual media displays.

#### **4.1 Judgmental differences of the basic chair structure among the three sampling methods**

The shapes of the chair reflected in the three sampling methods differ. When the proportion of the difference in the shape that can be recognized is lower (the difference in the shape is more subtle), the more helpful the sampling method is, as it signifies that the differences of the furniture shape are subtle and the subject can immediately distinguish the difference, and in the sampling process, it is easier to find the sampling difference between the 2D drawing and the 3D shape.

#### *4.1.1 Sampling Method I*

In the Sampling Method I (**Table 2**), the chair foot (71%) and the chair rail (65%) have the highest recognition level. When the modeling difference reaches an average of 68% or more, the shape difference can be correctly found; the armrest is the second. Only when the modeling difference reaches an average of 75%, can the difference in the shape change be evident between the 2D drawing and the 3D shape. The most difficult to recognize are the seat back (78%) and the seat surface (82%), which require an average of 80% or more for the subject to feel the difference.

#### *4.1.2 Sampling Method II*

In the Sampling Method II (**Table 2**), the order of recognition level has changed somewhat. The chair foot (58%) and the chair rail (53%) have the highest recognition level on the modeling difference. When it reaches an average of 55.5%, the difference can be noticed. The armrest (62%) and seat surface (68%) are second; the difference must reach an average of 65% before the obvious difference in the shape change can be felt. The most difficult to identify is the seat back, which requires an average of 72% for the subjects to tell the difference.

#### **Table 2.**

*Summarized differences in modeling identification of basic structure of furniture.*

#### **Figure 8.**

*In the Sampling Method III when AR technology is applied, significant differences (p < 0.05) lower than the others methods (Sampling Method I and II) in identifying the shape can be found.*

#### *4.1.3 Sampling Method III*

In the Sampling Method III (**Table 2**), in the overall shape structure of the chair, the difference between the basic structures of the chair is greatly reduced. The difference can be obviously felt when it is 28% in the seat back, 26% in the armrest, 32% in the seat surface, 14% in the chair foot and 18% in the chair rail. Generally, when the different structural parts reach an average of 23.6%, the difference can be felt, which is obviously more recognizable than Sampling Method I and Sampling Method II (**Figure 8**).

In the analysis, we conducted the paired-sample *t* test in SPSS 19.0 to compare the different sampling methods in terms of overall mean identifying rate on the comparing test. In the overall identification, it can be found that in the Sampling Method III when the AR technology is applied, significant differences (p < .05), lower than the other methods (Sampling Method I and II) in the identifying

**15**

*Using Augmented Reality Technology to Construct a Wood Furniture Sampling Platform…*

**5.1 Differences in modeling identification of basic structure of furniture**

**5.2 Differences in modeling recognition among three sampling methods**

When comparing the sampling of the shape, (1) the Sample Method I is consistent with the traditional paper, and it is quite difficult for the sample maker to distinguish the difference in the shape because on the 2D paper drawing it is hard to tell the shape change in the 3D space. Also it is impossible to touch the physical furniture or rotate different perspectives to compare the shape. To create a furniture entity in 3D space, the sample maker needs continuous physical speculation and spatial structure judgment based on the mental rotation. Such a sampling process is a necessary step in the initial furniture sampling process, which is quite difficult

And (2) in the Sample Method II, it is easier to compare the shape between the 2D and the 3D because the sample makers can change the visual perspective by manually rotating the physical furniture model. The visual aid of real shadows and the sensory feedback of physical senses add more sensory information than the 2D paper drawing does, but entails more production cost. Such sampling method is close to the actual sampling in real life and both need a scaled model to determine the difference in styling. Although using 3D printing technology in this experiment can save a lot of time, there will still be increased production cost and time in the actual furniture sampling process. It is helpful for the judgment, but multiple

AR's application in the Sampling Method III (3) is quite easy to implement and practice in the real world compared to the previous two sampling methods (**Table 2**) because the virtual body is in the physical comparison area and uses the visual image presented in the 3D space to enhance the difference between the

The experiment results confirmed that in the furniture Sampling Method III, the difference can be obviously felt. Especially, it is relatively easy to judge based on the basic rectangular surface, such as the chair foot and the rail because they are rectangular or cylindrical or their shape angle is vertical to the datum plane, so the subjects can have a better understanding of the difference in the shape, and are also more likely to determine and compare the modeling difference based on the symmetry and the relative position and regular angle in the space. It also visually reflects that compared with the absolute judgment, the relative judgment of the shape is easier when determining the space because the relative judgment has a visual reference, and the absolute judgment can only rely on human experience and modeling ability. The application of the AR has provided accurate visual reference for shape, and therefore increased the subjects' recognition and mastery rate of the

the shape can be found (**Figure 8**) because the 3D visual aid can quickly show the difference in the shape. In the Sampling Method II, the physical shape comparison and tactile sensation assistance can also be helpful for determining the modeling difference. However, the Sampling Method I focuses on the visual comparison of 2D drawing, which lacks spatially rotated image information and is also difficult for the subject to clarify the differences in some fuzzy modeling

*DOI: http://dx.doi.org/10.5772/intechopen.90471*

areas.

**5. Discussion**

shape difference.

and time consuming for the sample maker.

sampling models are necessary for the comparison.

*Using Augmented Reality Technology to Construct a Wood Furniture Sampling Platform… DOI: http://dx.doi.org/10.5772/intechopen.90471*

the shape can be found (**Figure 8**) because the 3D visual aid can quickly show the difference in the shape. In the Sampling Method II, the physical shape comparison and tactile sensation assistance can also be helpful for determining the modeling difference. However, the Sampling Method I focuses on the visual comparison of 2D drawing, which lacks spatially rotated image information and is also difficult for the subject to clarify the differences in some fuzzy modeling areas.

#### **5. Discussion**

*Mixed Reality and Three-Dimensional Computer Graphics*

**14**

*4.1.3 Sampling Method III*

**Figure 8.**

**Table 2.**

Method II (**Figure 8**).

In the Sampling Method III (**Table 2**), in the overall shape structure of the chair,

In the analysis, we conducted the paired-sample *t* test in SPSS 19.0 to compare the different sampling methods in terms of overall mean identifying rate on the comparing test. In the overall identification, it can be found that in the Sampling Method III when the AR technology is applied, significant differences (p < .05), lower than the other methods (Sampling Method I and II) in the identifying

the difference between the basic structures of the chair is greatly reduced. The difference can be obviously felt when it is 28% in the seat back, 26% in the armrest, 32% in the seat surface, 14% in the chair foot and 18% in the chair rail. Generally, when the different structural parts reach an average of 23.6%, the difference can be felt, which is obviously more recognizable than Sampling Method I and Sampling

*In the Sampling Method III when AR technology is applied, significant differences (p < 0.05) lower than the* 

*others methods (Sampling Method I and II) in identifying the shape can be found.*

*Summarized differences in modeling identification of basic structure of furniture.*

#### **5.1 Differences in modeling identification of basic structure of furniture**

The experiment results confirmed that in the furniture Sampling Method III, the difference can be obviously felt. Especially, it is relatively easy to judge based on the basic rectangular surface, such as the chair foot and the rail because they are rectangular or cylindrical or their shape angle is vertical to the datum plane, so the subjects can have a better understanding of the difference in the shape, and are also more likely to determine and compare the modeling difference based on the symmetry and the relative position and regular angle in the space. It also visually reflects that compared with the absolute judgment, the relative judgment of the shape is easier when determining the space because the relative judgment has a visual reference, and the absolute judgment can only rely on human experience and modeling ability. The application of the AR has provided accurate visual reference for shape, and therefore increased the subjects' recognition and mastery rate of the shape difference.

#### **5.2 Differences in modeling recognition among three sampling methods**

When comparing the sampling of the shape, (1) the Sample Method I is consistent with the traditional paper, and it is quite difficult for the sample maker to distinguish the difference in the shape because on the 2D paper drawing it is hard to tell the shape change in the 3D space. Also it is impossible to touch the physical furniture or rotate different perspectives to compare the shape. To create a furniture entity in 3D space, the sample maker needs continuous physical speculation and spatial structure judgment based on the mental rotation. Such a sampling process is a necessary step in the initial furniture sampling process, which is quite difficult and time consuming for the sample maker.

And (2) in the Sample Method II, it is easier to compare the shape between the 2D and the 3D because the sample makers can change the visual perspective by manually rotating the physical furniture model. The visual aid of real shadows and the sensory feedback of physical senses add more sensory information than the 2D paper drawing does, but entails more production cost. Such sampling method is close to the actual sampling in real life and both need a scaled model to determine the difference in styling. Although using 3D printing technology in this experiment can save a lot of time, there will still be increased production cost and time in the actual furniture sampling process. It is helpful for the judgment, but multiple sampling models are necessary for the comparison.

AR's application in the Sampling Method III (3) is quite easy to implement and practice in the real world compared to the previous two sampling methods (**Table 2**) because the virtual body is in the physical comparison area and uses the visual image presented in the 3D space to enhance the difference between the physical and the virtual entities, so the sampler maker can quickly generate visual clues under the slight shape difference and reduce the difficulty in judgment. Meanwhile, there is no need to make a large number of furniture sampling entities in future sampling. The shape of furniture can be presented by the virtual shape, which can greatly reduce the production cost and time.

#### **5.3 Research limitations**

There are several research limitations in this study. First, we simulated the form and state of sampling through innovative experimental methods and AR system design, aiming to quantify the objective data on furniture sampling on the shape, but because the size of the 3D printing is much smaller than the actual sampling size, the difference between the visual judgment and the spatial shape was affected. However, this study emphasizes the combination of visual space theories with the practice of technology and materials in application. The researchers are convinced that this does not affect the reliability and validity of the results because we used the same sampling method and provided the subjects with the correct ratio of the production process and the projection on the screen to make styling judgments. It turns out one-to-one proportional sampling experiments are feasible for future sampling design and technical breakthroughs. Second, while the subjects are professionally trained furniture design and production staff, they are still not real furniture sample makers. The sampling experience and handwriting skills may affect the ability to make judgments. However, the real sampler makers in the furniture factory are quite busy, and it is not easy for them to be involved in an experiment for a long time. That is the reason why we were looking for professional designers with similar work experience and styling ability as the subjects. Since this study emphasizes the shape comparison between the 2D and 3D under visual space, these subjects have the same ability as the sampler makers to determine the shape and are quite familiar with the design and sampling of furniture. We believe this can make up for the lack of real sample makers, but in future experiments, if funding and time permit, we still hope to conduct experiments with real sampler makers. Third, there is quite a variety of furniture and we only tested a chair, whose shape will affect the results of the sampling comparison data. Therefore, our main research results focus on the comparison of the sampling methods to understand whether AR technology is helpful for the sampling of furniture, but cannot fully explain the deviation of the basic structure of the furniture that is most likely to occur in the furniture sampling process. However, the structure of the furniture can still correspond to its basic shape, which is also an experiment to be carried out in future research because the basic shape (such as column, square, ellipsoid, cylinder, spindle and rectangular) is different in visual judgment; for instance, due to the lack of the right-angled structure, the ellipse is more difficult to identify and distinguish the difference in shape than the rectangular. Therefore, in the future research, this study will simplify the understanding of the model and increase the validity of the experiment through multiple visual viewpoints and modeling. However, in order to avoid focal vagueness in this study, we only focus on the basic structure of furniture rather than a unit body. Fourth, in the study, judgment on the proportion of furniture is used to determine the sampling mistakes, but there will inevitably be subtle differences which cannot be completely subdivided by the overall proportion. For example, the scaling extent at the edge of the chair back may be larger than that in the middle, but this study is only based on the overall chair back, which will cause some slight difference in the shape data. In the future, we must seek more objective software

**17**

furniture modeling.

*Using Augmented Reality Technology to Construct a Wood Furniture Sampling Platform…*

analysis to get a more precise design, but in general, this is an innovative and

AR technology has been applied to many innovative product designs, and is expected to evolve with the future development in the integration of virtual and real environments. The novel devices include Microsoft HoloLens, HTC VIVE Pro, and VR Oculus Rift. The development and emergence of these technological devices reflect the potential of using AR or VR to solve industrial production problems in the future. Moreover, these devices will blur the boundaries between virtual and real integration, and increase the number of gestures and innovative interfaces, which could make the manufacturing process more intuitive, and make the manual operation more flexible. Future users will have more options in operating the AR

In addition to the consumer side in the design process, AR is used in different product development and modeling discussion. Previous researches have pointed out that AR is helpful for the designer's continuous design and redesign in product development [12, 29, 34, 35]. However, in the literature exploration, the researchers found that in the current stage of AR in the furniture sampling and manufacturing production, we have not seen any application of similar concepts. Therefore, we believe that furniture sampling and manufacturing is an excellent research field with great design possibility, especially because the furniture sampling involves the design and processing of furniture production. For example, in furniture product development and styling continuation design, there are many details to consider; at present, it is still difficult for wooden furniture production to overcome the physical shape difference and master the space scale of furniture sampling because wooden furniture has the following characteristics: first, the sampling of its initial prototype relies on the subjective judgment and the handcraft of the special sample maker, thus resulting in a significant difference between the sample of the furniture prototype and the blueprint. Second, wooden furniture contains many complex structures such as composite curved surfaces, curved wood shapes, and gradient bends. It is difficult for 2D drawings to present the spatial structure and modeling concept of 3D furniture, so the spatial cognitive misunderstanding may lead to inaccurate judgments of the sampling appearance. Third, wooden furniture often uses the mortising technique to assemble and connect furniture parts, and the mortising structure is often hidden within the furniture and cannot be directly seen. Due to the above special factors, the sampling of the furniture often requires repeated discussions and trials. In this context, this study is aimed at the research and knowledge construction of furniture sampling in the framework of AR technology in the comparison between physical and virtual reality. Using the systematic experimental method has defined the knowledge structure and specific operational application strategies of AR technology in the comparison of furniture shape sampling, as well as the "difference between modeling" and "virtual reality" to study the topic of furniture styling. The results show that the application of AR to furniture sampling has obvious benefit for the mastery and judgment of the

This research has confirmed that AR technology is more helpful in the sampling

and development of furniture, especially in the visual and styling aspects. This study has developed a set of rigorous sampling methods for AR technology based on the "physical comparison" and the "furniture sampling development", and

*DOI: http://dx.doi.org/10.5772/intechopen.90471*

simple experimental method.

**6. Conclusions**

interface of tablets.

analysis to get a more precise design, but in general, this is an innovative and simple experimental method.

#### **6. Conclusions**

*Mixed Reality and Three-Dimensional Computer Graphics*

which can greatly reduce the production cost and time.

**5.3 Research limitations**

physical and the virtual entities, so the sampler maker can quickly generate visual clues under the slight shape difference and reduce the difficulty in judgment. Meanwhile, there is no need to make a large number of furniture sampling entities in future sampling. The shape of furniture can be presented by the virtual shape,

There are several research limitations in this study. First, we simulated the form and state of sampling through innovative experimental methods and AR system design, aiming to quantify the objective data on furniture sampling on the shape, but because the size of the 3D printing is much smaller than the actual sampling size, the difference between the visual judgment and the spatial shape was affected. However, this study emphasizes the combination of visual space theories with the practice of technology and materials in application. The researchers are convinced that this does not affect the reliability and validity of the results because we used the same sampling method and provided the subjects with the correct ratio of the production process and the projection on the screen to make styling judgments. It turns out one-to-one proportional sampling experiments are feasible for future sampling design and technical breakthroughs. Second, while the subjects are professionally trained furniture design and production staff, they are still not real furniture sample makers. The sampling experience and handwriting skills may affect the ability to make judgments. However, the real sampler makers in the furniture factory are quite busy, and it is not easy for them to be involved in an experiment for a long time. That is the reason why we were looking for professional designers with similar work experience and styling ability as the subjects. Since this study emphasizes the shape comparison between the 2D and 3D under visual space, these subjects have the same ability as the sampler makers to determine the shape and are quite familiar with the design and sampling of furniture. We believe this can make up for the lack of real sample makers, but in future experiments, if funding and time permit, we still hope to conduct experiments with real sampler makers. Third, there is quite a variety of furniture and we only tested a chair, whose shape will affect the results of the sampling comparison data. Therefore, our main research results focus on the comparison of the sampling methods to understand whether AR technology is helpful for the sampling of furniture, but cannot fully explain the deviation of the basic structure of the furniture that is most likely to occur in the furniture sampling process. However, the structure of the furniture can still correspond to its basic shape, which is also an experiment to be carried out in future research because the basic shape (such as column, square, ellipsoid, cylinder, spindle and rectangular) is different in visual judgment; for instance, due to the lack of the right-angled structure, the ellipse is more difficult to identify and distinguish the difference in shape than the rectangular. Therefore, in the future research, this study will simplify the understanding of the model and increase the validity of the experiment through multiple visual viewpoints and modeling. However, in order to avoid focal vagueness in this study, we only focus on the basic structure of furniture rather than a unit body. Fourth, in the study, judgment on the proportion of furniture is used to determine the sampling mistakes, but there will inevitably be subtle differences which cannot be completely subdivided by the overall proportion. For example, the scaling extent at the edge of the chair back may be larger than that in the middle, but this study is only based on the overall chair back, which will cause some slight difference in the shape data. In the future, we must seek more objective software

**16**

AR technology has been applied to many innovative product designs, and is expected to evolve with the future development in the integration of virtual and real environments. The novel devices include Microsoft HoloLens, HTC VIVE Pro, and VR Oculus Rift. The development and emergence of these technological devices reflect the potential of using AR or VR to solve industrial production problems in the future. Moreover, these devices will blur the boundaries between virtual and real integration, and increase the number of gestures and innovative interfaces, which could make the manufacturing process more intuitive, and make the manual operation more flexible. Future users will have more options in operating the AR interface of tablets.

In addition to the consumer side in the design process, AR is used in different product development and modeling discussion. Previous researches have pointed out that AR is helpful for the designer's continuous design and redesign in product development [12, 29, 34, 35]. However, in the literature exploration, the researchers found that in the current stage of AR in the furniture sampling and manufacturing production, we have not seen any application of similar concepts. Therefore, we believe that furniture sampling and manufacturing is an excellent research field with great design possibility, especially because the furniture sampling involves the design and processing of furniture production. For example, in furniture product development and styling continuation design, there are many details to consider; at present, it is still difficult for wooden furniture production to overcome the physical shape difference and master the space scale of furniture sampling because wooden furniture has the following characteristics: first, the sampling of its initial prototype relies on the subjective judgment and the handcraft of the special sample maker, thus resulting in a significant difference between the sample of the furniture prototype and the blueprint. Second, wooden furniture contains many complex structures such as composite curved surfaces, curved wood shapes, and gradient bends. It is difficult for 2D drawings to present the spatial structure and modeling concept of 3D furniture, so the spatial cognitive misunderstanding may lead to inaccurate judgments of the sampling appearance. Third, wooden furniture often uses the mortising technique to assemble and connect furniture parts, and the mortising structure is often hidden within the furniture and cannot be directly seen. Due to the above special factors, the sampling of the furniture often requires repeated discussions and trials. In this context, this study is aimed at the research and knowledge construction of furniture sampling in the framework of AR technology in the comparison between physical and virtual reality. Using the systematic experimental method has defined the knowledge structure and specific operational application strategies of AR technology in the comparison of furniture shape sampling, as well as the "difference between modeling" and "virtual reality" to study the topic of furniture styling. The results show that the application of AR to furniture sampling has obvious benefit for the mastery and judgment of the furniture modeling.

This research has confirmed that AR technology is more helpful in the sampling and development of furniture, especially in the visual and styling aspects. This study has developed a set of rigorous sampling methods for AR technology based on the "physical comparison" and the "furniture sampling development", and


**Table 3.**

*Advantages and disadvantages of applying AR technology to furniture sampling.*

continues this concept to gain an in-depth understanding of 2D and 3D vision and space under the guidance of how to use AR technology to design media-assisted interfaces on furniture sampling. The advantages offered by AR in furniture sampling are listed below (**Table 3**).

#### **6.1 Providing the relative shape judgment and spatial visual reference for the sampler maker**

AR technology can provide a reference for the sample maker in the comparison of the furniture shape, and quickly construct the relative state between the virtual furniture shape and the physical one. In addition to helping master the furniture type, it can also increase the iterative correction and the basis for the shape adjustment.

#### **6.2 Deploying flexible furniture components in real time**

AR technology can quickly change the component's shape, adjust the proportion, material and shape of the furniture, and even complete the disassembly

**19**

**Author details**

**Conflict of interest**

**Informed consent**

Technology, Taipei, Taiwan

I-Jui Lee1,2

study.

*Using Augmented Reality Technology to Construct a Wood Furniture Sampling Platform…*

design can be quickly and effectively conveyed to the producer.

**6.3 Providing long-distance space formation discussion**

simulation. These 3D animations can help the sample maker quickly understand the shape and assembly set by the furniture designer. The concept of the furniture

The AR System can synchronize sampling images to different participants to view and discuss in different sampling locations, which can be used simultaneously by multiple people. With their own tablet displays (**Figure 5**), users can view from different angles and discuss together without interfering with each other. In the future construction of the AR sampling system, the modeling annotation and visual aid guidance can be added to point out the problems of sampling and details during remote discussion. Currently, Taiwan or European furniture design workshops are developing towards a simple and diverse trend of manufacturing and off-site production; AR technology will have much potential and developments in remote discussion and sampling, whose applications will once again give it a new opportu-

Therefore, in future furniture design, in addition to the accuracy of the spatial scale of the sampling, AR will play a more important part in its application in the development of new modeling and styling design. More in-depth research and modeling analysis will be conducted on these characteristics in the future.

Informed consent was obtained from all individual participants included in the

1 Ergonomics and Interaction Design Lab, Department of Industrial Design,

2 Woodworking Training Design Research Center, National Taipei University of

© 2019 The Author(s). Licensee IntechOpen. This chapter is distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/ by/3.0), which permits unrestricted use, distribution, and reproduction in any medium,

National Taipei University of Technology, Taipei, Taiwan

The authors declare that they have no conflicts of interest.

\*Address all correspondence to: ericlee@mail.ntut.edu.tw

provided the original work is properly cited.

*DOI: http://dx.doi.org/10.5772/intechopen.90471*

nity for research and development.

*Using Augmented Reality Technology to Construct a Wood Furniture Sampling Platform… DOI: http://dx.doi.org/10.5772/intechopen.90471*

simulation. These 3D animations can help the sample maker quickly understand the shape and assembly set by the furniture designer. The concept of the furniture design can be quickly and effectively conveyed to the producer.

#### **6.3 Providing long-distance space formation discussion**

The AR System can synchronize sampling images to different participants to view and discuss in different sampling locations, which can be used simultaneously by multiple people. With their own tablet displays (**Figure 5**), users can view from different angles and discuss together without interfering with each other. In the future construction of the AR sampling system, the modeling annotation and visual aid guidance can be added to point out the problems of sampling and details during remote discussion. Currently, Taiwan or European furniture design workshops are developing towards a simple and diverse trend of manufacturing and off-site production; AR technology will have much potential and developments in remote discussion and sampling, whose applications will once again give it a new opportunity for research and development.

Therefore, in future furniture design, in addition to the accuracy of the spatial scale of the sampling, AR will play a more important part in its application in the development of new modeling and styling design. More in-depth research and modeling analysis will be conducted on these characteristics in the future.

#### **Conflict of interest**

*Mixed Reality and Three-Dimensional Computer Graphics*

Spatial cognitive difference

Communication barriers

Regional restrictions

**Table 3.**

**Disadvantages of traditional furniture sampling**

space and modeling.

The sample maker follows the 2D drawing and the 1:1 simulation prototype output drawing as the aid for the sampling of the furniture, but due to the manual manufacturing process, differences between the sampling prototype and the design prototype may easily occur due to the difference in

The communication barriers between the designer and the sample maker may be due to different professional backgrounds, opinions, and locations. Thus more sampling errors will occur and repeated corrections are necessary.

Traditional sampling relies on the faceto-face communication between the designer and the sample maker; they directly examine the physical prototype for discussion and correction. However, if the location of the sample is far away, it will take a lot of time and money to travel back and forth and the sampling

process may be lengthened.

*Advantages and disadvantages of applying AR technology to furniture sampling.*

are used in traditional sampling. If we want to try different material textures and color effects, multiple sampling prototypes will be needed, which is time-consuming and costly.

Materials The same materials as the real furniture

**Advantages of applying AR technology to furniture sampling**

AR technology can combine the virtual design prototype in 3D space with the real sampling prototype, which can provide modeling corrections and discussion, and reduce spatial and visual errors caused by direct translation from 2D drawing to stereoscopic space modeling.

When using AR technology to establish a "common image language" between different professions, we can improve communication efficiency and quality, reduce sampling errors caused by poor communication, and examine the demand for mass production of furniture and structure

On the platform that integrates the virtual and real entities, we can overcome the limitations of location, time and space, to save money and time costs, provide better-optimized design communication quality, and shorten the sampling process.

We can use computing and computer virtual to replace furniture materials, color, surface and different sets of parts in real-time, which is fast and

more objectively.

cost-effective.

continues this concept to gain an in-depth understanding of 2D and 3D vision and space under the guidance of how to use AR technology to design media-assisted interfaces on furniture sampling. The advantages offered by AR in furniture sam-

**6.1 Providing the relative shape judgment and spatial visual reference for the** 

**6.2 Deploying flexible furniture components in real time**

AR technology can provide a reference for the sample maker in the comparison of the furniture shape, and quickly construct the relative state between the virtual furniture shape and the physical one. In addition to helping master the furniture type, it can also increase the iterative correction and the basis for the shape adjustment.

AR technology can quickly change the component's shape, adjust the proportion, material and shape of the furniture, and even complete the disassembly

**18**

pling are listed below (**Table 3**).

**sampler maker**

The authors declare that they have no conflicts of interest.

#### **Informed consent**

Informed consent was obtained from all individual participants included in the study.

#### **Author details**

I-Jui Lee1,2

1 Ergonomics and Interaction Design Lab, Department of Industrial Design, National Taipei University of Technology, Taipei, Taiwan

2 Woodworking Training Design Research Center, National Taipei University of Technology, Taipei, Taiwan

\*Address all correspondence to: ericlee@mail.ntut.edu.tw

© 2019 The Author(s). Licensee IntechOpen. This chapter is distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/ by/3.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

### **References**

[1] Hoang N, Toppinen A, Lähtinen K. Foreign subsidiary development in the context of a global recession: A case of the furniture industry in Vietnam. International Forestry Review. 2015;**17**(4):427-437

[2] Tracogna A, di Belgiojoso GB. The Furniture Industry in Taiwan. CSIL Reports W05TW. CSIL Centre for Industrial Studies; 2009

[3] Yang MC. A study of prototypes, design activity, and design outcome. Design Studies. 2005;**26**(6):649-669

[4] Postell J. Furniture design. New York, United States: John Wiley & Sons; 2012

[5] Ni X. Technical research on computer-aided furniture design based on human-computer interaction. Chemical Engineering Transactions. 2015;**46**:871-876

[6] Wang J, Wu ZH. The application of digital technologies in furniture design. In: Applied Engineering, Materials and Mechanics: Proceedings of the 2016 International Conference on Applied Engineering, Materials and Mechanics (ICAEMM 2016). 2016. pp. 86-90

[7] Barata TQF, Rodrigues OV, Matos BM, Pinto RS. Furniture design using MDF boards applying concepts of sustainability. Product: Management & Development. 2016;**14**(1):68-83

[8] Mujir MS, Anwar R, Hassan OH. Advanced digital design prototyping for manufacturing of exclusive wood carving furniture products. In: Proceedings of the Art and Design International Conference (AnDIC 2016). Singapore: Springer; 2018. pp. 291-297

[9] Cuendet S, Dehler-Zufferey J, Arn C, Bumbacher E, Dillenbourg P. A study of carpenter apprentices' spatial

skills. Empirical Research in Vocational Education and Training. 2014;**6**(1):3

[10] Wang S. Classic Chinese Furniture: Ming and Early Qing Dynasties. Hong Kong: Joint Pub. Co. (HK), Han-Shan Tang; 1986

[11] Kavakli M, Gero JS. Sketching as mental imagery processing. Design Studies. 2001;**22**(4):347-364

[12] Tiainen T, Ellman A, Kaapu T. Virtual prototypes reveal more development ideas: Comparison between customers' evaluation of virtual and physical prototypes: This paper argues that virtual prototypes are better than physical prototypes for consumers-involved product development. Virtual and Physical Prototyping. 2014;**9**(3):169-180

[13] Adams WE, Adams IV WE. U.S. Patent No. 9,144,309. Washington, DC: U.S. Patent and Trademark Office; 2015

[14] Barua A, Chowdhury MATA, Mehidi SH, Muhiuddin HM. Residue reduction and reuse in wooden furniture manufacturing industry. International Journal of Scientific and Engineering Research. 2014;**5**(10):291-301

[15] Liu-ju BI. Analysis of the structure form in corrugated cardboard furniture design. Packaging Engineering. 2010;**2**(10):4-14

[16] Mossbeck N. U.S. Patent No. 5,516,384. Washington, DC: U.S. Patent and Trademark Office; 1996

[17] Schkolne S, Pruett M, Schröder P. Surface drawing: Creating organic 3D shapes with the hand and tangible tools. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. ACM; 2001. pp. 261-268

**21**

*Using Augmented Reality Technology to Construct a Wood Furniture Sampling Platform…*

Communication Engineering.

[26] Liu TY. The Feasibility of Augmented Reality Applied on Furniture Allocation Service in Real-Estate Pre-sale House. (Mater's thesis); 2010. Available from airiti Library,

[27] Yamakawa T, Dobashi Y, Okabe M, Iwasaki K, Yamamoto T. Computer simulation of furniture layout when moving from one house to another. In: Proceedings of the 33rd Spring Conference on Computer Graphics.

[28] Phan VT, Choo SY. Interior design in augmented reality environment. International Journal of Computer Applications. 2010;**5**(5):16-21

[29] Tang YM, Au KM, Leung Y. Comprehending products with mixed reality: Geometric relationships and creativity. International Journal of Engineering Business Management.

2018;**10**:1847979018809599

[30] Oh H, Yoon SY, Hawley J. What virtual reality can offer to the furniture

Apparel, Technology and Management.

[31] Fernandes AS, Wang RF, Simons DJ. Remembering the physical as virtual: Source confusion and physical interaction in augmented reality. In: Proceedings of the ACM SIGGRAPH Symposium on Applied Perception.

[32] Viyanon W, Songsuittipong T, Piyapaisarn P, Sudchid S. AR furniture: Integrating augmented reality technology to enhance interior design using marker and markerless tracking. In: Proceedings of the 2nd International Conference on Intelligent Information Processing.

industry. Journal of Textile and

2004;**4**(1):1-17

ACM; 2015. pp. 127-130

ACM; 2017. p. 32

2015;**4**(3):286-288

ACM; 2017. p. 4

1-51

*DOI: http://dx.doi.org/10.5772/intechopen.90471*

[18] Lee IJ. Using augmented reality to train students to visualize threedimensional drawings of mortise–tenon joints in furniture carpentry. Interactive Learning Environments. 2019;(1):1-15

[19] ARCHI CGI. Furniture Prototype Creation: 9 Clear Advantages of Virtual Prototypes Over the Physical Ones. 2017. Available at: https://archicgi. com/furniture-prototype-virtual-or-

[20] Hongqiang Y, Ji C, Nie Y, Yinxing H. China's wood furniture manufacturing industry: Industrial cluster and export competitiveness. Forest Products Journal. 2012;**62**(3):214-221

Implementations and future perspective

[22] Umentani N, Igarashi T, Mitra NJ. Guided exploration of physically valid

[23] Nee AY, Ong SK, Chryssolouris G, Mourtzis D. Augmented reality applications in design and manufacturing. CIRP Annals.

[24] Fuji T, Mitsukura Y, Moriya T. Furniture layout AR application using floor plans based on planar object tracking. In: 2012 IEEE RO-MAN: The 21st IEEE International Symposium on Robot and Human Interactive Communication. IEEE; 2012.

[25] Jani BY, Dahale P, Nagane A, Sathe B, Wadghule N. Interior Design in Augmented Reality Environtment. International Journal of Advanced

Research in Computer and

[21] Yu Y, Wang X, Zhong RY, Huang GQ. E-commerce logistics in supply chain management:

in furniture industry. Industrial Management & Data Systems. 2017;**117**(10):2263-2286

shapes for furniture design. Communications of the ACM.

2015;**58**(9):116-124

2012;**61**(2):657-679

pp. 670-675

physical/

*Using Augmented Reality Technology to Construct a Wood Furniture Sampling Platform… DOI: http://dx.doi.org/10.5772/intechopen.90471*

[18] Lee IJ. Using augmented reality to train students to visualize threedimensional drawings of mortise–tenon joints in furniture carpentry. Interactive Learning Environments. 2019;(1):1-15

[19] ARCHI CGI. Furniture Prototype Creation: 9 Clear Advantages of Virtual Prototypes Over the Physical Ones. 2017. Available at: https://archicgi. com/furniture-prototype-virtual-orphysical/

[20] Hongqiang Y, Ji C, Nie Y, Yinxing H. China's wood furniture manufacturing industry: Industrial cluster and export competitiveness. Forest Products Journal. 2012;**62**(3):214-221

[21] Yu Y, Wang X, Zhong RY, Huang GQ. E-commerce logistics in supply chain management: Implementations and future perspective in furniture industry. Industrial Management & Data Systems. 2017;**117**(10):2263-2286

[22] Umentani N, Igarashi T, Mitra NJ. Guided exploration of physically valid shapes for furniture design. Communications of the ACM. 2015;**58**(9):116-124

[23] Nee AY, Ong SK, Chryssolouris G, Mourtzis D. Augmented reality applications in design and manufacturing. CIRP Annals. 2012;**61**(2):657-679

[24] Fuji T, Mitsukura Y, Moriya T. Furniture layout AR application using floor plans based on planar object tracking. In: 2012 IEEE RO-MAN: The 21st IEEE International Symposium on Robot and Human Interactive Communication. IEEE; 2012. pp. 670-675

[25] Jani BY, Dahale P, Nagane A, Sathe B, Wadghule N. Interior Design in Augmented Reality Environtment. International Journal of Advanced Research in Computer and

Communication Engineering. 2015;**4**(3):286-288

[26] Liu TY. The Feasibility of Augmented Reality Applied on Furniture Allocation Service in Real-Estate Pre-sale House. (Mater's thesis); 2010. Available from airiti Library, 1-51

[27] Yamakawa T, Dobashi Y, Okabe M, Iwasaki K, Yamamoto T. Computer simulation of furniture layout when moving from one house to another. In: Proceedings of the 33rd Spring Conference on Computer Graphics. ACM; 2017. p. 4

[28] Phan VT, Choo SY. Interior design in augmented reality environment. International Journal of Computer Applications. 2010;**5**(5):16-21

[29] Tang YM, Au KM, Leung Y. Comprehending products with mixed reality: Geometric relationships and creativity. International Journal of Engineering Business Management. 2018;**10**:1847979018809599

[30] Oh H, Yoon SY, Hawley J. What virtual reality can offer to the furniture industry. Journal of Textile and Apparel, Technology and Management. 2004;**4**(1):1-17

[31] Fernandes AS, Wang RF, Simons DJ. Remembering the physical as virtual: Source confusion and physical interaction in augmented reality. In: Proceedings of the ACM SIGGRAPH Symposium on Applied Perception. ACM; 2015. pp. 127-130

[32] Viyanon W, Songsuittipong T, Piyapaisarn P, Sudchid S. AR furniture: Integrating augmented reality technology to enhance interior design using marker and markerless tracking. In: Proceedings of the 2nd International Conference on Intelligent Information Processing. ACM; 2017. p. 32

**20**

pp. 291-297

*Mixed Reality and Three-Dimensional Computer Graphics*

skills. Empirical Research in Vocational Education and Training. 2014;**6**(1):3

[10] Wang S. Classic Chinese Furniture: Ming and Early Qing Dynasties. Hong Kong: Joint Pub. Co. (HK), Han-Shan

[11] Kavakli M, Gero JS. Sketching as mental imagery processing. Design Studies. 2001;**22**(4):347-364

Kaapu T. Virtual prototypes reveal more development ideas: Comparison between customers' evaluation of virtual and physical prototypes: This paper argues that virtual prototypes are better than physical prototypes for consumers-involved product development. Virtual and Physical Prototyping. 2014;**9**(3):169-180

[13] Adams WE, Adams IV WE. U.S. Patent No. 9,144,309. Washington, DC: U.S. Patent and Trademark Office;

[14] Barua A, Chowdhury MATA, Mehidi SH, Muhiuddin HM. Residue reduction and reuse in wooden furniture manufacturing industry. International Journal of Scientific and Engineering

Research. 2014;**5**(10):291-301

design. Packaging Engineering.

[16] Mossbeck N. U.S. Patent No. 5,516,384. Washington, DC: U.S. Patent

[17] Schkolne S, Pruett M, Schröder P. Surface drawing: Creating organic 3D shapes with the hand and tangible tools. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. ACM; 2001.

and Trademark Office; 1996

2010;**2**(10):4-14

pp. 261-268

[15] Liu-ju BI. Analysis of the structure form in corrugated cardboard furniture

[12] Tiainen T, Ellman A,

Tang; 1986

2015

[1] Hoang N, Toppinen A, Lähtinen K. Foreign subsidiary development in the context of a global recession: A case of the furniture industry in Vietnam. International Forestry Review.

[2] Tracogna A, di Belgiojoso GB. The Furniture Industry in Taiwan. CSIL Reports W05TW. CSIL Centre for

[3] Yang MC. A study of prototypes, design activity, and design outcome. Design Studies. 2005;**26**(6):649-669

[5] Ni X. Technical research on

[7] Barata TQF, Rodrigues OV,

Development. 2016;**14**(1):68-83

[8] Mujir MS, Anwar R, Hassan OH. Advanced digital design prototyping for manufacturing of exclusive wood carving furniture products. In: Proceedings of the Art and Design International Conference (AnDIC 2016). Singapore: Springer; 2018.

[9] Cuendet S, Dehler-Zufferey J, Arn C, Bumbacher E, Dillenbourg P. A study of carpenter apprentices' spatial

Matos BM, Pinto RS. Furniture design using MDF boards applying concepts of sustainability. Product: Management &

2015;**46**:871-876

[4] Postell J. Furniture design. New York, United States: John Wiley & Sons; 2012

computer-aided furniture design based on human-computer interaction. Chemical Engineering Transactions.

[6] Wang J, Wu ZH. The application of digital technologies in furniture design. In: Applied Engineering, Materials and Mechanics: Proceedings of the 2016 International Conference on Applied Engineering, Materials and Mechanics (ICAEMM 2016). 2016. pp. 86-90

**References**

2015;**17**(4):427-437

Industrial Studies; 2009

[33] Azuma RT. A survey of augmented reality. Presence Teleoperators and Virtual Environments. 1997;**6**(4):355-385

[34] Simons DJ, Wang RF, Roddenberry D. Object recognition is mediated by extraretinal information. Perception & Psychophysics. 2002;**64**(4):521-530

[35] Evans G, Miller J, Pena MI, MacAllister A, Winer E. Evaluating the Microsoft HoloLens through an augmented reality assembly application. In: Degraded Environments: Sensing, Processing, and Display 2017. International Society for Optics and Photonics; Vol. 10197. 2017. p. 101970V

[36] Microsoft. Microsoft HoloLens: Partner Spotlight with Autodesk Fusion 360; 2015

[37] Shen Y, Ong SK, Nee AY. Augmented reality for collaborative product design and development. Design Studies. 2010;**31**(2):118-145

[38] Pejsa T, Kantor J, Benko H, Ofek E, Wilson A. Room2room: Enabling life-size telepresence in a projected augmented reality environment. In: Proceedings of the 19th ACM Conference on Computer-Supported Cooperative Work & Social Computing. ACM; 2016. pp. 1716-1725

[39] Lee W, Park J. Augmented foam: A tangible augmented reality for product design. In: Fourth IEEE and ACM International Symposium on Mixed and Augmented Reality (ISMAR'05). IEEE; 2005. pp. 106-109

[40] Lukosch S, Billinghurst M, Kiyokawa K, Alem L, Feiner S, Prilla M. Workshop on collaborative mixed reality environments (CoMiRE) summary. In: 2016 IEEE International Symposium on Mixed and Augmented Reality (ISMAR-Adjunct). IEEE; 2016. pp. xxxv-xxxvi

[41] Unity. Unite Berlin 2018. 2018. Available at: https://unity3d.com/

**23**

**Chapter 2**

**Abstract**

**1. Introduction**

the Medical Area

Augmented Reality as a New and

Innovative Learning Platform for

This research paper shows an Augmented Reality (AR) project applied to medicine. The project is crystallized through a system, based on this new technology that serves as an innovative and innovative learning platform, which, in turn, helps in both teaching and learning abstract concepts in medicine, which requires of visual and manipulable objects difficult to obtain, due to the large space they occupy in magnetic media or because of how difficult it is to get their models in physical form. The proposed system strengthens the anatomical identification process in the area of medicine, specifically in the physiological activity of the human heart. In addition, this system allows interaction with the students, through which certain body parts of the human heart are identified, and, consequently, facilitates their learning with an iterative operation. Finally, the system is focused, so that the student uses his/her sense of sight, hearing, and kinesthetic, which, together, will allow a better assimilation of knowledge.

*Gerardo Reyes-Ruiz and Marisol Hernández-Hernández*

**Keywords:** augmented reality, medicine, systems, interaction, teaching

The need to have better prepared human resources and innovative and/or entrepreneurial ideas, whether generated during their studies or not, motivates the research community to respond, even in a timely manner, to each of the problems generated, in turn, for these needs [1–3]. In this context, it makes sense to create and provide a new way of learning for a specific group of students, particularly for students pursuing a medical degree. In turn, it is clear that the transfer of knowledge has evolved gradually over time, and it is also logical that teaching (which is nothing more than the process of educating) has also shown changes [4]. Therefore, new generations of students should be prepared and adapt as soon as possible to the challenges that new technologies face in the near future. In this way, new technologies favor education because they generate human resources with a better educational quality. Moreover, [5] it is shown that those countries that have allocated considerable economic investments for the development and education of their population achieved more efficient production and, as a result, maintained higher growth rates. Regarding the transfer of technology and, simultaneously, specialized knowledge have been subjects of study in multiple research papers [6–8]. However, since the end of the last century in [9] it was emphasized that aid to developing countries consisting only in the transfer of economic capital would not be sufficient if the developing

#### **Chapter 2**

*Mixed Reality and Three-Dimensional Computer Graphics*

[33] Azuma RT. A survey of augmented reality. Presence Teleoperators and Virtual Environments. 1997;**6**(4):355-385

Roddenberry D. Object recognition is mediated by extraretinal information.

[34] Simons DJ, Wang RF,

2002;**64**(4):521-530

360; 2015

2010;**31**(2):118-145

2005. pp. 106-109

Perception & Psychophysics.

[35] Evans G, Miller J, Pena MI, MacAllister A, Winer E. Evaluating the Microsoft HoloLens through an augmented reality assembly application. In: Degraded Environments: Sensing,

Processing, and Display 2017. International Society for Optics and Photonics; Vol. 10197. 2017. p. 101970V

[36] Microsoft. Microsoft HoloLens: Partner Spotlight with Autodesk Fusion

[37] Shen Y, Ong SK, Nee AY. Augmented reality for collaborative product design and development. Design Studies.

[38] Pejsa T, Kantor J, Benko H, Ofek E, Wilson A. Room2room: Enabling life-size telepresence in a projected augmented reality environment. In: Proceedings of the 19th ACM Conference on Computer-Supported Cooperative Work & Social Computing. ACM; 2016. pp. 1716-1725

[39] Lee W, Park J. Augmented foam: A tangible augmented reality for product design. In: Fourth IEEE and ACM International Symposium on Mixed and Augmented Reality (ISMAR'05). IEEE;

[40] Lukosch S, Billinghurst M, Kiyokawa K, Alem L, Feiner S, Prilla M. Workshop

environments (CoMiRE) summary. In: 2016 IEEE International Symposium on Mixed and Augmented Reality (ISMAR-Adjunct). IEEE; 2016. pp. xxxv-xxxvi

[41] Unity. Unite Berlin 2018. 2018. Available at: https://unity3d.com/

on collaborative mixed reality

**22**

## Augmented Reality as a New and Innovative Learning Platform for the Medical Area

*Gerardo Reyes-Ruiz and Marisol Hernández-Hernández*

#### **Abstract**

This research paper shows an Augmented Reality (AR) project applied to medicine. The project is crystallized through a system, based on this new technology that serves as an innovative and innovative learning platform, which, in turn, helps in both teaching and learning abstract concepts in medicine, which requires of visual and manipulable objects difficult to obtain, due to the large space they occupy in magnetic media or because of how difficult it is to get their models in physical form. The proposed system strengthens the anatomical identification process in the area of medicine, specifically in the physiological activity of the human heart. In addition, this system allows interaction with the students, through which certain body parts of the human heart are identified, and, consequently, facilitates their learning with an iterative operation. Finally, the system is focused, so that the student uses his/her sense of sight, hearing, and kinesthetic, which, together, will allow a better assimilation of knowledge.

**Keywords:** augmented reality, medicine, systems, interaction, teaching

#### **1. Introduction**

The need to have better prepared human resources and innovative and/or entrepreneurial ideas, whether generated during their studies or not, motivates the research community to respond, even in a timely manner, to each of the problems generated, in turn, for these needs [1–3]. In this context, it makes sense to create and provide a new way of learning for a specific group of students, particularly for students pursuing a medical degree. In turn, it is clear that the transfer of knowledge has evolved gradually over time, and it is also logical that teaching (which is nothing more than the process of educating) has also shown changes [4]. Therefore, new generations of students should be prepared and adapt as soon as possible to the challenges that new technologies face in the near future. In this way, new technologies favor education because they generate human resources with a better educational quality. Moreover, [5] it is shown that those countries that have allocated considerable economic investments for the development and education of their population achieved more efficient production and, as a result, maintained higher growth rates. Regarding the transfer of technology and, simultaneously, specialized knowledge have been subjects of study in multiple research papers [6–8]. However, since the end of the last century in [9] it was emphasized that aid to developing countries consisting only in the transfer of economic capital would not be sufficient if the developing

country does not have an adequate level of human capital to take advantage of all possible benefit of that help. Therefore, and through new technologies, it is essential to create a new horizon in science and technology in terms of favoring and/or strengthening the current learning system and, in particular, that of medicine.

On the other hand, it is clear that technological advances have directly impacted all activities of daily life, and the educational level has not been the exception. In the latter, new strategies have been created and implemented that serve as support in the teaching-learning process, which systematically strengthen the way of teaching and learning the most current educational content. Scientific and technological changes are increasingly visible and it is clear that the interaction between technology and the human being has also changed over time. That is, the speed with which current technologies are developed, but especially the enormous amount of them, does not allow a person to assimilate them in an acceptable way. The latter is probably due to the fact that the information that is handled today is so much that contemporary human beings must quickly assimilate a growing amount, precisely, of the so-called new technologies instead of adapting them to their needs. In this way, the human being has gradually become a technodependent entity. As a consequence of all this, it can be mentioned that the current dynamics of almost all daily activities, including education and/or creation of highly specialized human resources, has led us, among other things, to be immersed in a society with technologies of vanguard. Undoubtedly, this is because the assimilation of new technologies is increasingly accessible, interactive and economical.

In this globalized world, universities are taking a fundamental role in generating human resources with cutting-edge knowledge. So much so, that [10] indicates that European universities are currently incorporating digital technologies as a support to facilitate educational processes. In this context, technological elements such as the computer, mobile devices, internet, applications for the cyber cloud, projectors, slides and other hardware or software resources have been the starting point for the creation of new learning environments. In recent years, concern for the study of learning design has increased markedly [11]. Under this approach, Augmented Reality (AR) plays a transcendental and very important role, since the combination of physical reality with virtual resources can be displayed in the form of multimedia content. In [12] the AR is defined as a sequence of digital shots or computer generated information, while in [13] it is identified as a technology of high relevance for teaching, learning and creative research.

Among the strategies that have shown great support for the new educational systems, generated through the AR, stand out the Intelligent Tutorial Systems (ITS). In addition, ITSs have proven to be a fairly efficient tool to support learning processes and if ICTs are added to this dynamic, then completely new results can be obtained [14]. For example, these intelligent systems make use of new technologies to improve the performance and motivation of workers when assembling an appliance, relying on a manual containing AR. In this sense, the "Intelligent augmented reality training for mother board assembly" has shown great contributions in the training for manual assembly tasks [15]. With this perspective, a more effective and faster learning experience has been presented, where less experienced users learn to mount a "motherboard" on the computer. Therefore, AR can be a technological tool of great support for learning, which can be verified in the system "Augmented reality in informal learning environments: A field experiment in a mathematics exhibition" [16]. Through this educational environment, students manage to associate and understand, in a more didactic way, mathematical figures with entities that exist in the physical world. Also, the "An interactive augmented reality system for learning anatomy structure" system, which is set out in [17], shows the parts of the anatomical structure of the human body. This system allows students to identify each anatomical part, but not only shows

**25**

*Augmented Reality as a New and Innovative Learning Platform for the Medical Area*

avatars of a story, that is, the user can enter the virtual world of the book.

in a novel way the functioning of the human heart.

program that was established was as follows:

functioning of the human heart.

webcam, a loud voice or a mobile device.

**2. Application description**

those anatomical parts but also that the interaction is more dynamic and in greater detail. For its part, the "Interactive augmented reality using Scratch 2.0 to improve physical activities for children with developmental disabilities" system, shown in [18], uses an interactive game for body movement, whose purpose is to improve children's driving force with a disability To conclude with the applications of the AR, in Magic Book [19], which is a normal book containing bookmarks, the system when it detects a bookmark then shows a three-dimensional image or starts a video story. In addition, in this system users can feel the sensation of flying and appreciate themselves as

In this way, the multiple scenarios offered by the AR are extremely competent, which can be well adopted in multiple fields of knowledge. There are applications that target the mass market for advertising, entertainment and education [20]. However, medicine has been little explored in this context. Learning medicine also requires optimal means, both technological and other, that allow students to learn by doing. This can be achieved with interactive systems, since this dynamic makes knowledge reach the brain in a sensitive, visual, auditory and kinesthetic way. This with the purpose of acquiring the necessary, sufficient and current academic competences for their adequate professional development as future doctors; they also require economic systems that do not need accessories that are not easily accessible and difficult to obtain. Therefore, and to acquire the skills of know-how, medical students must learn to perform medical procedures with living or dead people. It is here where, precisely, new technologies make sense and relevance because multiple medical processes can well be simulated by AR. Developing a platform that manages AR would be a means of training for each student, whose studies are related to medicine, to evaluate their own learning before performing any procedure with living people. Thus, the present work aims to create an interactive system or platform that serves as a support for learning medical procedures and, in particular, to know

The objective of this work is to apply the AR through a knowledge management system, which will function as a novel and efficient learning strategy to be applied in the area of medicine. Based on the software engineering mentioned in [21], where communication guidelines, requirements analysis, design, program construction, testing and support are used for the construction of software. The work

a.Analysis of the system requirements with AR. In this phase, we investigated the knowledge that the student should acquire with this material, that is, the

b.Formulate the abstract and physical design, code and create the software. In this stage the AR was carried out, that is, a human heart was designed to be shown three-dimensionally and the events (the characteristics of the Main Menu presented by the AR) they were encoded with the help of JavaScript, HTML5 and with the implementation of A-frame, free type software.

c.Prepare the files, identify and create test data, test and integrate the software. The elements of the AR were encapsulated, obtaining the file that can be executed in a Windows or Mac environment, with the help of a web server, a

*DOI: http://dx.doi.org/10.5772/intechopen.90871*

#### *Augmented Reality as a New and Innovative Learning Platform for the Medical Area DOI: http://dx.doi.org/10.5772/intechopen.90871*

those anatomical parts but also that the interaction is more dynamic and in greater detail. For its part, the "Interactive augmented reality using Scratch 2.0 to improve physical activities for children with developmental disabilities" system, shown in [18], uses an interactive game for body movement, whose purpose is to improve children's driving force with a disability To conclude with the applications of the AR, in Magic Book [19], which is a normal book containing bookmarks, the system when it detects a bookmark then shows a three-dimensional image or starts a video story. In addition, in this system users can feel the sensation of flying and appreciate themselves as avatars of a story, that is, the user can enter the virtual world of the book.

In this way, the multiple scenarios offered by the AR are extremely competent, which can be well adopted in multiple fields of knowledge. There are applications that target the mass market for advertising, entertainment and education [20]. However, medicine has been little explored in this context. Learning medicine also requires optimal means, both technological and other, that allow students to learn by doing. This can be achieved with interactive systems, since this dynamic makes knowledge reach the brain in a sensitive, visual, auditory and kinesthetic way. This with the purpose of acquiring the necessary, sufficient and current academic competences for their adequate professional development as future doctors; they also require economic systems that do not need accessories that are not easily accessible and difficult to obtain. Therefore, and to acquire the skills of know-how, medical students must learn to perform medical procedures with living or dead people. It is here where, precisely, new technologies make sense and relevance because multiple medical processes can well be simulated by AR. Developing a platform that manages AR would be a means of training for each student, whose studies are related to medicine, to evaluate their own learning before performing any procedure with living people. Thus, the present work aims to create an interactive system or platform that serves as a support for learning medical procedures and, in particular, to know in a novel way the functioning of the human heart.

#### **2. Application description**

The objective of this work is to apply the AR through a knowledge management system, which will function as a novel and efficient learning strategy to be applied in the area of medicine. Based on the software engineering mentioned in [21], where communication guidelines, requirements analysis, design, program construction, testing and support are used for the construction of software. The work program that was established was as follows:


*Mixed Reality and Three-Dimensional Computer Graphics*

country does not have an adequate level of human capital to take advantage of all possible benefit of that help. Therefore, and through new technologies, it is essential to create a new horizon in science and technology in terms of favoring and/or strengthening the current learning system and, in particular, that of medicine.

technologies is increasingly accessible, interactive and economical.

teaching, learning and creative research.

On the other hand, it is clear that technological advances have directly impacted all activities of daily life, and the educational level has not been the exception. In the latter, new strategies have been created and implemented that serve as support in the teaching-learning process, which systematically strengthen the way of teaching and learning the most current educational content. Scientific and technological changes are increasingly visible and it is clear that the interaction between technology and the human being has also changed over time. That is, the speed with which current technologies are developed, but especially the enormous amount of them, does not allow a person to assimilate them in an acceptable way. The latter is probably due to the fact that the information that is handled today is so much that contemporary human beings must quickly assimilate a growing amount, precisely, of the so-called new technologies instead of adapting them to their needs. In this way, the human being has gradually become a technodependent entity. As a consequence of all this, it can be mentioned that the current dynamics of almost all daily activities, including education and/or creation of highly specialized human resources, has led us, among other things, to be immersed in a society with technologies of vanguard. Undoubtedly, this is because the assimilation of new

In this globalized world, universities are taking a fundamental role in generating human resources with cutting-edge knowledge. So much so, that [10] indicates that European universities are currently incorporating digital technologies as a support to facilitate educational processes. In this context, technological elements such as the computer, mobile devices, internet, applications for the cyber cloud, projectors, slides and other hardware or software resources have been the starting point for the creation of new learning environments. In recent years, concern for the study of learning design has increased markedly [11]. Under this approach, Augmented Reality (AR) plays a transcendental and very important role, since the combination of physical reality with virtual resources can be displayed in the form of multimedia content. In [12] the AR is defined as a sequence of digital shots or computer generated information, while in [13] it is identified as a technology of high relevance for

Among the strategies that have shown great support for the new educational systems, generated through the AR, stand out the Intelligent Tutorial Systems (ITS). In addition, ITSs have proven to be a fairly efficient tool to support learning processes and if ICTs are added to this dynamic, then completely new results can be obtained [14]. For example, these intelligent systems make use of new technologies to improve the performance and motivation of workers when assembling an appliance, relying on a manual containing AR. In this sense, the "Intelligent augmented reality training for mother board assembly" has shown great contributions in the training for manual assembly tasks [15]. With this perspective, a more effective and faster learning experience has been presented, where less experienced users learn to mount a "motherboard" on the computer. Therefore, AR can be a technological tool of great support for learning, which can be verified in the system "Augmented reality in informal learning environments: A field experiment in a mathematics exhibition" [16]. Through this educational environment, students manage to associate and understand, in a more didactic way, mathematical figures with entities that exist in the physical world. Also, the "An interactive augmented reality system for learning anatomy structure" system, which is set out in [17], shows the parts of the anatomical structure of the human body. This system allows students to identify each anatomical part, but not only shows

**24**

#### **3. Basic concepts**

AR arose in the year of 1996, when ARQuake is presented, the first outdoor game with mobile devices of AR, developed in [22]. Then, in 2008, the Wikitude travel and tour guide application was released for sale [23], which was done through AR by means of a digital compass, orientation sensors and accelerometer, maps, video and informative content from Wikipedia. In 2009, ARToolKit [24] arises, which is a totally oriented platform to generate AR. From these applications, the AR has been used as the basis of numerous projects in different areas, ranging from entertainment, industry, maintenance, music, medicine and education, among others. Moreover, very specific applications have been made as a virtual harp that was designed for people with disabilities, which works through vibrations [25].

In the educational field, with the AR, molecular structures, mathematics, architecture, astronomy and physical activities can be taught to children with disabilities. In medicine and education, specialized projects have been developed that show AR as an efficient learning tool, such as the "An interactive augmented reality system for learning anatomy structure" system [26] which is integrated into three activities; The first is to show the parts of the anatomical structure of the human body, the second is that it allows students to identify each anatomical part of the human body, and the third allows a deep glimpse of the internal parts of the aforementioned anatomical structure.

In this way, our learning platform describes the design of a system such as the establishment of data structures, the general architecture of the software and the representations of interfaces and algorithms. That is, the system process translates the requirements into software specifications. The objective of the design phase is to publicize the behavior of the proposed solution; This is conceived taking into account that the design is a preface that begins the construction of programs and/or activity processes that are normally carried out by users, which seek to be improved by adding speed, efficiency, efficiency, savings and visual design. The first action that begins with the design is the determination of the system architecture, which refers to the hierarchical structure of the program modules and, in addition, is focused on the way to interact between its components and the structure of the data used by them. There are currently several software architectural styles. However, and due to the nature of the system presented in this work, the methodology applied (which is also called active or dynamic practice) has as its main feature to perform a search for the immediate application or use the knowledge acquired to then confront them with practical problems or real concrete circumstances [27]; All of these features adapt perfectly to the objective of our system.

The system oriented to the functioning of the human heart with AR aims to show and obtain an application (software) that helps medical students to identify the parts that make up this operation in a virtual model, through various sensory senses. With the approach shown in [28], the software presented in this work is aimed at making available to students an application that is easy to obtain and at the lowest possible cost, since the system can be run on a desktop computer and whose main requirement is that it contains a webcam for the detection of the images that will be processed with the AR. This technology makes it possible first, for medical students to learn in a three-dimensional entity and then, that they can simulate the physical constitution of a model and the functioning of the parts involved in the psychological process related to the functioning of the human heart.

#### **4. System architecture**

After analyzing the system and understanding and determining how it works, the requirements to create it are obtained and it is then time to proceed to elaborate

**27**

**Figure 2.**

**Figure 1.**

*Augmented Reality as a New and Innovative Learning Platform for the Medical Area*

the design and construction, represented by its general software architecture and interfaces and algorithms, that is, the process that translates the requirements into software specifications. The objective of the system design phase, show the behav-

*DOI: http://dx.doi.org/10.5772/intechopen.90871*

*System sequence diagram (source: prepared by the authors).*

*System activities diagram (source: prepared by the authors).*

ior of the solution proposal.

the design and construction, represented by its general software architecture and interfaces and algorithms, that is, the process that translates the requirements into software specifications. The objective of the system design phase, show the behavior of the solution proposal.

#### **Figure 1.**

*Mixed Reality and Three-Dimensional Computer Graphics*

AR arose in the year of 1996, when ARQuake is presented, the first outdoor game with mobile devices of AR, developed in [22]. Then, in 2008, the Wikitude travel and tour guide application was released for sale [23], which was done through AR by means of a digital compass, orientation sensors and accelerometer, maps, video and informative content from Wikipedia. In 2009, ARToolKit [24] arises, which is a totally oriented platform to generate AR. From these applications, the AR has been used as the basis of numerous projects in different areas, ranging from entertainment, industry, maintenance, music, medicine and education, among others. Moreover, very specific applications have been made as a virtual harp that was

designed for people with disabilities, which works through vibrations [25].

psychological process related to the functioning of the human heart.

After analyzing the system and understanding and determining how it works, the requirements to create it are obtained and it is then time to proceed to elaborate

In the educational field, with the AR, molecular structures, mathematics, architecture, astronomy and physical activities can be taught to children with disabilities. In medicine and education, specialized projects have been developed that show AR as an efficient learning tool, such as the "An interactive augmented reality system for learning anatomy structure" system [26] which is integrated into three activities; The first is to show the parts of the anatomical structure of the human body, the second is that it allows students to identify each anatomical part of the human body, and the third allows a deep glimpse of the internal parts of the aforementioned anatomical structure. In this way, our learning platform describes the design of a system such as the establishment of data structures, the general architecture of the software and the representations of interfaces and algorithms. That is, the system process translates the requirements into software specifications. The objective of the design phase is to publicize the behavior of the proposed solution; This is conceived taking into account that the design is a preface that begins the construction of programs and/or activity processes that are normally carried out by users, which seek to be improved by adding speed, efficiency, efficiency, savings and visual design. The first action that begins with the design is the determination of the system architecture, which refers to the hierarchical structure of the program modules and, in addition, is focused on the way to interact between its components and the structure of the data used by them. There are currently several software architectural styles. However, and due to the nature of the system presented in this work, the methodology applied (which is also called active or dynamic practice) has as its main feature to perform a search for the immediate application or use the knowledge acquired to then confront them with practical problems or real concrete circumstances [27]; All of these features adapt perfectly to the objective of our system. The system oriented to the functioning of the human heart with AR aims to show and obtain an application (software) that helps medical students to identify the parts that make up this operation in a virtual model, through various sensory senses. With the approach shown in [28], the software presented in this work is aimed at making available to students an application that is easy to obtain and at the lowest possible cost, since the system can be run on a desktop computer and whose main requirement is that it contains a webcam for the detection of the images that will be processed with the AR. This technology makes it possible first, for medical students to learn in a three-dimensional entity and then, that they can simulate the physical constitution of a model and the functioning of the parts involved in the

**3. Basic concepts**

**26**

**4. System architecture**

*System sequence diagram (source: prepared by the authors).*

#### **Figure 2.** *System activities diagram (source: prepared by the authors).*

#### **Figure 3.**

*Prototype of the system. (source: prepared by the authors).*

The logical view of the system architecture, describes its structure and functionality, for this, a sequence diagram was used, where you can observe the actions that the entities of the system do, since the user of the RA focuses the camera of the device on the marker that triggers the RA, through the learning actions and finally the evaluation of the acquired knowledge.

The system can be seen in **Figure 1**, where the diagram shows the actions, starting with the one that gives access to the system, which is stored on a website, and contains link buttons showing the functions with the options: video, audio and learning, from the last one an evaluation is derived.

Process view: to represent this view, it is done with an activity diagram shown in **Figure 2**, for this view only the core activities of the systems are shown, which is to demonstrate the AR.

For the development of the system, it was taken into account that the users already have an academic trajectory and that, their required disciplinary knowledge, are more specific, so, the resources made include knowledge directed towards students of professional level, so that in prototype of the system, which can be seen in **Figure 3**.

#### **5. User interface**

AR is shown with virtual objects that can be shown by the association between the two-dimensional (2D), called marker, initial image and a three-dimensional (3D) image, where the first image is focused on the camera and takes the form of a physiological model of the human heart (the image was made in SketchUp and the AR was encoded with JavaScript and HTML5 on an A-frame platform), which can be manipulated to visualize it from a different position and angle, giving the appearance of a real physical model (**Figure 4**).

The system displays a Menu of options (Functions, Audio, Video and Learning), which "trigger" an event when the user clicks on any of its options (these actions were programmed using JavaScript on an A-frame platform). That way, when the user clicks on the Functions option, a web page will be displayed, whose purpose will be to show each element that is involved in the functioning of the human heart, showing images and text that describe it (**Figure 5**). When the user selects

**29**

**Figure 5.**

**Figure 4.**

*Augmented Reality as a New and Innovative Learning Platform for the Medical Area*

*Image of a two-dimensional (2D) human heart placed in front of a camera (source: prepared by the authors).*

*Result when selecting the menu functions option (source: prepared by the authors).*

*DOI: http://dx.doi.org/10.5772/intechopen.90871*

*Augmented Reality as a New and Innovative Learning Platform for the Medical Area DOI: http://dx.doi.org/10.5772/intechopen.90871*

**Figure 4.**

*Mixed Reality and Three-Dimensional Computer Graphics*

the evaluation of the acquired knowledge.

*Prototype of the system. (source: prepared by the authors).*

demonstrate the AR.

in **Figure 3**.

**Figure 3.**

**5. User interface**

learning, from the last one an evaluation is derived.

appearance of a real physical model (**Figure 4**).

The logical view of the system architecture, describes its structure and functionality, for this, a sequence diagram was used, where you can observe the actions that the entities of the system do, since the user of the RA focuses the camera of the device on the marker that triggers the RA, through the learning actions and finally

The system can be seen in **Figure 1**, where the diagram shows the actions, starting with the one that gives access to the system, which is stored on a website, and contains link buttons showing the functions with the options: video, audio and

Process view: to represent this view, it is done with an activity diagram shown in **Figure 2**, for this view only the core activities of the systems are shown, which is to

For the development of the system, it was taken into account that the users already have an academic trajectory and that, their required disciplinary knowledge, are more specific, so, the resources made include knowledge directed towards students of professional level, so that in prototype of the system, which can be seen

AR is shown with virtual objects that can be shown by the association between the two-dimensional (2D), called marker, initial image and a three-dimensional (3D) image, where the first image is focused on the camera and takes the form of a physiological model of the human heart (the image was made in SketchUp and the AR was encoded with JavaScript and HTML5 on an A-frame platform), which can be manipulated to visualize it from a different position and angle, giving the

The system displays a Menu of options (Functions, Audio, Video and Learning), which "trigger" an event when the user clicks on any of its options (these actions were programmed using JavaScript on an A-frame platform). That way, when the user clicks on the Functions option, a web page will be displayed, whose purpose will be to show each element that is involved in the functioning of the human heart, showing images and text that describe it (**Figure 5**). When the user selects

**28**

*Image of a two-dimensional (2D) human heart placed in front of a camera (source: prepared by the authors).*

the Audio option in the Menu then an audio file will be displayed where the main functions of the human heart are related (**Figure 6**).

On the other hand, when the user selects the Video option from the Menu, a video file will be displayed, showing the operation, in extenso, of the heart (see **Figure 7**). Finally, when the user selects the Learning Menu option then the system will show a didactic test where the student will have the experience of experiencing and practicing, through the AR, the functioning of all parts of the human heart. In this didactic test, you can accumulate the evolution of your learning and different levels of learning will be shown, such as a video game (**Figure 8**).

**31**

**Figure 9.**

*Didactic test carried out by AR (source: prepared by the authors).*

**Figure 8.**

*Augmented Reality as a New and Innovative Learning Platform for the Medical Area*

*Result when selecting the menu learning option (source: prepared by the authors).*

*DOI: http://dx.doi.org/10.5772/intechopen.90871*

**Figure 7.** *Result when selecting the video option from the menu (source: prepared by the authors).*

*Augmented Reality as a New and Innovative Learning Platform for the Medical Area DOI: http://dx.doi.org/10.5772/intechopen.90871*

**Figure 8.** *Result when selecting the menu learning option (source: prepared by the authors).*

**Figure 9.** *Didactic test carried out by AR (source: prepared by the authors).*

*Mixed Reality and Three-Dimensional Computer Graphics*

functions of the human heart are related (**Figure 6**).

levels of learning will be shown, such as a video game (**Figure 8**).

*Result when selecting the video option from the menu (source: prepared by the authors).*

*Result when selecting the audio option from the menu (source: prepared by the authors).*

the Audio option in the Menu then an audio file will be displayed where the main

On the other hand, when the user selects the Video option from the Menu, a video file will be displayed, showing the operation, in extenso, of the heart (see **Figure 7**). Finally, when the user selects the Learning Menu option then the system will show a didactic test where the student will have the experience of experiencing and practicing, through the AR, the functioning of all parts of the human heart. In this didactic test, you can accumulate the evolution of your learning and different

**30**

**Figure 7.**

**Figure 6.**

In this didactic test, a series of questions are asked for student self-evaluation, where you can appreciate the evolution of their acquired knowledge. This evaluation contains questions with dichotomous answers (the questions were implemented with JavaScript events carried out on the SDK platform), which serve as input for the system to show the score obtained by the user (**Figure 9**).

#### **6. Results and tests**

To evaluate the system presented in this work, computers were installed in a school where medical learning units are taught: 2 text sheets were provided to 28 fifth-semester students to learn about the functioning of the human heart and, subsequently, they were shown the system developed in this work. Since the students had the opportunity and experience to know the system, a survey was applied. The purpose of this survey was to know three assessment parameters (according to the objective of this research and whose concept implies that students learn by doing) about the AR-based system: learning, motivation and ease of use. Likewise, the level of knowledge retention was evaluated with respect to the aforementioned variables, which was carried out by comparing the two ways in which students studied the functioning of the human heart, that is, with and without AR. **Table 1** summarizes the responses of the students surveyed regarding; (1) the self-assessment score in the system; (2) how they felt with the use of the system and; (3) how accessible or complicated the system management seemed to them.


**Table 1.**

*Results for the AR-based learning system (source: prepared by the authors).*

#### **7. Conclusions**

The creation of a new learning platform, through the AR, will allow medical students, particularly anatomy and physiology, to obtain quality knowledge, as well as a correct approach to new technologies for timely execution. of your activities either in an operating room or in your classroom. To acquire the skills of know-how, medical students must learn to perform medical procedures with living or dead people, these processes can be simulated with AR. A platform with these qualities is a means of training for each student to evaluate their own learning, before doing it with living people.

There are multiple complements to make RA a more interesting and attractive experience. Among them we can highlight the lenses for AR (which can even handle and display certain data when used), helmets adapted with visors for AR and surround sound, or smartphones of the latest generation with AR. In virtual or mixed environments, you can use gloves, screens, or rooms equipped with specific objects to feel and/or appreciate the reality extended virtually. Undoubtedly, there are countless options to generate new environments, whether educational or not, where the AR is the main tool for its development. These complements and

**33**

**Author details**

Reyes Ruiz.

Gerardo Reyes-Ruiz1

**Acknowledgements**

**Conflict of interest**

Chalco Valley, Mexico

Institute, Mexico City, Mexico

*Augmented Reality as a New and Innovative Learning Platform for the Medical Area*

new environments will depend on the budget available for their implementation, development or creation, but, in general, their creation is not too expensive. What is clear is that these environments favor and facilitate the learning process of students

Through this work, the following is verified: (1) AR is a useful and easy-to-use tool that works to build suitable learning environments, which allow students to feel motivated, encouraged and eager to continue learning; (2) With the support of the AR systems can be generated that help the learning of abstract or difficult to perceive knowledge; (3) Medicine handles models whose appearance and form help and strengthen learning, which can be represented with three-dimensional entities (3D); (4) The interaction with the AR and the multimedia materials that are added to the physical reality allow the sensory senses of the human being, and in particular of the students, to be progressively stimulated, thereby achieving that the student learns in an auditory, visual and kinesthetic; (5) For its part, the design cost will depend on how much is available to invest in accessories and/or complements to show the AR, but the programming and design of virtual reality (three-dimensional (3D) design, simulation and page web) are generally not very expensive. However, an advantage of the proposed system is that it could be reused, adapting new knowledge and free three-dimensional (3D) models on the web. Finally, mention that the challenges of the AR for educational environments are vast and transcendental. However, a contribution of this nature allows cementing the foundations to broaden the current horizon of learning and create a new mosaic of knowledge.

\* and Marisol Hernández-Hernández2

2 Chalco Valley University Center, Autonomous Mexico State University,

\*Address all correspondence to: greyesruiz@hotmail.com

provided the original work is properly cited.

The authors declare no conflict of interest.

1 Center for Economic, Administrative and Social Research, National Polytechnic

© 2020 The Author(s). Licensee IntechOpen. This chapter is distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/ by/3.0), which permits unrestricted use, distribution, and reproduction in any medium,

The authors also thank to CONACYT for the postdoctoral grant of Dr. Gerardo

*DOI: http://dx.doi.org/10.5772/intechopen.90871*

at any educational level.

*Augmented Reality as a New and Innovative Learning Platform for the Medical Area DOI: http://dx.doi.org/10.5772/intechopen.90871*

new environments will depend on the budget available for their implementation, development or creation, but, in general, their creation is not too expensive. What is clear is that these environments favor and facilitate the learning process of students at any educational level.

Through this work, the following is verified: (1) AR is a useful and easy-to-use tool that works to build suitable learning environments, which allow students to feel motivated, encouraged and eager to continue learning; (2) With the support of the AR systems can be generated that help the learning of abstract or difficult to perceive knowledge; (3) Medicine handles models whose appearance and form help and strengthen learning, which can be represented with three-dimensional entities (3D); (4) The interaction with the AR and the multimedia materials that are added to the physical reality allow the sensory senses of the human being, and in particular of the students, to be progressively stimulated, thereby achieving that the student learns in an auditory, visual and kinesthetic; (5) For its part, the design cost will depend on how much is available to invest in accessories and/or complements to show the AR, but the programming and design of virtual reality (three-dimensional (3D) design, simulation and page web) are generally not very expensive. However, an advantage of the proposed system is that it could be reused, adapting new knowledge and free three-dimensional (3D) models on the web. Finally, mention that the challenges of the AR for educational environments are vast and transcendental. However, a contribution of this nature allows cementing the foundations to broaden the current horizon of learning and create a new mosaic of knowledge.

#### **Acknowledgements**

*Mixed Reality and Three-Dimensional Computer Graphics*

**6. Results and tests**

In this didactic test, a series of questions are asked for student self-evaluation, where you can appreciate the evolution of their acquired knowledge. This evaluation contains questions with dichotomous answers (the questions were implemented with JavaScript events carried out on the SDK platform), which serve as

To evaluate the system presented in this work, computers were installed in a school where medical learning units are taught: 2 text sheets were provided to 28 fifth-semester students to learn about the functioning of the human heart and, subsequently, they were shown the system developed in this work. Since the students had the opportunity and experience to know the system, a survey was applied. The purpose of this survey was to know three assessment parameters (according to the objective of this research and whose concept implies that students learn by doing) about the AR-based system: learning, motivation and ease of use. Likewise, the level of knowledge retention was evaluated with respect to the aforementioned variables, which was carried out by comparing the two ways in which students studied the functioning of the human heart, that is, with and without AR. **Table 1** summarizes the responses of the students surveyed regarding; (1) the self-assessment score in the system; (2) how they felt with the use of the system and; (3) how

The creation of a new learning platform, through the AR, will allow medical students, particularly anatomy and physiology, to obtain quality knowledge, as well as a correct approach to new technologies for timely execution. of your activities either in an operating room or in your classroom. To acquire the skills of know-how, medical students must learn to perform medical procedures with living or dead people, these processes can be simulated with AR. A platform with these qualities is a means of training for each student to evaluate their own learning, before doing it

**With AR Without AR**

There are multiple complements to make RA a more interesting and attractive experience. Among them we can highlight the lenses for AR (which can even handle and display certain data when used), helmets adapted with visors for AR and surround sound, or smartphones of the latest generation with AR. In virtual or mixed environments, you can use gloves, screens, or rooms equipped with specific objects to feel and/or appreciate the reality extended virtually. Undoubtedly, there are countless options to generate new environments, whether educational or not, where the AR is the main tool for its development. These complements and

input for the system to show the score obtained by the user (**Figure 9**).

accessible or complicated the system management seemed to them.

*Results for the AR-based learning system (source: prepared by the authors).*

Significant learning 89.3% 35% Motivation 94.6 35.7 Easy to use 93.8 96.6 Performance 82% 60%

**32**

**7. Conclusions**

**Table 1.**

with living people.

The authors also thank to CONACYT for the postdoctoral grant of Dr. Gerardo Reyes Ruiz.

#### **Conflict of interest**

The authors declare no conflict of interest.

#### **Author details**

Gerardo Reyes-Ruiz1 \* and Marisol Hernández-Hernández2

1 Center for Economic, Administrative and Social Research, National Polytechnic Institute, Mexico City, Mexico

2 Chalco Valley University Center, Autonomous Mexico State University, Chalco Valley, Mexico

\*Address all correspondence to: greyesruiz@hotmail.com

© 2020 The Author(s). Licensee IntechOpen. This chapter is distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/ by/3.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

#### **References**

[1] Naudé W, Szirmai A, Goedhuys M. Innovation and entrepreneurship in developing countries. UNU Policy Brief. 2011;**1**:1-7. ISBN 978-92-808-3093-4

[2] Wong PK, Ho YP, Autio E. Entrepreneurship, innovation and economic growth, evidence from GEM data. Small Business Economics. 2005;**24**(3):335-350. DOI: 10.1007/ s11187-005-2000-1

[3] Beugelsdijk S, Noorderhaven N. Entrepreneurial attitude and economic growth. A cross-section of 54 regions. The Annals of Regional Science. 2004;**38**:199-218. DOI: 10.1007/ s00168-004-0192-y

[4] Webber DJ. Policies to stimulate growth: Should we invest in health or education? Applied Economics. 2002;**34**:1633-1643. DOI: 10.1080/00036840110115109

[5] Khurram J, Kirsten W, Phanindra VW. The effect of political regimes and technology on economic growth. Applied Economics. 2007;**39**(11):1425-1432. DOI: 10.1080/00036840500447906

[6] Crespi G, Zuniga P. Innovation and productivity: Evidence from six Latin American countries. World Development. 2012;**40**(2):273-290. DOI: 10.1016/j.worlddev.2011.07.010

[7] Tomokazu N. Contribution of education and educational equality to economic growth. Applied Economics Letters. 2007;**14**:627-630. DOI: 10.1080/13504850500425857

[8] Moon YJ, Kym HG. A model for the value of intellectual capital. Canadian Journal of Administrative Sciences. 2006;**23**(3):253-269. DOI: 10.1111/ j.1936-4490.2006.tb00630.x

[9] Krueger A, Ruttan V. Development thought and development assistance. In: Aid and Development. Baltimore, USA: The Johns Hopkins University Press; 1989. pp. 13-28

[10] Marcelo C, Yot C, Mayor C. University teaching with digital technologies. Comunicar. 2015;**45**(23):117-124. DOI: 10.3916/ C45-2015-12

[11] Laurillard D. Teaching as a Design Science: Building Pedagogical Patterns for Learning and Technology, Routledge. 1st ed. New York: USA; 2013. 272p. ISBN-10: 041580387X

[12] Kipper G, Rampolla J. Augmented Reality: An Emerging Technologies Guide to AR. USA: Syngress Publishing/ Elsevier; 2012. ISBN: 1597497339 9781597497336

[13] Durall E, Gros B, Maina M, Johnson L, Adams S. Perspectivas tecnológicas: educación superior en Iberoamérica 2012-2017. Austin, Texas: The New Media Consortium; 2012. ISBN: 978-0-9846601-9-3

[14] Rodríguez RM, Castillo JLM, Lira AL. Diseño de un sistema tutorial inteligente. Apertura. 2013;**5**(1):36-47

[15] Westerfield G, Mitrovic A, Billinghurst M. Intelligent augmented reality training for assembly tasks. In: Lane HC, Yacef K, Mostow J, Pavlik P, editors. Artificial Intelligence in Education (AIED 2013). Lecture Notes in Computer Science. Vol. 7926. Berlin, Heidelberg: Springer; 2013. DOI: 10.1007/978-3-642-39112-5\_55

[16] Sommerauer P, Müller O. Augmented reality in informal learning environments: A field experiment in a mathematics exhibition. Computers

**35**

artoolkit/

*Augmented Reality as a New and Innovative Learning Platform for the Medical Area*

reality. In: 2013 Virtual Reality, IEEE Annual International Symposium, Lake Buena Vista, FL, USA. 2013. pp. 145- 146. DOI: 10.1109/VR.2013.6549404

[26] Chien CH, Chien-Hsu C, Jeng TS. An interactive augmented reality system for learning anatomy structure. In: Proceedings of the International Multi Conference of Engineers and Computer Scientists, IMECS 2010 March 17-19, Hong Kong.

Vol. 1. 2010. pp. 1-6

b18703-15

[27] Leventon J, Fleskens L, Claringbould H, Schwilch G, Hessel R. An applied methodology for stakeholder identification in transdisciplinary research.

Sustainability Science. 2016;**11**(5):763- 775. DOI: 10.1007/s11625-016-0385-1

[28] Azuma R. Location-based mixed and augmented reality storytelling. In: Barfield W, editor. Fundamentals of Wearable Computers and Augmented Reality (chapter 11). 2nd ed. CRC Press; 2015. pp. 259-276. DOI: 10.1201/

*DOI: http://dx.doi.org/10.5772/intechopen.90871*

in Education. 2014;**79**:59-68. DOI: 10.1016/j.compedu.2014.07.013

[17] Chien CH, Chien-Hsu C, Jeng TS. An interactive augmented reality system for learning anatomy structure. In: Proceedings of the International Multi Conference of Engineers and Computer Scientists IMECS 2010 March 17-19, Hong Kong. Vol. 1. 2010. pp. 1-6

[18] Lin CY, Chang YM. Interactive augmented reality using scratch 2.0 to improve physical activities for children with developmental disabilities. Research in Developmental Disabilities. 2015;**37**: 1-8. DOI: 10.1016/j.ridd.2014.10.016

[19] Westerfield G, Mitrovic A, Billinghurst M. Intelligent augmented reality training form mother board assembly. International Journal of Artificial Intelligence in Education. 2015;**25**(1):157-172. DOI: 10.1007/

[20] Rabbi I, Ullah S. A survey on augmented reality challenges and tracking. Acta Graphica. 2013;**24**(1-

[21] Kendall KE, Kendall JE. Systems Analysis and Design. USA: Pearson Education, Inc.; 2011. 600p. ISBN:

[22] Piekarski W, Thomas B. ARQuake: The outdoor augmented reality gaming system. Communications of the ACM. 2002;**45**(1):36-38. DOI:

[23] Wikitude [Internet]. 2019. Available from: https://www.wikitude.com/

[24] ARToolkit [Internet]. Available from: http://www.hitl.washington.edu/

framework for visuo-haptic augmented

[25] Eck U, Sandor C. HARP: A

2):29-46. ISSN: 0353-4707

s40593-014-0032-x

978-607-32-0577-1

10.1145/502269.502291

*Augmented Reality as a New and Innovative Learning Platform for the Medical Area DOI: http://dx.doi.org/10.5772/intechopen.90871*

in Education. 2014;**79**:59-68. DOI: 10.1016/j.compedu.2014.07.013

[17] Chien CH, Chien-Hsu C, Jeng TS. An interactive augmented reality system for learning anatomy structure. In: Proceedings of the International Multi Conference of Engineers and Computer Scientists IMECS 2010 March 17-19, Hong Kong. Vol. 1. 2010. pp. 1-6

[18] Lin CY, Chang YM. Interactive augmented reality using scratch 2.0 to improve physical activities for children with developmental disabilities. Research in Developmental Disabilities. 2015;**37**: 1-8. DOI: 10.1016/j.ridd.2014.10.016

[19] Westerfield G, Mitrovic A, Billinghurst M. Intelligent augmented reality training form mother board assembly. International Journal of Artificial Intelligence in Education. 2015;**25**(1):157-172. DOI: 10.1007/ s40593-014-0032-x

[20] Rabbi I, Ullah S. A survey on augmented reality challenges and tracking. Acta Graphica. 2013;**24**(1- 2):29-46. ISSN: 0353-4707

[21] Kendall KE, Kendall JE. Systems Analysis and Design. USA: Pearson Education, Inc.; 2011. 600p. ISBN: 978-607-32-0577-1

[22] Piekarski W, Thomas B. ARQuake: The outdoor augmented reality gaming system. Communications of the ACM. 2002;**45**(1):36-38. DOI: 10.1145/502269.502291

[23] Wikitude [Internet]. 2019. Available from: https://www.wikitude.com/

[24] ARToolkit [Internet]. Available from: http://www.hitl.washington.edu/ artoolkit/

[25] Eck U, Sandor C. HARP: A framework for visuo-haptic augmented reality. In: 2013 Virtual Reality, IEEE Annual International Symposium, Lake Buena Vista, FL, USA. 2013. pp. 145- 146. DOI: 10.1109/VR.2013.6549404

[26] Chien CH, Chien-Hsu C, Jeng TS. An interactive augmented reality system for learning anatomy structure. In: Proceedings of the International Multi Conference of Engineers and Computer Scientists, IMECS 2010 March 17-19, Hong Kong. Vol. 1. 2010. pp. 1-6

[27] Leventon J, Fleskens L, Claringbould H, Schwilch G, Hessel R. An applied methodology for stakeholder identification in transdisciplinary research. Sustainability Science. 2016;**11**(5):763- 775. DOI: 10.1007/s11625-016-0385-1

[28] Azuma R. Location-based mixed and augmented reality storytelling. In: Barfield W, editor. Fundamentals of Wearable Computers and Augmented Reality (chapter 11). 2nd ed. CRC Press; 2015. pp. 259-276. DOI: 10.1201/ b18703-15

**34**

*Mixed Reality and Three-Dimensional Computer Graphics*

[9] Krueger A, Ruttan V. Development thought and development assistance. In: Aid and Development. Baltimore, USA: The Johns Hopkins University Press;

[10] Marcelo C, Yot C, Mayor C. University teaching with digital technologies. Comunicar.

[11] Laurillard D. Teaching as a Design Science: Building Pedagogical Patterns for Learning and Technology, Routledge. 1st ed. New York: USA; 2013.

272p. ISBN-10: 041580387X

[13] Durall E, Gros B, Maina M, Johnson L, Adams S. Perspectivas tecnológicas: educación superior en Iberoamérica 2012-2017. Austin, Texas: The New Media Consortium; 2012.

[14] Rodríguez RM, Castillo JLM, Lira AL. Diseño de un sistema tutorial inteligente. Apertura. 2013;**5**(1):36-47

[15] Westerfield G, Mitrovic A, Billinghurst M. Intelligent augmented reality training for assembly tasks. In: Lane HC, Yacef K, Mostow J, Pavlik P, editors. Artificial Intelligence in Education (AIED 2013). Lecture Notes in Computer Science. Vol. 7926. Berlin, Heidelberg: Springer; 2013. DOI:

10.1007/978-3-642-39112-5\_55

[16] Sommerauer P, Müller O.

Augmented reality in informal learning environments: A field experiment in a mathematics exhibition. Computers

ISBN: 978-0-9846601-9-3

[12] Kipper G, Rampolla J. Augmented Reality: An Emerging Technologies Guide to AR. USA: Syngress Publishing/ Elsevier; 2012. ISBN: 1597497339

2015;**45**(23):117-124. DOI: 10.3916/

1989. pp. 13-28

C45-2015-12

9781597497336

[1] Naudé W, Szirmai A, Goedhuys M. Innovation and entrepreneurship in developing countries. UNU Policy Brief. 2011;**1**:1-7. ISBN 978-92-808-3093-4

[2] Wong PK, Ho YP, Autio E. Entrepreneurship, innovation and economic growth, evidence from GEM data. Small Business Economics. 2005;**24**(3):335-350. DOI: 10.1007/

[3] Beugelsdijk S, Noorderhaven N. Entrepreneurial attitude and economic growth. A cross-section of 54 regions. The Annals of Regional Science. 2004;**38**:199-218. DOI: 10.1007/

[4] Webber DJ. Policies to stimulate growth: Should we invest in health or education? Applied

Economics. 2002;**34**:1633-1643. DOI:

Phanindra VW. The effect of political regimes and technology on economic

10.1080/00036840110115109

growth. Applied Economics. 2007;**39**(11):1425-1432. DOI: 10.1080/00036840500447906

[6] Crespi G, Zuniga P. Innovation and productivity: Evidence from six Latin American countries. World Development. 2012;**40**(2):273-290. DOI: 10.1016/j.worlddev.2011.07.010

[7] Tomokazu N. Contribution of education and educational equality to economic growth. Applied Economics Letters. 2007;**14**:627-630. DOI: 10.1080/13504850500425857

[8] Moon YJ, Kym HG. A model for the value of intellectual capital. Canadian Journal of Administrative Sciences. 2006;**23**(3):253-269. DOI: 10.1111/ j.1936-4490.2006.tb00630.x

[5] Khurram J, Kirsten W,

s11187-005-2000-1

s00168-004-0192-y

**References**

**37**

**Chapter 3**

**Abstract**

3D visualization

**1. Introduction**

An Interactive VR System for

**Keywords:** anatomy training, virtual reality, 3D interaction, usability,

done by the trainer and to be explained for the students.

ages medical learners to be familiar with VR.

*Djamel Aouam, Nadia Zenati-Henda, Samir Benbelkacem* 

In recent decades, virtual reality (VR) becomes a potential solution to enhance clinical medical (functional reeducation, training, etc.), especially with the growth evolution of technologies form both visualization (e.g., HoloLens, VR in Case, etc.) and 3D gestural interaction (Ray Casting, Free Hand, etc.) point of views. The 3D visualization of the human anatomy could be a serious asset for students in medicine. This new technology could provide a clear and realistic representation of the internal organs of the human body, without having to resort to surgery. 3D organs based-course supports visualization could be a useful tool for students, especially in their first graduate studies, to enhance their perception on human's internal composition. This system is composed of two modules, 3D human's anatomy visualization module and interaction module for organs manipulation. Finally, the system will be tested and evaluated with several subjects.

Several applications of training are arranged in a virtual environment. In these simulated environments, users could be exposed to critical situations that affect directly their security and life. This case is highly common in combat strategy training and surgical operations. Thus, experts in medical field strongly recommend the use of simulators and virtual reality technology for training surgery for future doctors. This solution offers more efficiency for training. Realistic virtual environments have been realized in order to immerse the user in this environment so that he can simulate real situations that can be realized. Medicine today knows technical and technological progress. This evolution makes the training task so complex to be

In recent decades, VR technologies offer a great opportunity in medical training for students and health learns. Also, the vulgarization of new technologies encour-

In this paper, we address the issue of VR-based medical training. In particular, we focus on designing human body anatomy teaching system using VR. The purpose is to facilitate understanding complex anatomy course. The proposed system allows trainers as well as learns to interact with human organs in order to obtain further explanation about the selected organ in a 3D form. Our system provides a VR educational tool with several functionalities: (1) the first consists in displaying

Anatomy Training

*and Chafiaa Hamitouche*

#### **Chapter 3**

## An Interactive VR System for Anatomy Training

*Djamel Aouam, Nadia Zenati-Henda, Samir Benbelkacem and Chafiaa Hamitouche*

#### **Abstract**

In recent decades, virtual reality (VR) becomes a potential solution to enhance clinical medical (functional reeducation, training, etc.), especially with the growth evolution of technologies form both visualization (e.g., HoloLens, VR in Case, etc.) and 3D gestural interaction (Ray Casting, Free Hand, etc.) point of views. The 3D visualization of the human anatomy could be a serious asset for students in medicine. This new technology could provide a clear and realistic representation of the internal organs of the human body, without having to resort to surgery. 3D organs based-course supports visualization could be a useful tool for students, especially in their first graduate studies, to enhance their perception on human's internal composition. This system is composed of two modules, 3D human's anatomy visualization module and interaction module for organs manipulation. Finally, the system will be tested and evaluated with several subjects.

**Keywords:** anatomy training, virtual reality, 3D interaction, usability, 3D visualization

#### **1. Introduction**

Several applications of training are arranged in a virtual environment. In these simulated environments, users could be exposed to critical situations that affect directly their security and life. This case is highly common in combat strategy training and surgical operations. Thus, experts in medical field strongly recommend the use of simulators and virtual reality technology for training surgery for future doctors. This solution offers more efficiency for training. Realistic virtual environments have been realized in order to immerse the user in this environment so that he can simulate real situations that can be realized. Medicine today knows technical and technological progress. This evolution makes the training task so complex to be done by the trainer and to be explained for the students.

In recent decades, VR technologies offer a great opportunity in medical training for students and health learns. Also, the vulgarization of new technologies encourages medical learners to be familiar with VR.

In this paper, we address the issue of VR-based medical training. In particular, we focus on designing human body anatomy teaching system using VR. The purpose is to facilitate understanding complex anatomy course. The proposed system allows trainers as well as learns to interact with human organs in order to obtain further explanation about the selected organ in a 3D form. Our system provides a VR educational tool with several functionalities: (1) the first consists in displaying skeleton, organs and blood network, (2) the second provides a body part explanation selected by the learner using an interactive tool and (3) finally, our system allows decomposing a selected organ to see its internal details.

Through our VR teaching system, we can implement interactive medical training case studies. This is given in more details in this paper. In Section 2, we present the role of VR in medical teaching. Section 3 provides related work on medical learning domain. In Section 4, a conceptual model of our system is described. Section 5 presents the application usage scenario and the sequence of events. Finally, Sections 6 gives the different stages of asset modeling and their integration into the virtual environment by showing some results of the designed environment.

#### **2. Contributions of virtual reality for medical training**

Classical training methods do not always meet current educational objectives as the large amount of data to be transmitted to learners. At this time virtual reality (VR) can be an alternative to the data management problems encountered in previous approaches.

VR offers new solutions to all control and control simulation and communication problems. It is presented as an improvement of classical simulation techniques. The use of virtual reality for training has many advantages over training in real environments such as:

It allows us to carry out tasks without danger: to work in immersion we are in contact with virtual objects that do not risk to hurt us by manipulating several objects (objects of operating room such as the scalpel) [1].

The tolerance that means we have the permission to make and to make mistakes without the security being questioned because the errors are formative (ex: to make errors on a virtual body no risks if one made these errors on a human body who can put his life at risk). In the same context the VR allows us to realize scenarios with realistic sensations in order to put the learner in more realistic situations we can simulate more realistic environments and in rare conditions and impossible to realize them in reality.

Another positive point of the VR is the availability as it provides you with training that does not have time constraints or presential because training in VR does not require a specific time or attendance requirement in a room. Course you can take your course in any place at any desired time.

More to the advantages mentioned above is the cost and the occupied space because the use of the VR occupies a minimal space compared to a model or a classic skeleton, moreover the use of the same equipment for other modules and even of other training which makes training using VR less expensive compared to training using conventional teaching methods.

Immersion in a virtual world enriches learning and enriches this environment by integrating important sensory aspects in many contexts.

#### **3. Related work**

A lot of work has been done in the field of teaching with the help of the VR since the latter has had the attention of the researchers in order to facilitate the task to the trainers and help the future doctors to assimilate the course, thus there are those who have simply made a state of the art on both teaching methods and draw conclusions on the advantages of each and the disadvantages and others have had to propose applications in different medicinal specialties.

**39**

*An Interactive VR System for Anatomy Training DOI: http://dx.doi.org/10.5772/intechopen.91358*

loss of precision.

the system.

any time.

the VR assistance.

teaching of VR in education.

Khwanngern et al. [2] studied a rare case that consists of craniofacial disorder, seen as a rare case of this phenomenon that student doctors may never have a real case of the cutting process and make that in theory what is not beneficial for students, to remedy this have created an application that simulates the human skull in an operating room and thanks to the motion controllers can cut, drill and manipulate the latter. In this work, we focus on an important action during jaw surgery, the process of cutting the mandibular bone (lower jaw). The cut requires great precision and a small mistake can damage the facial nerve, which can lead to paralysis of the facial muscle. The proposed idea is interesting but it does not simulate reality because it simulates the skeleton of a skull but in reality we are confronted with a human body preferably to simulate a human body and proceed to cut, and second point we visualize the helmet controllers which may hinder us during the cut by a

Alfalah et al. [3] conducted a comparative study between traditional medical education methods and that of VR technology, the VR came to solve the shortcomings of the traditional method since the latter is a tool that offers additional means to teach in order to improve the quality of skills and to meet the requirements of modern medical training in order to overcome the difficulties encountered by students and teachers in conveying the message. A comparative study was conducted between the two teaching methods in VR and classical method. The experiment that has been conducted is to test students by offering them a questionnaire on both teaching methods and the VR offers the possibility to display the organ (human heart) in 3D and manipulate it. Does not exist in conventional method, dissect the 3D cardiac model in layers to clarify anatomical relations of different parts, explore the information on each component of the model, explore the features provided in

Huang et al. [4] conducted a student acceptance study of new technologies such as VR. Have set up a learning application in VR by distributing a questionnaire to students when using the application in order to have their opinions, after such an experience have deduced that the percentage of acceptability is high. And these applications allow students to take their courses without being tied to the constraints of time or face to face, we can follow the course of the place we want and at

De Faria et al. [5] seen to the methods used in the formation and teaching of the future doctors is really complex of the methods that are based on conferences or the dissertations in laboratory, to do have composed 3 working groups with different tasks and comparing their perception during different attempts using the two learning methods and comparing the statistics of these, the results reveal satisfactory results using the VR in the teaching that the students better assimilate the course to

Izard et al. [6] have excellent 3D human body anatomical models, and many VR applications that have been designed based on DICOM and Asteion CT scan, by Toshiba Medical Systems, of Complejo Hospitalario. Universitario de Salamanca, following the study protocol of the skull: one in anteroposterior projection and one in lateral position. Applications allow users to move within different parts of the human body using the stereoscopic system and interact to make a decision about which information to display by conducting a study of potential and contribution

Mathur [7] has studied a really important and important case by initiating each

project that the cost, and most of the existing virtual reality systems are really expensive especially when it comes to specialized systems, to remedy it he proposed

a system with an oculus helmet and razer controllers hydra a system.

#### *An Interactive VR System for Anatomy Training DOI: http://dx.doi.org/10.5772/intechopen.91358*

*Mixed Reality and Three-Dimensional Computer Graphics*

allows decomposing a selected organ to see its internal details.

environment by showing some results of the designed environment.

**2. Contributions of virtual reality for medical training**

objects (objects of operating room such as the scalpel) [1].

your course in any place at any desired time.

integrating important sensory aspects in many contexts.

propose applications in different medicinal specialties.

using conventional teaching methods.

**3. Related work**

ous approaches.

environments such as:

skeleton, organs and blood network, (2) the second provides a body part explanation selected by the learner using an interactive tool and (3) finally, our system

Through our VR teaching system, we can implement interactive medical training case studies. This is given in more details in this paper. In Section 2, we present the role of VR in medical teaching. Section 3 provides related work on medical learning domain. In Section 4, a conceptual model of our system is described. Section 5 presents the application usage scenario and the sequence of events. Finally, Sections 6 gives the different stages of asset modeling and their integration into the virtual

Classical training methods do not always meet current educational objectives as the large amount of data to be transmitted to learners. At this time virtual reality (VR) can be an alternative to the data management problems encountered in previ-

VR offers new solutions to all control and control simulation and communication problems. It is presented as an improvement of classical simulation techniques. The use of virtual reality for training has many advantages over training in real

It allows us to carry out tasks without danger: to work in immersion we are in contact with virtual objects that do not risk to hurt us by manipulating several

The tolerance that means we have the permission to make and to make mistakes without the security being questioned because the errors are formative (ex: to make errors on a virtual body no risks if one made these errors on a human body who can put his life at risk). In the same context the VR allows us to realize scenarios with realistic sensations in order to put the learner in more realistic situations we can simulate more realistic environments and in rare conditions and impossible to realize them in reality. Another positive point of the VR is the availability as it provides you with training that does not have time constraints or presential because training in VR does not require a specific time or attendance requirement in a room. Course you can take

More to the advantages mentioned above is the cost and the occupied space because the use of the VR occupies a minimal space compared to a model or a classic skeleton, moreover the use of the same equipment for other modules and even of other training which makes training using VR less expensive compared to training

A lot of work has been done in the field of teaching with the help of the VR since the latter has had the attention of the researchers in order to facilitate the task to the trainers and help the future doctors to assimilate the course, thus there are those who have simply made a state of the art on both teaching methods and draw conclusions on the advantages of each and the disadvantages and others have had to

Immersion in a virtual world enriches learning and enriches this environment by

**38**

Khwanngern et al. [2] studied a rare case that consists of craniofacial disorder, seen as a rare case of this phenomenon that student doctors may never have a real case of the cutting process and make that in theory what is not beneficial for students, to remedy this have created an application that simulates the human skull in an operating room and thanks to the motion controllers can cut, drill and manipulate the latter. In this work, we focus on an important action during jaw surgery, the process of cutting the mandibular bone (lower jaw). The cut requires great precision and a small mistake can damage the facial nerve, which can lead to paralysis of the facial muscle. The proposed idea is interesting but it does not simulate reality because it simulates the skeleton of a skull but in reality we are confronted with a human body preferably to simulate a human body and proceed to cut, and second point we visualize the helmet controllers which may hinder us during the cut by a loss of precision.

Alfalah et al. [3] conducted a comparative study between traditional medical education methods and that of VR technology, the VR came to solve the shortcomings of the traditional method since the latter is a tool that offers additional means to teach in order to improve the quality of skills and to meet the requirements of modern medical training in order to overcome the difficulties encountered by students and teachers in conveying the message. A comparative study was conducted between the two teaching methods in VR and classical method. The experiment that has been conducted is to test students by offering them a questionnaire on both teaching methods and the VR offers the possibility to display the organ (human heart) in 3D and manipulate it. Does not exist in conventional method, dissect the 3D cardiac model in layers to clarify anatomical relations of different parts, explore the information on each component of the model, explore the features provided in the system.

Huang et al. [4] conducted a student acceptance study of new technologies such as VR. Have set up a learning application in VR by distributing a questionnaire to students when using the application in order to have their opinions, after such an experience have deduced that the percentage of acceptability is high. And these applications allow students to take their courses without being tied to the constraints of time or face to face, we can follow the course of the place we want and at any time.

De Faria et al. [5] seen to the methods used in the formation and teaching of the future doctors is really complex of the methods that are based on conferences or the dissertations in laboratory, to do have composed 3 working groups with different tasks and comparing their perception during different attempts using the two learning methods and comparing the statistics of these, the results reveal satisfactory results using the VR in the teaching that the students better assimilate the course to the VR assistance.

Izard et al. [6] have excellent 3D human body anatomical models, and many VR applications that have been designed based on DICOM and Asteion CT scan, by Toshiba Medical Systems, of Complejo Hospitalario. Universitario de Salamanca, following the study protocol of the skull: one in anteroposterior projection and one in lateral position. Applications allow users to move within different parts of the human body using the stereoscopic system and interact to make a decision about which information to display by conducting a study of potential and contribution teaching of VR in education.

Mathur [7] has studied a really important and important case by initiating each project that the cost, and most of the existing virtual reality systems are really expensive especially when it comes to specialized systems, to remedy it he proposed a system with an oculus helmet and razer controllers hydra a system.

#### **4. Conceptual model**

In our case we designed an application by providing interactive educational tools that offer the possibility of interaction in order to make decisions on the explanations that we want to display in the virtual environment, the whole experience is in 3D immersion and we provide interactive educational tools. In the environment we have an avatar of a human body or the user has the ability to manipulate this body using the touch of the HMD helmet. we provide him with a menu that he can configure using the touch or the latter can play on the rotation of the human body and play on the transparencies in order to separate the skeleton alone or just the organs, by targeting an organ or a bone you will have the definition of the latter which appears on a side panel and thanks With the laser keys you can scroll so that you can read the entire text.

Our work consists in the development of an orgVR application which is a teaching platform (application) dedicated to medical students but the advantage of the latter that the application can be used not only not for medical students but also people who are curious and want to deepen their knowledge in the field of human anatomy.

The desktop application orgVR is an educational application on the anatomy of the human being which allows a user equipped with the oculus rift helmet and touch controllers to interact with the human body, the possible interactions are numerous (X-ray, rotation, change of opacity, appearance of documentation, etc.) implemented thanks to c # and xml scripts and shaders that we have developed on organs that we have modeled.

The proposed environment is an immersive and interactive environment, or the user is endowed with an oculus rift helmet and touch or the latter will be immerse in a laboratory and thanks to the touch can use the interaction principles that are implemented (navigation, manipulation and selection).


#### **4.1 Conceptual diagram**

The conceptual diagram represents the main modules of our application. Our product is made up of three main models (rendering module, interaction module and tracking module). Conceptual diagram describes the global functioning of our application. By dissecting this diagram we start with the database is an essential part where the different 3D assets and the scenes are stored in when the application

**41**

**Figure 1.**

*Conceptual diagram.*

*An Interactive VR System for Anatomy Training DOI: http://dx.doi.org/10.5772/intechopen.91358*

triggered (**Figure 1**).

*4.1.1 Tracking module*

*4.1.2 Interaction module*

*4.1.3 Rendering module*

click on the trigger an interaction that occurs.

is launched the game engine loads the assets and the environments from the database as well when using the application, we use this data depending on the event

As we can see on the diagram a user equipped with an HMD helmet (oculus rift) and touch which is immersed in the virtual environment and performs movements this is where the tracking module plays an important role in retrieving the geometrical position of each movement and movement of the user in order to ensure navigation within the environment. The tracking module is composed of two parts seen we have two main objects to track down the HMD helmet and touch them. The aim is to recover different positions of the helmet, while being able to touch them in real time by sending information to the rendering module. The renderer part projects 3D scenes accordingly to the import information, which is collected by the tracking module and interprets each of performed movement (or event) to ensure effective immersion and navigation tasks, as well as interaction with the virtual environment.

Is responsible for retrieving the geometric position of the hands (touch), the head

(HMD helmet) and translating the latter from the real world to the virtual world.

Retrieves the geometric data and interprets them in interactions following an event from a peripheral as in our case by targeting an organ with the touch with a

Is the essential module in virtual reality given the visual context is important in immersive environments. This module allows synchronization of movements and scenes. In addition, this module allows you to load the necessary 3D models and other data such as organs and their definitions in pop-up loaded from the database. The rendered model selects the appropriate 3D models and projects them

through the lenses of the Head Mounted Display (HMD) helmet.

#### *An Interactive VR System for Anatomy Training DOI: http://dx.doi.org/10.5772/intechopen.91358*

is launched the game engine loads the assets and the environments from the database as well when using the application, we use this data depending on the event triggered (**Figure 1**).

As we can see on the diagram a user equipped with an HMD helmet (oculus rift) and touch which is immersed in the virtual environment and performs movements this is where the tracking module plays an important role in retrieving the geometrical position of each movement and movement of the user in order to ensure navigation within the environment. The tracking module is composed of two parts seen we have two main objects to track down the HMD helmet and touch them. The aim is to recover different positions of the helmet, while being able to touch them in real time by sending information to the rendering module. The renderer part projects 3D scenes accordingly to the import information, which is collected by the tracking module and interprets each of performed movement (or event) to ensure effective immersion and navigation tasks, as well as interaction with the virtual environment.

#### *4.1.1 Tracking module*

*Mixed Reality and Three-Dimensional Computer Graphics*

In our case we designed an application by providing interactive educational tools that offer the possibility of interaction in order to make decisions on the explanations that we want to display in the virtual environment, the whole experience is in 3D immersion and we provide interactive educational tools. In the environment we have an avatar of a human body or the user has the ability to manipulate this body using the touch of the HMD helmet. we provide him with a menu that he can configure using the touch or the latter can play on the rotation of the human body and play on the transparencies in order to separate the skeleton alone or just the organs, by targeting an organ or a bone you will have the definition of the latter which appears on a side panel and thanks With the laser keys you can scroll so that

Our work consists in the development of an orgVR application which is a teaching platform (application) dedicated to medical students but the advantage of the latter that the application can be used not only not for medical students but also people who are curious and want to deepen their knowledge in the field of human

The desktop application orgVR is an educational application on the anatomy of the human being which allows a user equipped with the oculus rift helmet and touch controllers to interact with the human body, the possible interactions are numerous (X-ray, rotation, change of opacity, appearance of documentation, etc.) implemented thanks to c # and xml scripts and shaders that we have developed on

The proposed environment is an immersive and interactive environment, or the user is endowed with an oculus rift helmet and touch or the latter will be immerse in a laboratory and thanks to the touch can use the interaction principles that are

1.Navigation: is the ability to move in a virtual environment to make translations while being immersed, the proposed environment is an immersive environment which offers the possibility of navigating in this environment thanks to the translations and the movement sensors which translate between our posi-

2.Selection: the principle of selection is applied when you want to have the definition of an organ or other thanks to the laser of the oculus rift touch it is enough just to target (target) this organ or the part of the body you will have the definition of the latter which appears in pop-up window format on the side and you can scroll in order to be able to read the entire text, thus allowing us to

3.Manipulation: the application has a menu that carries several functions that allow you to manipulate the human body, this menu that allows you to manipu-

The conceptual diagram represents the main modules of our application. Our product is made up of three main models (rendering module, interaction module and tracking module). Conceptual diagram describes the global functioning of our application. By dissecting this diagram we start with the database is an essential part where the different 3D assets and the scenes are stored in when the application

late the body and play on the transparency and rotation functions.

**4. Conceptual model**

you can read the entire text.

organs that we have modeled.

implemented (navigation, manipulation and selection).

tion in the real world and the virtual world.

select the different menu items.

**4.1 Conceptual diagram**

anatomy.

**40**

Is responsible for retrieving the geometric position of the hands (touch), the head (HMD helmet) and translating the latter from the real world to the virtual world.

#### *4.1.2 Interaction module*

Retrieves the geometric data and interprets them in interactions following an event from a peripheral as in our case by targeting an organ with the touch with a click on the trigger an interaction that occurs.

#### *4.1.3 Rendering module*

Is the essential module in virtual reality given the visual context is important in immersive environments. This module allows synchronization of movements and scenes. In addition, this module allows you to load the necessary 3D models and other data such as organs and their definitions in pop-up loaded from the database.

The rendered model selects the appropriate 3D models and projects them through the lenses of the Head Mounted Display (HMD) helmet.

**Figure 1.** *Conceptual diagram.*

#### **5. The orgVR usage scenario**

By launching the application the first step is to wear the oculus rift helmet and put the oculus touch controllers and set up the two movement sensors in order to track our position once everything is in place the user is immersed in the first scene or window where he can choose the gender of the body to manipulate or study (male or female).

Once the choice is made, he will be teleported to another scene or will be immersed with the body he has chosen for study. Thanks to the side menu, manipulate this body and interact on it such as doing rotations in order to rotate the body on itself, playing on the level of opacity of the different layers of the body. By pointing the laser of the touch on an organ, the organ in question lights up if we wish to have a description or the role of the latter, just press the button of the keys with our pushes a window in pop-up format appears using the laser, you can scroll to be able to read the entire message.

If you want a more detailed study of an organ, you just have to aim it with the laser and press the trigger, you will be teleported to another room (scene) or you will be isolated with the selected organ. The same principle as the previous scene the presence of a side menu which allows you to make rotations for the organ or cut the latter, move the cutting plane, so that we can catch this organ with the virtual hands present thanks to the touch, play on opacity which means transparency, so that we can also reload the scene to start again or return to the first scene where there is the human body.

#### **6. Methodology**

In order to reach our objective we need two main parts or configurations the hard configuration consists of adequate material for the development and use of our application, and the software part which allows us to model our environment and the assets used.

To develop our environment we went through two main stages.

#### **6.1 Preparation and integration models**

This part is the longest and most complex part in the work that has been done; this work consists of developing assets. We were not satisfied with the assets that exist on the net; we wanted to offer our own assets with a more realistic and detailed touch. To do this we had to use several modeling utilities.

#### *6.1.1 Modeling (sculpting)*

This is the most delicate step of the implementation because the objective was to create realistic models to improve the realism of the models and the user immersion for this we used the Blender software. The models were sculpted in highpoly (large number of polygons) (**Figures 2** and **3**).

#### *6.1.2 Rotopology*

It is strongly advised not to use highpoly models in 3D scenes because the very high number of polygons negatively affects the performance that is why 3D modelers use a technique called retopology, this modeling technique will use an existing

**43**

**Figure 3.** *Highpoly model.*

**Figure 2.** *Heart modeling.*

*An Interactive VR System for Anatomy Training DOI: http://dx.doi.org/10.5772/intechopen.91358*

3D model to redo (better) its topology. This will consist of coming to magnetize on existing surfaces, new points, edges and faces. Thus, the extrusions and transformations of the new topology will perfectly follow the faces of the model object, after the application of this technique the model will have a lower number of

We used instant mesh for this step (**Figures 4** and **5**).

*6.1.3 UV mapping and UV unwrapping*

polygons that will improve performance without reducing the details on the models.

A UV map is the flat representation of the surface of a 3D model used to easily

wrap textures. The process of creating a UV card is called UV unpacking.

#### *An Interactive VR System for Anatomy Training DOI: http://dx.doi.org/10.5772/intechopen.91358*

**Figure 2.** *Heart modeling.*

*Mixed Reality and Three-Dimensional Computer Graphics*

By launching the application the first step is to wear the oculus rift helmet and put the oculus touch controllers and set up the two movement sensors in order to track our position once everything is in place the user is immersed in the first scene or window where he can choose the gender of the body to manipulate or study

Once the choice is made, he will be teleported to another scene or will be immersed with the body he has chosen for study. Thanks to the side menu, manipulate this body and interact on it such as doing rotations in order to rotate the body on itself, playing on the level of opacity of the different layers of the body. By pointing the laser of the touch on an organ, the organ in question lights up if we wish to have a description or the role of the latter, just press the button of the keys with our pushes a window in pop-up format appears using the laser, you can scroll to be able

If you want a more detailed study of an organ, you just have to aim it with the laser and press the trigger, you will be teleported to another room (scene) or you will be isolated with the selected organ. The same principle as the previous scene the presence of a side menu which allows you to make rotations for the organ or cut the latter, move the cutting plane, so that we can catch this organ with the virtual hands present thanks to the touch, play on opacity which means transparency, so that we can also reload the scene to start again or return to the first scene where

In order to reach our objective we need two main parts or configurations the hard configuration consists of adequate material for the development and use of our application, and the software part which allows us to model our environment

This part is the longest and most complex part in the work that has been done; this work consists of developing assets. We were not satisfied with the assets that exist on the net; we wanted to offer our own assets with a more realistic and detailed

This is the most delicate step of the implementation because the objective was to create realistic models to improve the realism of the models and the user immersion for this we used the Blender software. The models were sculpted in highpoly (large

It is strongly advised not to use highpoly models in 3D scenes because the very high number of polygons negatively affects the performance that is why 3D modelers use a technique called retopology, this modeling technique will use an existing

To develop our environment we went through two main stages.

touch. To do this we had to use several modeling utilities.

**5. The orgVR usage scenario**

(male or female).

to read the entire message.

there is the human body.

**6. Methodology**

and the assets used.

*6.1.1 Modeling (sculpting)*

*6.1.2 Rotopology*

**6.1 Preparation and integration models**

number of polygons) (**Figures 2** and **3**).

**42**

3D model to redo (better) its topology. This will consist of coming to magnetize on existing surfaces, new points, edges and faces. Thus, the extrusions and transformations of the new topology will perfectly follow the faces of the model object, after the application of this technique the model will have a lower number of polygons that will improve performance without reducing the details on the models. We used instant mesh for this step (**Figures 4** and **5**).

#### *6.1.3 UV mapping and UV unwrapping*

A UV map is the flat representation of the surface of a 3D model used to easily wrap textures. The process of creating a UV card is called UV unpacking.

**Figure 4.** *Retopology stage.*

**Figure 5.** *Lowpoly model.*

Once the polygonal mesh has been created, the next step is to "decompress" it in a UV map. Now, to give life to the mesh and to give it a more realistic aspect However, there is no 3D texture, because they are always based on a 2D image. This is where UV mapping comes in, because it is the process of converting your 3D mesh into 2D information so that a 2D texture can be wrapped around it (**Figure 6**).

#### *6.1.4 Texturing and painting*

In this last step, we apply a texture on our 3D models with Substance Painter. A texture is an image representing a surface offering the possibility of simulating the appearance of this when we paste it on a 3D object. Textures are particularly used generally in video games, and textures offer an aspect close to reality. After this step the models are ready to be exported to unity (**Figure 7**).

**45**

**6.2 Results and test**

**Figure 7.** *Texturing stage.*

**Figure 6.** *UV mapping stage.*

*6.2.1 Pointing laser and interaction*

After the design of our assets, the role came to integrate them into our virtual environment. The biggest challenge after the integration of assets is the implementation of interaction methods in order to offer better visualization and perception to the user. Within our virtual environment we have implemented and used several methods of interaction with objects, among these shader method or raycasting to enrich the functionality of our environment for better interaction more detailed because the functionality of shader allows us to perform (cuts, grabbing, etc.).

The biggest difficulty we encountered was to implement the pointing on the organs while managing the interaction with the UI menus, this pushed us to take

*An Interactive VR System for Anatomy Training DOI: http://dx.doi.org/10.5772/intechopen.91358*

*An Interactive VR System for Anatomy Training DOI: http://dx.doi.org/10.5772/intechopen.91358*

**Figure 6.** *UV mapping stage.*

*Mixed Reality and Three-Dimensional Computer Graphics*

Once the polygonal mesh has been created, the next step is to "decompress" it in a UV map. Now, to give life to the mesh and to give it a more realistic aspect However, there is no 3D texture, because they are always based on a 2D image. This is where UV mapping comes in, because it is the process of converting your 3D mesh into 2D information so that a 2D texture can be wrapped around it (**Figure 6**).

In this last step, we apply a texture on our 3D models with Substance Painter. A texture is an image representing a surface offering the possibility of simulating the appearance of this when we paste it on a 3D object. Textures are particularly used generally in video games, and textures offer an aspect close to reality. After this step

the models are ready to be exported to unity (**Figure 7**).

**44**

**Figure 5.** *Lowpoly model.*

**Figure 4.** *Retopology stage.*

*6.1.4 Texturing and painting*

**Figure 7.** *Texturing stage.*

#### **6.2 Results and test**

After the design of our assets, the role came to integrate them into our virtual environment. The biggest challenge after the integration of assets is the implementation of interaction methods in order to offer better visualization and perception to the user. Within our virtual environment we have implemented and used several methods of interaction with objects, among these shader method or raycasting to enrich the functionality of our environment for better interaction more detailed because the functionality of shader allows us to perform (cuts, grabbing, etc.).

#### *6.2.1 Pointing laser and interaction*

The biggest difficulty we encountered was to implement the pointing on the organs while managing the interaction with the UI menus, this pushed us to take a very particular interest in the system of ray jets (raycasting) of unity in effect in a 3D scene in virtual reality the user cannot use the mouse and that is why we have implemented a laser play the role of a pointer allowing interactions with the virtual environment. As the following figure shows us the result obtained by implementing the two raycasting scripts so that our laser is operational (**Figure 8**).

#### *6.2.2 Shader managing the object cut*

The cup was the most important interaction we wanted to achieve, it was also the most difficult to achieve here we explain as we implemented it.

**Figure 8.** *Interaction laser.*

**47**

**Figure 11.** *Heart grabbing.*

**Figure 10.** *Transparency effects.*

*An Interactive VR System for Anatomy Training DOI: http://dx.doi.org/10.5772/intechopen.91358*

from being drawn on the screen (**Figure 9**).

First of all we had to define how we wanted the user to carry out the cut from a methodical point of view (by what he will apply the cut) so we deemed it more realistic and interactive that the user can do the cut by passing a glass plane through the body, the latter can either be manipulated by catching it directly with the hand or by means of sliders on the UI menu. There the cutting effect is implemented thanks to a "OnePlaneBSP" shader which is applied to the organ, this shader calculates the position of the vertexes of the organ by adding to that of the cutting plane to identify the vertexes of the organs which are at—above the plan to prevent them

**Figure 9.** *Cut from the heart by the plane.*

*An Interactive VR System for Anatomy Training DOI: http://dx.doi.org/10.5772/intechopen.91358*

*Mixed Reality and Three-Dimensional Computer Graphics*

*6.2.2 Shader managing the object cut*

a very particular interest in the system of ray jets (raycasting) of unity in effect in a 3D scene in virtual reality the user cannot use the mouse and that is why we have implemented a laser play the role of a pointer allowing interactions with the virtual environment. As the following figure shows us the result obtained by implementing

The cup was the most important interaction we wanted to achieve, it was also

the two raycasting scripts so that our laser is operational (**Figure 8**).

the most difficult to achieve here we explain as we implemented it.

**46**

**Figure 9.**

**Figure 8.** *Interaction laser.*

*Cut from the heart by the plane.*

First of all we had to define how we wanted the user to carry out the cut from a methodical point of view (by what he will apply the cut) so we deemed it more realistic and interactive that the user can do the cut by passing a glass plane through the body, the latter can either be manipulated by catching it directly with the hand or by means of sliders on the UI menu. There the cutting effect is implemented thanks to a "OnePlaneBSP" shader which is applied to the organ, this shader calculates the position of the vertexes of the organ by adding to that of the cutting plane to identify the vertexes of the organs which are at—above the plan to prevent them from being drawn on the screen (**Figure 9**).

**Figure 10.** *Transparency effects.*

**Figure 11.** *Heart grabbing.*

**Figure 12.** *A student using the application.*

#### *6.2.3 Shader managing opacity and X-ray effect*

The change of opacity and the X-ray effect are two purely visual effects, we implemented the first thanks to the shader "TransparentDiffuse ZWrite" which is a simple transparent shader and to the script "Opacity Controller" which applies the effect on the organ. The second effect is achieved with a simple "XrayEffect" shader which is a simple transparency shader but which applies a white color to all the vertices to give an X-ray effect (**Figure 10**).

#### *6.2.4 Object grabbing implementation*

In this last part we will explain how we implemented grabbing objects. This interaction was very easily implemented using the oculus integration kit in unity, we added hands to the user avatar and a simple collision circle around the objects made it possible to catch objects intuitively as in the real environment (**Figures 11** and **12**).

#### **7. Conclusion**

We managed, through this work, to create a medical educational training application in virtual reality which works under Windows and which allows users to visualize the human anatomy and interact with organs in numerous ways (section, X-ray, grabbing, rotation, etc.).

To design an effective virtual environment, usable and useful for training, we proposed a design methodology taking into account the educational objectives and technological capabilities.

This methodology allowed us to specify an optimal EV for the educational training of young medical students in anatomy. This EV does not reproduce reality as perfectly as possible. Indeed, realism is not the most effective solution in all cases of training. Sometimes, it is interesting to walk away from it to show hidden realities or to understand abstract concepts. The many potentials of virtual reality allow us to offer different levels of realism and abstraction.

**49**

*An Interactive VR System for Anatomy Training DOI: http://dx.doi.org/10.5772/intechopen.91358*

choice.

reality headsets.

In the proposed environment, the oculus rift virtual reality headset was used as hardware, a headset that allows total immersion in a virtual environment and ensures navigation and interaction in the latter. The choice of equipment is based on the availability of the helmet in our laboratory because this choice is not the ideal

Because the use of the oculus rift headset is not the best solution seen after the use of the latter we noted shortcomings such as the problem of latency and the feeling of dizziness for use for a certain duration and since the proposed application is an educational application the student or learner is called to use the application for

To remedy this kind of problem HTC Vive pro a headset which offers a better resolution of rendering (display) and reduce the problem of latencies observed within the oculus rift and also it has remedied the major problem of vertigo especially for first use of the headset or long-term use suffered by users of other virtual

But we see that the two types of helmets suffer from a common problem which consists in freedom of movement and risk of accidents since they are more wired helmets in order to be able to use these two helmets. PC gamer with graphics power which is really expensive and users cannot afford such hardware due to the excessive prices of these machines. In order for these applications to be used by the general public, the oculus firm has launched the oculus quest headset, which is an autonomous wireless headset which does not require a PC. The latter offers similar functionality to previous headsets with a lower price and a degree of freedom. Higher compared to the oculus rift and HTC Vive. Certainly the oculus quest has solved some shortcomings encountered on previous headsets also does not escape shortcomings despite these advantages. The quest's major problems is the reduced quality of the rendering compared to the rift headset and the HTC vivid second

long periods of time plus the reduced quality of the image quality.

point is the storage space the latter does not allow to make an extension.

etc.) in order to guarantee navigation in the 3D environment.

network methods in local network or remote network.

The last type of helmet that VR helmets for smart mobiles is the most popular among the general public. This model is ideal in terms of production cost and deployment seen is within the reach of the large number of users except that the latter is really limited in terms of interaction because this solution just allows visualization and offers no alternative in order to manipulate 3D objects. In addition, the smart mobile must be equipped with adequate sensors (gyroscope, accelerometer,

The developed environment is an application that runs in local host by way of

• Extend the application to support collaborative work. Access of several users at the same time or on time to the same shared environment by implementing

• Implementation of the application on several interoperable virtual platforms.

perspectives. Several improvements are envisaged for the developed system:

#### *An Interactive VR System for Anatomy Training DOI: http://dx.doi.org/10.5772/intechopen.91358*

*Mixed Reality and Three-Dimensional Computer Graphics*

*6.2.3 Shader managing opacity and X-ray effect*

vertices to give an X-ray effect (**Figure 10**).

*6.2.4 Object grabbing implementation*

X-ray, grabbing, rotation, etc.).

to offer different levels of realism and abstraction.

technological capabilities.

**7. Conclusion**

**Figure 12.**

*A student using the application.*

The change of opacity and the X-ray effect are two purely visual effects, we implemented the first thanks to the shader "TransparentDiffuse ZWrite" which is a simple transparent shader and to the script "Opacity Controller" which applies the effect on the organ. The second effect is achieved with a simple "XrayEffect" shader which is a simple transparency shader but which applies a white color to all the

In this last part we will explain how we implemented grabbing objects. This interaction was very easily implemented using the oculus integration kit in unity, we added hands to the user avatar and a simple collision circle around the objects made it possible to catch objects intuitively as in the real environment (**Figures 11** and **12**).

We managed, through this work, to create a medical educational training application in virtual reality which works under Windows and which allows users to visualize the human anatomy and interact with organs in numerous ways (section,

To design an effective virtual environment, usable and useful for training, we proposed a design methodology taking into account the educational objectives and

This methodology allowed us to specify an optimal EV for the educational training of young medical students in anatomy. This EV does not reproduce reality as perfectly as possible. Indeed, realism is not the most effective solution in all cases of training. Sometimes, it is interesting to walk away from it to show hidden realities or to understand abstract concepts. The many potentials of virtual reality allow us

**48**

In the proposed environment, the oculus rift virtual reality headset was used as hardware, a headset that allows total immersion in a virtual environment and ensures navigation and interaction in the latter. The choice of equipment is based on the availability of the helmet in our laboratory because this choice is not the ideal choice.

Because the use of the oculus rift headset is not the best solution seen after the use of the latter we noted shortcomings such as the problem of latency and the feeling of dizziness for use for a certain duration and since the proposed application is an educational application the student or learner is called to use the application for long periods of time plus the reduced quality of the image quality.

To remedy this kind of problem HTC Vive pro a headset which offers a better resolution of rendering (display) and reduce the problem of latencies observed within the oculus rift and also it has remedied the major problem of vertigo especially for first use of the headset or long-term use suffered by users of other virtual reality headsets.

But we see that the two types of helmets suffer from a common problem which consists in freedom of movement and risk of accidents since they are more wired helmets in order to be able to use these two helmets. PC gamer with graphics power which is really expensive and users cannot afford such hardware due to the excessive prices of these machines. In order for these applications to be used by the general public, the oculus firm has launched the oculus quest headset, which is an autonomous wireless headset which does not require a PC. The latter offers similar functionality to previous headsets with a lower price and a degree of freedom. Higher compared to the oculus rift and HTC Vive. Certainly the oculus quest has solved some shortcomings encountered on previous headsets also does not escape shortcomings despite these advantages. The quest's major problems is the reduced quality of the rendering compared to the rift headset and the HTC vivid second point is the storage space the latter does not allow to make an extension.

The last type of helmet that VR helmets for smart mobiles is the most popular among the general public. This model is ideal in terms of production cost and deployment seen is within the reach of the large number of users except that the latter is really limited in terms of interaction because this solution just allows visualization and offers no alternative in order to manipulate 3D objects. In addition, the smart mobile must be equipped with adequate sensors (gyroscope, accelerometer, etc.) in order to guarantee navigation in the 3D environment.

The developed environment is an application that runs in local host by way of perspectives. Several improvements are envisaged for the developed system:


#### **Author details**

Djamel Aouam1 \*, Nadia Zenati-Henda1 , Samir Benbelkacem1 and Chafiaa Hamitouche2

1 Center for Development of Advanced Technologies (CDTA), Algiers, Algeria

2 Image and Information Processing Department, Institut Mines-Telecom (IMT) Atlantique (Ex: Telecom Bretagne), Technopôle Brest-Iroise, France

\*Address all correspondence to: daouam@cdta.dz

© 2020 The Author(s). Licensee IntechOpen. This chapter is distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/ by/3.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

**51**

*An Interactive VR System for Anatomy Training DOI: http://dx.doi.org/10.5772/intechopen.91358*

> human anatomy. Journal of Medical Systems. 2017;**41**(5):76. DOI: 10.1007/

[7] Mathur AS. Low cost virtual reality for medical training. In: 2015 IEEE Virtual Reality (VR). Arles, France: IEEE; 2015. pp. 345-346. Available from: https://ieeexplore.ieee.org/abstract/

s10916-017-0723-6. Available from: https://link.springer.com/ article/10.1007/s10916-017-0723-6

document/7223437

[1] Schild J, Misztal S, Roth B, et al. Applying multi-user virtual reality to collaborative medical training. In: 2018 IEEE Conference on Virtual Reality and 3D User Interfaces (VR). Reutlingen, Germany: IEEE; 2018. pp. 775-776. Available from: https://ieeexplore.ieee. org/abstract/document/8446160

[2] Khwanngern K, Tiangtae N, Natwichai J, et al. Jaw surgery simulation in virtual reality for medical training. In: International Conference on Network-Based Information Systems. Cham:

Springer; 2019. pp. 475-483. Available from: https://link.springer.com/

chapter/10.1007/978-3-030-29029-0\_45

[3] Alfalah SFM et al. A comparative study between a virtual reality heart anatomy system and traditional medical teaching modalities. Virtual Reality. 2019;**23**(3):229-234. DOI: 10.1007/s10055-018-0359-y. Available from: https://link.springer.com/ article/10.1007/s10055-018-0359-y

[4] Huang H-M, Liaw S-S, Lai C-M. Exploring learner acceptance of the use of virtual reality in medical education: A case study of desktop and projectionbased display systems. Interactive Learning Environments. 2016;**24**(1):3- 19. DOI: 10.1080/10494820.2013.817436.

Available from: https://www.

820.2013.817436

article-p1105.xml

tandfonline.com/doi/abs/10.1080/10494

[5] De Faria JWV, Teixeira MJ, de Moura Sousa Júnior L, et al. Virtual and stereoscopic anatomy: When virtual reality meets medical education. Journal of Neurosurgery. 2016;**125**(5):1105- 1111. DOI: 10.3171/2015.8.JNS141563. Available from: https://thejns.org/ view/journals/j-neurosurg/125/5/

[6] Izard SG, Méndez JAJ, Palomera PR. Virtual reality educational tool for

**References**

*An Interactive VR System for Anatomy Training DOI: http://dx.doi.org/10.5772/intechopen.91358*

#### **References**

*Mixed Reality and Three-Dimensional Computer Graphics*

**50**

**Author details**

Djamel Aouam1

Chafiaa Hamitouche2

\*, Nadia Zenati-Henda1

\*Address all correspondence to: daouam@cdta.dz

provided the original work is properly cited.

, Samir Benbelkacem1

1 Center for Development of Advanced Technologies (CDTA), Algiers, Algeria

Atlantique (Ex: Telecom Bretagne), Technopôle Brest-Iroise, France

2 Image and Information Processing Department, Institut Mines-Telecom (IMT)

© 2020 The Author(s). Licensee IntechOpen. This chapter is distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/ by/3.0), which permits unrestricted use, distribution, and reproduction in any medium,

and

[1] Schild J, Misztal S, Roth B, et al. Applying multi-user virtual reality to collaborative medical training. In: 2018 IEEE Conference on Virtual Reality and 3D User Interfaces (VR). Reutlingen, Germany: IEEE; 2018. pp. 775-776. Available from: https://ieeexplore.ieee. org/abstract/document/8446160

[2] Khwanngern K, Tiangtae N, Natwichai J, et al. Jaw surgery simulation in virtual reality for medical training. In: International Conference on Network-Based Information Systems. Cham: Springer; 2019. pp. 475-483. Available from: https://link.springer.com/ chapter/10.1007/978-3-030-29029-0\_45

[3] Alfalah SFM et al. A comparative study between a virtual reality heart anatomy system and traditional medical teaching modalities. Virtual Reality. 2019;**23**(3):229-234. DOI: 10.1007/s10055-018-0359-y. Available from: https://link.springer.com/ article/10.1007/s10055-018-0359-y

[4] Huang H-M, Liaw S-S, Lai C-M. Exploring learner acceptance of the use of virtual reality in medical education: A case study of desktop and projectionbased display systems. Interactive Learning Environments. 2016;**24**(1):3- 19. DOI: 10.1080/10494820.2013.817436. Available from: https://www. tandfonline.com/doi/abs/10.1080/10494 820.2013.817436

[5] De Faria JWV, Teixeira MJ, de Moura Sousa Júnior L, et al. Virtual and stereoscopic anatomy: When virtual reality meets medical education. Journal of Neurosurgery. 2016;**125**(5):1105- 1111. DOI: 10.3171/2015.8.JNS141563. Available from: https://thejns.org/ view/journals/j-neurosurg/125/5/ article-p1105.xml

[6] Izard SG, Méndez JAJ, Palomera PR. Virtual reality educational tool for

human anatomy. Journal of Medical Systems. 2017;**41**(5):76. DOI: 10.1007/ s10916-017-0723-6. Available from: https://link.springer.com/ article/10.1007/s10916-017-0723-6

[7] Mathur AS. Low cost virtual reality for medical training. In: 2015 IEEE Virtual Reality (VR). Arles, France: IEEE; 2015. pp. 345-346. Available from: https://ieeexplore.ieee.org/abstract/ document/7223437

**53**

**Chapter 4**

**Abstract**

Learning by Augmented Reality:

Because the use of augmented reality (AR) is increasing, it is important to study its possibilities within both formal and informal learning contexts. We clustered 146 sixth graders using AR at a science center based on their reasoning, motivation, and science learning results using the self-organizing maps method (SOM) to identify AR-using subgroups. The aim was to consider reasons why the AR method could be of more beneficial for some students than others. The clustering results complemented earlier findings on AR gains in learning, as an unexpected response to intervention was discovered using this nonlinear analysis. The previous results had indicated that after the AR experience, science test results generally improved and particularly among students with the lowest achievement. The SOM-clustering results showed a majority group of boys, especially those interested in science learning both at school and at the science center using AR. Despite low school achievement, their high motivation led to good science learning results. The prior results, according to which girls closed the science knowledge gap between boys after using AR, became more relative, as two girldominated subgroups were identified. The reasons for the results were considered on the basis of motivation, multimedia learning theory, and concept formation theories.

**Keywords:** science learning, augmented reality, informal learning environment,

Augmented reality (AR) technology offers possibilities to demonstrate complex phenomena in a novel way. At its best, the novelty of AR makes it an effective servant [1], but on the other hand, it sometimes has been shown to increase cognitive load due to bad practical usability and also because the tasks used are too complicated [2]. The AR advantages can be theoretically understood through multimedia learning theory, which explains how blending virtual contents into the real world can support brain functioning in cognition and learning [3]. The theory stresses the use of pictures in learning instead of just words [4]. Afandi et al. [5] elaborate on the theoretical points further as applied to AR by replacing pictures with real objects and words with symbols and virtual text. From the sociological perspective, the AR method can likely enhance the fulfillment of the essential idea of big principles and ideas of science education [6] and advance understanding about science even for people who otherwise would remain outsiders. AR can be viewed as a great example of tools which, for their

part, pave the way for attaining the twenty-first-century competences [7, 8].

In the Finnish national core curriculum [9], the twenty-first-century competences are called *transversal competences*, which include seven areas. The area with

SOM-clustering, self-determination theory

**1. Introduction**

Cluster Analysis Approach

*Helena Thuneberg and Hannu S. Salmi*

#### **Chapter 4**

## Learning by Augmented Reality: Cluster Analysis Approach

*Helena Thuneberg and Hannu S. Salmi*

#### **Abstract**

Because the use of augmented reality (AR) is increasing, it is important to study its possibilities within both formal and informal learning contexts. We clustered 146 sixth graders using AR at a science center based on their reasoning, motivation, and science learning results using the self-organizing maps method (SOM) to identify AR-using subgroups. The aim was to consider reasons why the AR method could be of more beneficial for some students than others. The clustering results complemented earlier findings on AR gains in learning, as an unexpected response to intervention was discovered using this nonlinear analysis. The previous results had indicated that after the AR experience, science test results generally improved and particularly among students with the lowest achievement. The SOM-clustering results showed a majority group of boys, especially those interested in science learning both at school and at the science center using AR. Despite low school achievement, their high motivation led to good science learning results. The prior results, according to which girls closed the science knowledge gap between boys after using AR, became more relative, as two girldominated subgroups were identified. The reasons for the results were considered on the basis of motivation, multimedia learning theory, and concept formation theories.

**Keywords:** science learning, augmented reality, informal learning environment, SOM-clustering, self-determination theory

#### **1. Introduction**

Augmented reality (AR) technology offers possibilities to demonstrate complex phenomena in a novel way. At its best, the novelty of AR makes it an effective servant [1], but on the other hand, it sometimes has been shown to increase cognitive load due to bad practical usability and also because the tasks used are too complicated [2]. The AR advantages can be theoretically understood through multimedia learning theory, which explains how blending virtual contents into the real world can support brain functioning in cognition and learning [3]. The theory stresses the use of pictures in learning instead of just words [4]. Afandi et al. [5] elaborate on the theoretical points further as applied to AR by replacing pictures with real objects and words with symbols and virtual text. From the sociological perspective, the AR method can likely enhance the fulfillment of the essential idea of big principles and ideas of science education [6] and advance understanding about science even for people who otherwise would remain outsiders. AR can be viewed as a great example of tools which, for their part, pave the way for attaining the twenty-first-century competences [7, 8].

In the Finnish national core curriculum [9], the twenty-first-century competences are called *transversal competences*, which include seven areas. The area with which the connection with AR is most direct is the *information and communication technology* (*ICT*) competence*.* It relates to understanding the principles and essential concepts of ICT and involves creative manipulation of ICT applications and through it communicating thoughts and ideas. In addition to being as an essential skill itself*,* it is asserted to support the *thinking and learning-to-learn* competence and to be a sub-skill of the *multiliteracy* competence.

Augmented reality as a support method for learning has previously mainly been studied in a classroom context [10], although positive results have also been reported in informal learning environments [11]. Most of the previous studies have been qualitative, but based on a meta-analysis of 87 articles [3] and in another 64 analyses [12], a medium effect of AR on learning has been identified, usually in cross-sectional designs. The goal of these variable-oriented studies has been to show general tendencies, usability, advantages, and disadvantages of AR [13], and the compared variables have been knowledge tests, school achievement, motivation, collaboration, and other variables related in learning [14].

The most important article relating to the present book article is "Making invisible observable, learning abstract phenomena in an abstract way" [15]. In that study AR was applied in a quasi-experimental pre- and posttest design in an informal learning environment, i.e., in the science center. As expected based on the research literature, positive learning results were obtained, although without a controlled design, using test and control group interpretation of the results must be cautious. The effect found between the pre- and posttest in knowledge gain was of medium size (partial *η* 2 = .10), which the result was further analyzed using a structural equation path-model controlling motivational and cognitive variables. Pupils' prior interest in science and readiness to take responsibility for setting their own goals have previously been found to enhance learning in an informal learning environment [16].

Our study showed two routes which seemed to enhance the post-knowledge scores. The stronger one was going via preknowledge and the other less effective through attitudes and motivation. Knowledge before the exhibition had a direct medium prediction effect on the post-results, but a positive attitude towards science center education had a direct effect, as well. School achievement, gender and autonomy experience, positive attitude, and situation motivation towards the science center education all predicted indirectly some of the knowledge results after the intervention. Based on the results, the AR technology experience was shown to be beneficial, particularly for the lowest-achieving group. Also, girls took advantage of AR and had as high knowledge scores as boys in the posttest.

Now, in the present book chapter, the aim is to complement and cross-validate our previously reported results by using methodological triangulation by a personcentered approach elaborated from the Finnish version [17]. The aim is to elaborate the general tendency found, the *general rule*, that the low in-school achieving students and girls would specially benefit from the use of AR. In order to identify the deviation from the general tendency, the possible subgroups, and the potentially interesting nonlinear connections, the students are clustered based on the results of the learning, cognitive, and motivational test results.

The research questions are:

1.What kind of subgroups and results complementing the previous study can be identified by clustering the AR-using students based on cognitive reasoning, motivation, science interest, and knowledge learning test results?

**55**

**Figure 1.**

*Learning by Augmented Reality: Cluster Analysis Approach*

mini wing exhibit, and (5) rolling double cone.

applied as a pretest (**Table 1**).

*AR equipment showing the functions of the airplane wing with air flow.*

The participating 146 pupils were 11–13 years old, and 51% (n = 75) of them were girls. They were from seven schools from the Helsinki Metropolitan Area

The pupils visited a typical science center exhibition, which included five AR technology-supported exhibits. They were (1) the Doppler phenomenon, (2) Boltzmann's molecule movement, (3) the Young experiment, (4) the airplane

The context of the study was formed as an open learning environment consisting of AR equipment (**Figure 1**), hands-on exhibit (**Figure 2**), experimenting with small-scale real objects (**Figure 3**), and testing AR demonstrations (**Figure 4**). The photos above are showing just one case related to flying. Also all the four other topics were taught based on the same pedagogical principle: *the mixed reality as an open learning environment* was formed by bridging the gap between virtual AR technology, real hands-on objects, and interactive learning by science center

1.*Deci-Ryan motivation*. A self-determination theory (SDT)-based SRQ-A questionnaire was used to examine relatively stable academic motivation (32 test items, Likert scale 1–4, α *=* .92). The SRQ-A test includes a formula [18] based on which the relative autonomy index (RAI) was calculated. It describes the overall autonomy level experienced by the pupil. It was only

*DOI: http://dx.doi.org/10.5772/intechopen.91252*

**2. Method**

in Finland.

**2.2 Context**

exhibition objects.

**2.3 Instruments**

**2.1 Participants**

2.How are boys and girls and students achieving differently in the school environment represented in the subgroups?

### **2. Method**

*Mixed Reality and Three-Dimensional Computer Graphics*

sub-skill of the *multiliteracy* competence.

of medium size (partial *η*

environment [16].

collaboration, and other variables related in learning [14].

2

of AR and had as high knowledge scores as boys in the posttest.

the learning, cognitive, and motivational test results.

environment represented in the subgroups?

The research questions are:

which the connection with AR is most direct is the *information and communication technology* (*ICT*) competence*.* It relates to understanding the principles and essential concepts of ICT and involves creative manipulation of ICT applications and through it communicating thoughts and ideas. In addition to being as an essential skill itself*,* it is asserted to support the *thinking and learning-to-learn* competence and to be a

Augmented reality as a support method for learning has previously mainly been studied in a classroom context [10], although positive results have also been reported in informal learning environments [11]. Most of the previous studies have been qualitative, but based on a meta-analysis of 87 articles [3] and in another 64 analyses [12], a medium effect of AR on learning has been identified, usually in cross-sectional designs. The goal of these variable-oriented studies has been to show general tendencies, usability, advantages, and disadvantages of AR [13], and the compared variables have been knowledge tests, school achievement, motivation,

The most important article relating to the present book article is "Making invisible observable, learning abstract phenomena in an abstract way" [15]. In that study AR was applied in a quasi-experimental pre- and posttest design in an informal learning environment, i.e., in the science center. As expected based on the research literature, positive learning results were obtained, although without a controlled design, using test and control group interpretation of the results must be cautious. The effect found between the pre- and posttest in knowledge gain was

structural equation path-model controlling motivational and cognitive variables. Pupils' prior interest in science and readiness to take responsibility for setting their own goals have previously been found to enhance learning in an informal learning

Our study showed two routes which seemed to enhance the post-knowledge scores. The stronger one was going via preknowledge and the other less effective through attitudes and motivation. Knowledge before the exhibition had a direct medium prediction effect on the post-results, but a positive attitude towards science center education had a direct effect, as well. School achievement, gender and autonomy experience, positive attitude, and situation motivation towards the science center education all predicted indirectly some of the knowledge results after the intervention. Based on the results, the AR technology experience was shown to be beneficial, particularly for the lowest-achieving group. Also, girls took advantage

Now, in the present book chapter, the aim is to complement and cross-validate our previously reported results by using methodological triangulation by a personcentered approach elaborated from the Finnish version [17]. The aim is to elaborate the general tendency found, the *general rule*, that the low in-school achieving students and girls would specially benefit from the use of AR. In order to identify the deviation from the general tendency, the possible subgroups, and the potentially interesting nonlinear connections, the students are clustered based on the results of

1.What kind of subgroups and results complementing the previous study can be identified by clustering the AR-using students based on cognitive reasoning,

motivation, science interest, and knowledge learning test results?

2.How are boys and girls and students achieving differently in the school

= .10), which the result was further analyzed using a

**54**

#### **2.1 Participants**

The participating 146 pupils were 11–13 years old, and 51% (n = 75) of them were girls. They were from seven schools from the Helsinki Metropolitan Area in Finland.

#### **2.2 Context**

The pupils visited a typical science center exhibition, which included five AR technology-supported exhibits. They were (1) the Doppler phenomenon, (2) Boltzmann's molecule movement, (3) the Young experiment, (4) the airplane mini wing exhibit, and (5) rolling double cone.

The context of the study was formed as an open learning environment consisting of AR equipment (**Figure 1**), hands-on exhibit (**Figure 2**), experimenting with small-scale real objects (**Figure 3**), and testing AR demonstrations (**Figure 4**).

The photos above are showing just one case related to flying. Also all the four other topics were taught based on the same pedagogical principle: *the mixed reality as an open learning environment* was formed by bridging the gap between virtual AR technology, real hands-on objects, and interactive learning by science center exhibition objects.

#### **2.3 Instruments**

1.*Deci-Ryan motivation*. A self-determination theory (SDT)-based SRQ-A questionnaire was used to examine relatively stable academic motivation (32 test items, Likert scale 1–4, α *=* .92). The SRQ-A test includes a formula [18] based on which the relative autonomy index (RAI) was calculated. It describes the overall autonomy level experienced by the pupil. It was only applied as a pretest (**Table 1**).

**Figure 2.** *Pupils testing the real hands-on wind tunnel.*

**Figure 3.** *Pupils building and testing a small-scale airplane.*

**57**

*Learning by Augmented Reality: Cluster Analysis Approach*

2.*Situation motivation test*. This provides information about how attractive pupils found the exhibition (14 items, Likert scale 1–5, α *=* .91). It was applied only as

3.*Learning context*. School vs. augmented reality in a science center. Contextspecific interest was measured in the school context vs. the informal science center context by applying the semantic differential method [19] (14 pairs of adjective alternatives, Likert scale 1–5, pretest α *=* .81, posttest α *=* .88).

4.*Raven test*. The Raven standard progressive matrices [20] were used to test

Each of these groups contains 12 tasks (α = .79, 60 items).

(pretest, α = .72; posttest, α = .77, 31 items).

cluster membership is the information obtained [23–25].

early language learning [29] and in sociological research [30].

2 + 3 quartiles, 4 highest quartile).

**2.4 Statistical analysis method**

cognitive visual reasoning. The test contains 60 items divided into 5 sets (A–E).

5.*The knowledge tests*. These consisted of 31 items related to the content areas of the school curriculum of the science subjects, and these contents were combined with the AR solutions in the science exhibition. The questions were piloted 2 months before the actual preknowledge test. The post-knowledge tests were conducted 1 week after the science exhibition visit. In the test the pupils were asked to assess whether they thought the knowledge statements presented were *correct* or *incorrect* or whether they were *uncertain* about them

The background variables were gender and school achievement, for which we used four school grades (physics, chemistry, mathematics, and mother tongue). The students were grouped into three groups based on achievement (1 lowest quartile,

The pupils were clustered on the basis of their scores by applying the selforganizing maps method (SOM) [21], a neural network model [22] which is based on unsupervised learning of fuzzy logic. Compared, for example, to K-means clustering, the advantage is that within the SOM cluster, the nearer one pupil ends up to another, the closer the likeness between them is. In the K-means cluster, the neighborhood does not count, and the pupils are merely listed in the cluster, and

The SOM method has been widely applied internationally, especially in biotechnology, economy, and technical industries [26]. In social sciences the applications are rare, although it has been shown to have promising possibilities in educational and learning research [27, 28], in the area of psychology, for example, relating in

Using the SOM method, the goal was to identify subgroups particularly benefitting or non-benefitting from AR. The data of the cluster were transferred to SPPS 25.

*DOI: http://dx.doi.org/10.5772/intechopen.91252*

a posttest.

**Table 1.** *Design of the study.*

**Figure 4.** *Testing AR plane in informal settings.*


#### **Table 1.**

*Mixed Reality and Three-Dimensional Computer Graphics*

**56**

**Figure 4.**

**Figure 2.**

**Figure 3.**

*Pupils testing the real hands-on wind tunnel.*

*Pupils building and testing a small-scale airplane.*

*Testing AR plane in informal settings.*

*Design of the study.*


The background variables were gender and school achievement, for which we used four school grades (physics, chemistry, mathematics, and mother tongue). The students were grouped into three groups based on achievement (1 lowest quartile, 2 + 3 quartiles, 4 highest quartile).

#### **2.4 Statistical analysis method**

The pupils were clustered on the basis of their scores by applying the selforganizing maps method (SOM) [21], a neural network model [22] which is based on unsupervised learning of fuzzy logic. Compared, for example, to K-means clustering, the advantage is that within the SOM cluster, the nearer one pupil ends up to another, the closer the likeness between them is. In the K-means cluster, the neighborhood does not count, and the pupils are merely listed in the cluster, and cluster membership is the information obtained [23–25].

The SOM method has been widely applied internationally, especially in biotechnology, economy, and technical industries [26]. In social sciences the applications are rare, although it has been shown to have promising possibilities in educational and learning research [27, 28], in the area of psychology, for example, relating in early language learning [29] and in sociological research [30].

Using the SOM method, the goal was to identify subgroups particularly benefitting or non-benefitting from AR. The data of the cluster were transferred to SPPS 25. The statistical significance of the difference between the theoretically expected and observed number of students in each cluster was tested using the chi-square test, and the adjusted residuals (criterion: absolute value ≥2) were used to pinpoint the overor underrepresentation in each cross-tabulated cell. The differences between the clusters (dummy variables: each cluster vs. all others) were compared using one-way analysis of variance. The change between the pre- and posttests results was analyzed using the general linear modeling method (GLM repeated measures) and its effect size by the partial η<sup>2</sup> -coefficient (interpretation: >.01 small; >.06 middle; >.14 large).

#### **3. Results**

Using the SOM method, five clusters were obtained. When the clusters were crosstabulated against gender and school achievement groups, the result was that boys and girls were not represented equally as expected in the clusters (χ<sup>2</sup> = 18.63, *p <* .001) and that was also true with the achievement groups (χ<sup>2</sup> = 25.38, *p <* .001). The statistical descriptives are presented in **Table 2** and the 95% confidence plots of the knowledge test results (correct, incorrect, uncertain) in the two time points by cluster in **Figure 5**.

In order to illuminate how the science knowledge test results looked like before and after the science center visit intervention, and the change, we present the 95% confidence plots of the knowledge test results. They are divided into correct, incorrect, uncertain answers in the two time points by cluster in **Figure 5**.

#### **3.1 Cluster 1 (n = 26; 18%): motivated, low school achievers, boy majority**

Significantly more boys (adjusted residual = 3.2) and lowest in-school achievers (adjusted residual = 2.2) than expected. When dummy variable cluster 1 was compared to all the others, in cluster 1, cognitive reasoning was lower; situation motivation and interest in science both in the school and in the exhibition were higher. In the pretest there were more incorrect answers. The correct answers increased (η<sup>2</sup> = .43) and incorrect ones decreased (η<sup>2</sup> = .44) after the science exhibition and AR-assisted method.

#### **3.2 Cluster 2 (n = 29; 20%): high achievers**

Cluster 2 was not gendered; there were an equal number of boys and girls. However, there was a strong representation of in-school highly achieving


**59**

(η<sup>2</sup>

*Learning by Augmented Reality: Cluster Analysis Approach*

students (adjusted residual = 3.8). In comparison, in this cluster the pupils had the highest cognitive reasoning scores, higher situation motivation, and interest in science learning at school than the others. They had more pre- and postknowledge correct answers and fewer incorrect answers at both time points. The

In cluster 3, the different school achievers were equally represented, but gender played a role: girls were significantly more represented than boys (adjusted residual = 2.6). In the dummy comparison, students in this cluster had a higher autonomy experience and situation motivation than others. It was notable that in the post-knowledge test after the exhibition, their incorrect answers were found to

= .24) and were the highest scores of all clusters but uncertainty

**3.3 Cluster 3 (n = 32; 22%): motivated but non-learners, girl majority**

*Knowledge test T0 (pre-test) and T1 (post-test), correct, incorrect and uncertain answers.*

**3.4 Cluster 4 (n = 35; 24%): non-motivated by exhibition but learning**

In this cluster, the background variables made no difference: there were an equal number of boys and girls and different type of school achievers. The dummy comparison revealed that they had the lowest autonomy experience (RAI) and situation motivation. They were less interested in science learning in school and especially in the exhibition than others. In the preknowledge test, they had more incorrect but fewer uncertain answers than others. After the exhibition

= .39), incorrect (η<sup>2</sup>

= .16), and the uncertain answers

= .23), and uncertain ones

*DOI: http://dx.doi.org/10.5772/intechopen.91252*

correct answers increased significantly (η<sup>2</sup>

= .26).

= .26).

the correct answers increased (η<sup>2</sup>

= .18) decreased.

decreased (η<sup>2</sup>

**Figure 5.**

even increase (η<sup>2</sup>

decreased (η<sup>2</sup>

**Table 2.** *The statistical descriptives.* *Learning by Augmented Reality: Cluster Analysis Approach DOI: http://dx.doi.org/10.5772/intechopen.91252*

**Figure 5.**

*Mixed Reality and Three-Dimensional Computer Graphics*

size by the partial η<sup>2</sup>

**3. Results**

The statistical significance of the difference between the theoretically expected and observed number of students in each cluster was tested using the chi-square test, and the adjusted residuals (criterion: absolute value ≥2) were used to pinpoint the overor underrepresentation in each cross-tabulated cell. The differences between the clusters (dummy variables: each cluster vs. all others) were compared using one-way analysis of variance. The change between the pre- and posttests results was analyzed using the general linear modeling method (GLM repeated measures) and its effect

Using the SOM method, five clusters were obtained. When the clusters were crosstabulated against gender and school achievement groups, the result was that boys and

descriptives are presented in **Table 2** and the 95% confidence plots of the knowledge test results (correct, incorrect, uncertain) in the two time points by cluster in **Figure 5**. In order to illuminate how the science knowledge test results looked like before and after the science center visit intervention, and the change, we present the 95% confidence plots of the knowledge test results. They are divided into correct, incor-

girls were not represented equally as expected in the clusters (χ<sup>2</sup>

rect, uncertain answers in the two time points by cluster in **Figure 5**.

**3.1 Cluster 1 (n = 26; 18%): motivated, low school achievers, boy majority**

Significantly more boys (adjusted residual = 3.2) and lowest in-school achievers (adjusted residual = 2.2) than expected. When dummy variable cluster 1 was compared to all the others, in cluster 1, cognitive reasoning was lower; situation motivation and interest in science both in the school and in the exhibition were higher. In the pretest there were more incorrect answers. The correct answers increased

Cluster 2 was not gendered; there were an equal number of boys and girls. However, there was a strong representation of in-school highly achieving

that was also true with the achievement groups (χ<sup>2</sup>

= .43) and incorrect ones decreased (η<sup>2</sup>

**3.2 Cluster 2 (n = 29; 20%): high achievers**


= 18.63, *p <* .001) and

= 25.38, *p <* .001). The statistical

= .44) after the science exhibition and

**58**

**Table 2.**

*The statistical descriptives.*

(η<sup>2</sup>

AR-assisted method.

*Knowledge test T0 (pre-test) and T1 (post-test), correct, incorrect and uncertain answers.*

students (adjusted residual = 3.8). In comparison, in this cluster the pupils had the highest cognitive reasoning scores, higher situation motivation, and interest in science learning at school than the others. They had more pre- and postknowledge correct answers and fewer incorrect answers at both time points. The correct answers increased significantly (η<sup>2</sup> = .16), and the uncertain answers decreased (η<sup>2</sup> = .26).

#### **3.3 Cluster 3 (n = 32; 22%): motivated but non-learners, girl majority**

In cluster 3, the different school achievers were equally represented, but gender played a role: girls were significantly more represented than boys (adjusted residual = 2.6). In the dummy comparison, students in this cluster had a higher autonomy experience and situation motivation than others. It was notable that in the post-knowledge test after the exhibition, their incorrect answers were found to even increase (η<sup>2</sup> = .24) and were the highest scores of all clusters but uncertainty decreased (η<sup>2</sup> = .26).

#### **3.4 Cluster 4 (n = 35; 24%): non-motivated by exhibition but learning**

In this cluster, the background variables made no difference: there were an equal number of boys and girls and different type of school achievers. The dummy comparison revealed that they had the lowest autonomy experience (RAI) and situation motivation. They were less interested in science learning in school and especially in the exhibition than others. In the preknowledge test, they had more incorrect but fewer uncertain answers than others. After the exhibition the correct answers increased (η<sup>2</sup> = .39), incorrect (η<sup>2</sup> = .23), and uncertain ones (η<sup>2</sup> = .18) decreased.

#### **3.5 Cluster 5 (n = 24; 16%): non-motivated, girl majority**

In this cluster there was an overrepresentation of girls (adjusted residual = 2.1), a little less so than in cluster 3. In the dummy comparison cluster 5 versus others, these students were found to have a lower autonomy experience, lower situation motivation, and interest in science learning. In both time points, they had fewer correct and incorrect answers in the knowledge tests than others. However, the most striking difference was that both in the pre- and post-situation, they had more uncertain answers than others. In this cluster the knowledge test results did not change significantly from the pretest to the posttest.

#### **4. Discussion**

Previous studies [14] have shown that AR usually enhances learning in an effective way, despite some results that show that the effect seems to fade in the long run [11]. Similar results were found in our earlier study [15]: in general AR improved learning results, and this was supported by interest and situation motivation, especially among boys. In a girls' group, in turn, experienced autonomy was an important explainer of science learning results. Importantly, the lowest school achievement group especially was found to gain from the use of AR technology. By clustering the same data, we could detect deviations from these general tendencies, i.e., relating to the role of reasoning, school achievement, motivation, and science interest in knowledge learning.

#### **4.1 Low in-school achieving pupils can also have high learning results when they act in an informal learning environment**

The cluster analysis revealed that good school achievement is not the only factor leading to motivation in science and to good learning results. Although cluster 2 with an equal number of boys and girls was formed, in which everything (reasoning, school grades, interest in science at school, motivation, and knowledge results) was optimal, we also found that prior school achievement and cognitive reasoning were not totally deterministic factors, and one of the most encouraging result of the clustering study was that a deviant group (cluster 1) from the general rule could be identified. In this cluster, boys were the majority, who, in spite of low school achievement, were especially interested in science learning both at school and at the science center using AR. This led to good science learning results subsequent to the exhibition. Even though they had more incorrect knowledge answers in the beginning than the others, their interest clearly supported learning, as correct answers increased after the AR experience. This result deviates from most meta-results obtained from informal learning contexts [31, 32].

It seems that the idea of using AR-supported learning was successful in its goal to introduce abstract phenomena in a concrete way. Learning by doing and personal experimentation made a crucial link between theory and practice possible, and deduction and induction were combined in a pedagogically effective way. That the pupils were interested showed up both in that they found the science-centered environment attractive (situation motivation) and in that they were deeply engaged in the theme contents (intrinsic motivation) as they learned the scientific knowledge.

#### **4.2 Experienced lack of choice, low autonomy, and low motivation may unexpectedly lead to high results in tests—but at high costs**

The somewhat puzzling and unexpected finding relates to the role of autonomy and situation motivation in learning. The general rule, which shows that a low

**61**

*Learning by Augmented Reality: Cluster Analysis Approach*

autonomy experience and situation motivation usually lead to less learning and worse learning outcomes, did not materialize in the fourth cluster. In that group the students learned, although they were less autonomous than all the others, less motivated by the AR-assisted learning situation, and in general less interested in science learning overall. The number of correct knowledge answers increased, while incorrect and uncertain answers decreased. This result opens up an interesting

According to the SDT theory [33], nonautonomous motivation is based on the avoidance of sanctions, on hope of rewards, or on experienced pressure. It has been found that the learning results of externally motivated and, thus, less autonomy experienced students might remain more superficial and short-term than those of their more intrinsically motivated classmates. Previous research further indicates that if a student acts because she or he feels anxiety and pressure, psychological energy is consumed for defense of self, and less energy remains for learning new things [34, 35]. Our earlier path-model results, according to which the girls closed the science knowledge gap between boys after using AR, became more relative, as two girl-

**4.3 Experienced autonomy and motivation usually—but not always—**

In cluster 3, with the girl majority, the pupils experienced more autonomy than others and were also motivated by the AR experience, but in both pre- and postsituation had incorrect conceptualizations—they even increased after the science center visit—in contrast with the boy-majority group just described. When, on the other hand, the low rate of uncertainty of knowing found already in the pretest further (unrealistically) decreased after the science center visit, one might wonder what the reason was: why did the experienced autonomy and being attracted by AR use in the exhibition not lead to self-correction of the wrong ideas and learning? Theoretically, AR usually seems to be a practical tool to support the Kolb learning cycle, which starts by being exposed to a concrete experience, following by reflective observation and further transferring to abstract conceptualization and finally

Perhaps the phase of reflective observation remained superficial, and/or the abstract conceptualization phase failed [36]. In the research on learning concepts [37, 38], it has been observed that if one builds concepts on incomplete information and only partially, these misconceptions are later very hard to change—most probably even in interesting contexts such as in the AR-connected science exhibition. The big challenge for science teaching is, thus, to identify possible misconceptions and partially formed concepts and to make a necessary return to the earliest phase

**4.4 Low autonomy, no interest in AR, much confusion showing as uncertainty,** 

In cluster 5, the pupils, with a girl majority, were highly uncertain both before and after the AR experience. While the students in this group were less autonomous and motivated by the exhibition situation, their lowest knowledge results were only as one could expect. Theoretically [36], interpreting the result of this cluster, these pupils (i.e., especially because there were girls in this group) have not been successful in creating meaning from their AR- and science-centered learning experience, and therefore, the whole experiential learning cycle process has been interrupted. The worst conclusion is this failure may be only one in a series of previous failures.

**and lowest learning results—as expected, but how to intervene?**

*DOI: http://dx.doi.org/10.5772/intechopen.91252*

dominated subgroups were identified.

leading to active experimentation [36].

of the conceptualization process.

**correlate with learning**

theoretical point.

#### *Learning by Augmented Reality: Cluster Analysis Approach DOI: http://dx.doi.org/10.5772/intechopen.91252*

*Mixed Reality and Three-Dimensional Computer Graphics*

change significantly from the pretest to the posttest.

**4. Discussion**

**3.5 Cluster 5 (n = 24; 16%): non-motivated, girl majority**

In this cluster there was an overrepresentation of girls (adjusted residual = 2.1), a little less so than in cluster 3. In the dummy comparison cluster 5 versus others, these students were found to have a lower autonomy experience, lower situation motivation, and interest in science learning. In both time points, they had fewer correct and incorrect answers in the knowledge tests than others. However, the most striking difference was that both in the pre- and post-situation, they had more uncertain answers than others. In this cluster the knowledge test results did not

Previous studies [14] have shown that AR usually enhances learning in an effective way, despite some results that show that the effect seems to fade in the long run [11]. Similar results were found in our earlier study [15]: in general AR improved learning results, and this was supported by interest and situation motivation, especially among boys. In a girls' group, in turn, experienced autonomy was an important explainer of science learning results. Importantly, the lowest school achievement group especially was found to gain from the use of AR technology. By clustering the same data, we could detect deviations from these general tendencies, i.e., relating to the role of reasoning, school achievement, motivation, and science interest in knowledge learning.

The cluster analysis revealed that good school achievement is not the only factor leading to motivation in science and to good learning results. Although cluster 2 with an equal number of boys and girls was formed, in which everything (reasoning, school grades, interest in science at school, motivation, and knowledge results) was optimal, we also found that prior school achievement and cognitive reasoning were not totally deterministic factors, and one of the most encouraging result of the clustering study was that a deviant group (cluster 1) from the general rule could be identified. In this cluster, boys were the majority, who, in spite of low school achievement, were especially interested in science learning both at school and at the science center using AR. This led to good science learning results subsequent to the exhibition. Even though they had more incorrect knowledge answers in the beginning than the others, their interest clearly supported learning, as correct answers increased after the AR experience. This result deviates from most meta-results

It seems that the idea of using AR-supported learning was successful in its goal to introduce abstract phenomena in a concrete way. Learning by doing and personal experimentation made a crucial link between theory and practice possible, and deduction and induction were combined in a pedagogically effective way. That the pupils were interested showed up both in that they found the science-centered environment attractive (situation motivation) and in that they were deeply engaged in the theme contents (intrinsic motivation) as they learned the scientific knowledge.

The somewhat puzzling and unexpected finding relates to the role of autonomy

**4.2 Experienced lack of choice, low autonomy, and low motivation may unexpectedly lead to high results in tests—but at high costs**

and situation motivation in learning. The general rule, which shows that a low

**4.1 Low in-school achieving pupils can also have high learning results** 

**when they act in an informal learning environment**

obtained from informal learning contexts [31, 32].

**60**

autonomy experience and situation motivation usually lead to less learning and worse learning outcomes, did not materialize in the fourth cluster. In that group the students learned, although they were less autonomous than all the others, less motivated by the AR-assisted learning situation, and in general less interested in science learning overall. The number of correct knowledge answers increased, while incorrect and uncertain answers decreased. This result opens up an interesting theoretical point.

According to the SDT theory [33], nonautonomous motivation is based on the avoidance of sanctions, on hope of rewards, or on experienced pressure. It has been found that the learning results of externally motivated and, thus, less autonomy experienced students might remain more superficial and short-term than those of their more intrinsically motivated classmates. Previous research further indicates that if a student acts because she or he feels anxiety and pressure, psychological energy is consumed for defense of self, and less energy remains for learning new things [34, 35].

Our earlier path-model results, according to which the girls closed the science knowledge gap between boys after using AR, became more relative, as two girldominated subgroups were identified.

#### **4.3 Experienced autonomy and motivation usually—but not always correlate with learning**

In cluster 3, with the girl majority, the pupils experienced more autonomy than others and were also motivated by the AR experience, but in both pre- and postsituation had incorrect conceptualizations—they even increased after the science center visit—in contrast with the boy-majority group just described. When, on the other hand, the low rate of uncertainty of knowing found already in the pretest further (unrealistically) decreased after the science center visit, one might wonder what the reason was: why did the experienced autonomy and being attracted by AR use in the exhibition not lead to self-correction of the wrong ideas and learning? Theoretically, AR usually seems to be a practical tool to support the Kolb learning cycle, which starts by being exposed to a concrete experience, following by reflective observation and further transferring to abstract conceptualization and finally leading to active experimentation [36].

Perhaps the phase of reflective observation remained superficial, and/or the abstract conceptualization phase failed [36]. In the research on learning concepts [37, 38], it has been observed that if one builds concepts on incomplete information and only partially, these misconceptions are later very hard to change—most probably even in interesting contexts such as in the AR-connected science exhibition. The big challenge for science teaching is, thus, to identify possible misconceptions and partially formed concepts and to make a necessary return to the earliest phase of the conceptualization process.

#### **4.4 Low autonomy, no interest in AR, much confusion showing as uncertainty, and lowest learning results—as expected, but how to intervene?**

In cluster 5, the pupils, with a girl majority, were highly uncertain both before and after the AR experience. While the students in this group were less autonomous and motivated by the exhibition situation, their lowest knowledge results were only as one could expect. Theoretically [36], interpreting the result of this cluster, these pupils (i.e., especially because there were girls in this group) have not been successful in creating meaning from their AR- and science-centered learning experience, and therefore, the whole experiential learning cycle process has been interrupted. The worst conclusion is this failure may be only one in a series of previous failures.

#### **5. Concluding remarks**

These results are of interest especially because they add information about different types of pupils who use AR for learning. Some of them clearly somewhat unexpectedly seem to take advantage of the use of AR as a pedagogical tool more than many of the others, even though the preconditions of low school achievement and reasoning skills would have predicted less optimistic AR-learning gains. On the other hand, the results also illuminate situations in which pedagogical intervention would be advisable: an essential notion was that unrealistically, high expectations might arise for AR usage based on superficial observation about the seemingly motivated pupils. In those cases, pupils might be autonomously and eagerly engaging in the task, but in reality, it remains unnoticed that they do not necessarily have enough guidance to make correct conclusions based on their AR experiences.

The most vulnerable group of pupils from the perspective of the *big principles and ideas of science education* [6] was, however, those on whom the AR experience showed to have no or little effect. Despite the fact that they were fulfilling the expectation based on low motivation and interest proposed by the multimedia learning theory [3, 5], one could have hoped that AR as a novel method would have been more successful. The result is most worrying: what could be done differently, if the novelty effect, a new method to change abstract concepts to more concrete ones, and a fresh, untraditional learning environment were not working? At least we could design the AR-learning situation even more carefully [39].

Perhaps one way would be to assure that AR technology really is as easy as possible to use and that there is enough guidance at the start and support available during the whole process. Simultaneously, it should be assured that the goals are realistic and simple enough. In addition to that, Cheng and Tsai [2] suggest care should be taken that the numerous possibilities of AR do not become too overwhelming and lead to cognitive overload, as short-term memory resources are limited.

Most of the research around mixed reality has focused on technical and practical issues or an evaluation of usability. Educational research on learning aspects has already produced some useful meta-articles on the strengths and weaknesses related to augmented reality [14, 40]. Digitalization has doubtless changed our everyday life—also related to learning. However, the research-based evidence, especially related to the latest, brand new technologies also tend to give false promises or create ambiguous future visions [41] (Säntti & Saari, 2017). Accurate results are needed not only for more meaningful learning but also to create cost-effective solutions and to avoid wrong, often expensive investments.

This study related to learning outcomes has produced some new evidence based on cluster analysis supplementing the earlier findings. Making an invisible phenomenon observable is clearly the strongest input of AR—especially when it offers an opportunity to learn an abstract and difficult topic in a concrete, observable way. Mixed reality combines visible elements with an already existing realistic environment and makes it more understandable. Augmented reality is applicable and works best when teaching real and restricted learning contents, and thus, it is really a challenging superficial "phenomenon-based education."

Although one has to keep in mind that the process is not straightforward, there are many intervening factors relating to motivation and factors of self; encouraging possibilities emerge through the AR method. One of the most promising results is that this type of intervention and learning method really can support low-achiever students to close the gap on other students. However, it also provides opportunities and challenges for high-achieving students. It also seems to give valuable opportunities for bridging the gap between formal education and informal learning.

**63**

**Author details**

Helena Thuneberg and Hannu S. Salmi\*

provided the original work is properly cited.

\*Address all correspondence to: hannu.salmi@helsinki.fi

© 2020 The Author(s). Licensee IntechOpen. This chapter is distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/ by/3.0), which permits unrestricted use, distribution, and reproduction in any medium,

University of Helsinki, Finland

*Learning by Augmented Reality: Cluster Analysis Approach*

*DOI: http://dx.doi.org/10.5772/intechopen.91252*

*Learning by Augmented Reality: Cluster Analysis Approach DOI: http://dx.doi.org/10.5772/intechopen.91252*

*Mixed Reality and Three-Dimensional Computer Graphics*

conclusions based on their AR experiences.

These results are of interest especially because they add information about different types of pupils who use AR for learning. Some of them clearly somewhat unexpectedly seem to take advantage of the use of AR as a pedagogical tool more than many of the others, even though the preconditions of low school achievement and reasoning skills would have predicted less optimistic AR-learning gains. On the other hand, the results also illuminate situations in which pedagogical intervention would be advisable: an essential notion was that unrealistically, high expectations might arise for AR usage based on superficial observation about the seemingly motivated pupils. In those cases, pupils might be autonomously and eagerly engaging in the task, but in reality, it remains unnoticed that they do not necessarily have enough guidance to make correct

The most vulnerable group of pupils from the perspective of the *big principles and ideas of science education* [6] was, however, those on whom the AR experience showed to have no or little effect. Despite the fact that they were fulfilling the expectation based on low motivation and interest proposed by the multimedia learning theory [3, 5], one could have hoped that AR as a novel method would have been more successful. The result is most worrying: what could be done differently, if the novelty effect, a new method to change abstract concepts to more concrete ones, and a fresh, untraditional learning environment were not working? At least

Perhaps one way would be to assure that AR technology really is as easy as possible to use and that there is enough guidance at the start and support available during the whole process. Simultaneously, it should be assured that the goals are realistic and simple enough. In addition to that, Cheng and Tsai [2] suggest care should be taken that the numerous possibilities of AR do not become too overwhelming and

Most of the research around mixed reality has focused on technical and practical issues or an evaluation of usability. Educational research on learning aspects has already produced some useful meta-articles on the strengths and weaknesses related to augmented reality [14, 40]. Digitalization has doubtless changed our everyday life—also related to learning. However, the research-based evidence, especially related to the latest, brand new technologies also tend to give false promises or create ambiguous future visions [41] (Säntti & Saari, 2017). Accurate results are needed not only for more meaningful learning but also to create cost-effective solu-

This study related to learning outcomes has produced some new evidence based on cluster analysis supplementing the earlier findings. Making an invisible phenomenon observable is clearly the strongest input of AR—especially when it offers an opportunity to learn an abstract and difficult topic in a concrete, observable way. Mixed reality combines visible elements with an already existing realistic environment and makes it more understandable. Augmented reality is applicable and works best when teaching real and restricted learning contents, and thus, it is really a

Although one has to keep in mind that the process is not straightforward, there are many intervening factors relating to motivation and factors of self; encouraging possibilities emerge through the AR method. One of the most promising results is that this type of intervention and learning method really can support low-achiever students to close the gap on other students. However, it also provides opportunities and challenges for high-achieving students. It also seems to give valuable opportunities for bridging the gap between formal education and informal learning.

we could design the AR-learning situation even more carefully [39].

lead to cognitive overload, as short-term memory resources are limited.

tions and to avoid wrong, often expensive investments.

challenging superficial "phenomenon-based education."

**5. Concluding remarks**

**62**

#### **Author details**

Helena Thuneberg and Hannu S. Salmi\* University of Helsinki, Finland

\*Address all correspondence to: hannu.salmi@helsinki.fi

© 2020 The Author(s). Licensee IntechOpen. This chapter is distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/ by/3.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

### **References**

[1] Zhang J, Sung Y-T, Hou H-T, Chang K-E. The development and evaluation of an augmented reality-based armillary sphere for astronomical observation instruction. Computers & Education. 2014;**73**:178-188

[2] Cheng K-H, Tsai C-C. Affordances of augmented reality in science learning: Suggestions for future research. Journal of Science Education and Technology. 2013;**22**(4):449-462

[3] Santos MEC, Chen A, Taketomi T, Yamamoto G, Miyazaki J, Kato H. Augmented reality learning experiences: Survey of prototype design and evaluation. IEEE Transactions on Learning Technologies. 2014;**7**(1):38-56

[4] Akçayır M, Akçayır G. Advantages and challenges associated with augmented reality for education: A systematic review of the literature. Educational Research Review. 2017;**20**:1-11

[5] Afandi B, Kustiawan I, Herman N. Exploration of the augmented reality model in learning. Jornal of Physics Conference Series. 2019;**1375**:012082. DOI: 10.1088/1742-6596/1375/1/012082

[6] Harlen W. In: Harlen W, Bell D, Devés R, Dyasi H, de la Garza DF, Léna P, Millar R, Reiss M, Rowell P, Yu W, editors. Principles and Big Ideas of Science Education. Association for Science Education. Gosport, Hants: Ashford Colour Press Ltd., 2010. Available from: www.ase.org.uk/ bigideas. ISBN 978 0 86357 4 313

[7] Deakin R, Stringer C, Ren K. Learning to learn. In: International Perspectives from Theory and Practice. London: Routledge; 2014; 354 p. DOI: 10.4324/9780203078044

[8] Voogt J, Pareja Roblin N. A comparative analysis of international frameworks for 21st century competences: Implications for national curriculum policies. Journal of Curriculum Studies. 2014;**44**(3):299-321. Available from: https://pdfs.semanticscholar.org/831e/ f07db8a589b91a508e0118f6f3c2a1f18c19. pdf

[9] Finnish National Board of Education. FNCC: Finnish National Core Curriculum. Helsinki: Opetushallitus; 2019

[10] Salmi H, Kaasinen A, Kallunki V. Towards an open learning environment via augmented reality (AR): Visualising the invisible in science centres and schools for teacher education. Procedia: Social and Behavioral Sciences. 2012;**45**:284-295. DOI: 10.1016/j. sbspro.2012.06.565

[11] Sommerauer P, Müller O. Augmented reality in informal learning environments: A field experiment in a mathematics exhibition. Computer Education. 2014;**79**:59-68. Available from: https://core.ac.uk/download/ pdf/143481015.pdf

[12] Garzón J, Acevedo J. Meta-analysis of the impact of augmented reality on students' learning gains. Educational Research Review. 2019;**27**:244-260. DOI: 10.1016/j.edurev.2019.04.001

[13] Yilmaz Z, Batdi V. A meta-analytic and thematic comparative analysis of the integration of augmented reality into education. Education in Science. 2016;**41**(188):273-289. DOI: 10.15390/ EB.2016.6707

[14] Bacca J, Baldris S, Fabregat R, Graf S, Kinshuk K. Augmented reality trends in education: A systematic review of research and applications. Education Technology & Society. 2014;**17**(4):133-149

**65**

2017]

*Learning by Augmented Reality: Cluster Analysis Approach*

Scandinavica, Mathematics, Computing

and Management in Engineering Serie. Espoo: The Finnish Academy of

[25] Oja E, Mannila H. From data to knowledge, chapter 7. Self-Organizing Map. Available from: http://www.cis. hut.fi/Opinnot/T61.2010/Luennot/

[26] Laboratory of computer and information science—Bibliography. Bibliography of SOM-Papers. 2017. Available from: http://www.cis.hut.fi/ research/refs/ [Accessed: 02 June 2017]

[27] Thuneberg H, Hotulainen R. Contributions of data mining for psycho-educational research: What self-organizing maps tell us about the well-being of gifted learners. High Ability Studies. 2006;**17**(1):87-100

[28] Thuneberg H. Is a majority enough? Psychological wellbeing and its relation to academic and prosocial motivation, self-regulation and achievement at school. In: Research Report 281. Helsinki: Department of Applied Sciences of Education, Faculty of Behavioural Sciences, Helsinki

[29] Mayor J, Plunkett K. A neurocomputational account of taxonomic responding and fast mapping in early word learning. Psychological Review. 2010;**117**(1):1-31. DOI: 10.1037/

[30] Carboni O, Russu P. Assessing regional wellbeing in Italy: An application of Malmquist–DEA and self-organizing map neural clustering. Social Indicators Research.

[31] Rennie L. Learning science outside of school. In: Lederman N, Abell S, editors. Handbook of Research on Science Education. Vol. II. London:

Routledge; 2014. pp. 120-144

University Press; 2007

2015;**122**(3):677-700

a00181300

Technology; 1997. p. 82

JohdantoWhite.pdf

*DOI: http://dx.doi.org/10.5772/intechopen.91252*

Vainikainen M. Making the invisible observable by augmented reality in informal science education context. International Journal of Science Education. 2017;**7**(3):253-268. DOI: 10.1080/21548455.2016.1254358

[15] Salmi H, Thuneberg H,

[16] Renninger K. Individual interest and its implications for understanding intrinsic motivation. In: Sansone C, Harakiewicz J, editors. Intrinsic Motivation: Controversies and New Directions. San Diego, CA:

Academic Press. pp. 373-404

Summary

havaittavaksi—klusterianalyysi abstraktin ilmiön oppimisesta konkreettisesti lisäketodellisuuden (augmented reality) avulla. Psykologia. 2018;**53**(2-3):203-217. With English

[18] Ryan R, Connell J. Perceived locus of causality and internalisation: Examining reasons for acting in two domains. Journal of Personality and Social Psychology. 1989;**57**:749-761

[19] Osgood CE. Semantic differential technique in the comparative study of cultures. American Anthropologist.

[20] Raven J, Raven JC, Court J. Manual for Raven's Progressive Matrices and Vocabulary Scales. Oxford: OPP; 2003

[21] Kohonen T. Self-Organizing Maps.

[22] Kohonen T. The self-organizing map (SOM). 2005. Available from: www. cis.hut.fi/projects/somtoolbox/theory/ somalgorithm.shtml [Accessed: 02 June

[23] Kohonen T. Personal Consultancy. Espoo: Technical University; 2006

[24] Kaski S. Data exploration using selforganizing maps. In: Acta Polytechnica

1964;**66**(3):171-200

New York: Springer; 2001

[17] Thuneberg H, Salmi H. Näkymätön

*Learning by Augmented Reality: Cluster Analysis Approach DOI: http://dx.doi.org/10.5772/intechopen.91252*

[15] Salmi H, Thuneberg H, Vainikainen M. Making the invisible observable by augmented reality in informal science education context. International Journal of Science Education. 2017;**7**(3):253-268. DOI: 10.1080/21548455.2016.1254358

[16] Renninger K. Individual interest and its implications for understanding intrinsic motivation. In: Sansone C, Harakiewicz J, editors. Intrinsic Motivation: Controversies and New Directions. San Diego, CA: Academic Press. pp. 373-404

[17] Thuneberg H, Salmi H. Näkymätön havaittavaksi—klusterianalyysi abstraktin ilmiön oppimisesta konkreettisesti lisäketodellisuuden (augmented reality) avulla. Psykologia. 2018;**53**(2-3):203-217. With English Summary

[18] Ryan R, Connell J. Perceived locus of causality and internalisation: Examining reasons for acting in two domains. Journal of Personality and Social Psychology. 1989;**57**:749-761

[19] Osgood CE. Semantic differential technique in the comparative study of cultures. American Anthropologist. 1964;**66**(3):171-200

[20] Raven J, Raven JC, Court J. Manual for Raven's Progressive Matrices and Vocabulary Scales. Oxford: OPP; 2003

[21] Kohonen T. Self-Organizing Maps. New York: Springer; 2001

[22] Kohonen T. The self-organizing map (SOM). 2005. Available from: www. cis.hut.fi/projects/somtoolbox/theory/ somalgorithm.shtml [Accessed: 02 June 2017]

[23] Kohonen T. Personal Consultancy. Espoo: Technical University; 2006

[24] Kaski S. Data exploration using selforganizing maps. In: Acta Polytechnica

Scandinavica, Mathematics, Computing and Management in Engineering Serie. Espoo: The Finnish Academy of Technology; 1997. p. 82

[25] Oja E, Mannila H. From data to knowledge, chapter 7. Self-Organizing Map. Available from: http://www.cis. hut.fi/Opinnot/T61.2010/Luennot/ JohdantoWhite.pdf

[26] Laboratory of computer and information science—Bibliography. Bibliography of SOM-Papers. 2017. Available from: http://www.cis.hut.fi/ research/refs/ [Accessed: 02 June 2017]

[27] Thuneberg H, Hotulainen R. Contributions of data mining for psycho-educational research: What self-organizing maps tell us about the well-being of gifted learners. High Ability Studies. 2006;**17**(1):87-100

[28] Thuneberg H. Is a majority enough? Psychological wellbeing and its relation to academic and prosocial motivation, self-regulation and achievement at school. In: Research Report 281. Helsinki: Department of Applied Sciences of Education, Faculty of Behavioural Sciences, Helsinki University Press; 2007

[29] Mayor J, Plunkett K. A neurocomputational account of taxonomic responding and fast mapping in early word learning. Psychological Review. 2010;**117**(1):1-31. DOI: 10.1037/ a00181300

[30] Carboni O, Russu P. Assessing regional wellbeing in Italy: An application of Malmquist–DEA and self-organizing map neural clustering. Social Indicators Research. 2015;**122**(3):677-700

[31] Rennie L. Learning science outside of school. In: Lederman N, Abell S, editors. Handbook of Research on Science Education. Vol. II. London: Routledge; 2014. pp. 120-144

**64**

*Mixed Reality and Three-Dimensional Computer Graphics*

frameworks for 21st century competences: Implications for national curriculum policies. Journal of Curriculum Studies. 2014;**44**(3):299-321. Available from: https://pdfs.semanticscholar.org/831e/ f07db8a589b91a508e0118f6f3c2a1f18c19.

[9] Finnish National Board of Education.

Curriculum. Helsinki: Opetushallitus;

[10] Salmi H, Kaasinen A, Kallunki V. Towards an open learning environment via augmented reality (AR): Visualising the invisible in science centres and schools for teacher education. Procedia:

Social and Behavioral Sciences. 2012;**45**:284-295. DOI: 10.1016/j.

[11] Sommerauer P, Müller O.

10.1016/j.edurev.2019.04.001

Augmented reality in informal learning environments: A field experiment in a mathematics exhibition. Computer Education. 2014;**79**:59-68. Available from: https://core.ac.uk/download/

[12] Garzón J, Acevedo J. Meta-analysis of the impact of augmented reality on students' learning gains. Educational Research Review. 2019;**27**:244-260. DOI:

[13] Yilmaz Z, Batdi V. A meta-analytic and thematic comparative analysis of the integration of augmented reality into education. Education in Science. 2016;**41**(188):273-289. DOI: 10.15390/

[14] Bacca J, Baldris S, Fabregat R, Graf S, Kinshuk K. Augmented reality trends in education: A systematic review of research and applications. Education Technology & Society.

sbspro.2012.06.565

pdf/143481015.pdf

EB.2016.6707

2014;**17**(4):133-149

FNCC: Finnish National Core

pdf

2019

[1] Zhang J, Sung Y-T, Hou H-T, Chang K-E. The development and evaluation of an augmented reality-based armillary sphere for astronomical observation instruction. Computers & Education.

[2] Cheng K-H, Tsai C-C. Affordances of augmented reality in science learning: Suggestions for future research. Journal of Science Education and Technology.

[3] Santos MEC, Chen A, Taketomi T, Yamamoto G, Miyazaki J, Kato H. Augmented reality learning experiences:

[4] Akçayır M, Akçayır G. Advantages and challenges associated with augmented reality for education: A systematic review of the literature. Educational Research Review.

[5] Afandi B, Kustiawan I, Herman N. Exploration of the augmented reality model in learning. Jornal of Physics Conference Series. 2019;**1375**:012082. DOI: 10.1088/1742-6596/1375/1/012082

[6] Harlen W. In: Harlen W, Bell D, Devés R, Dyasi H, de la Garza DF, Léna P, Millar R, Reiss M, Rowell P, Yu W, editors. Principles and Big Ideas of Science Education. Association for Science Education. Gosport, Hants: Ashford Colour Press Ltd., 2010. Available from: www.ase.org.uk/ bigideas. ISBN 978 0 86357 4 313

[7] Deakin R, Stringer C, Ren K. Learning to learn. In: International Perspectives from Theory and Practice. London: Routledge; 2014; 354 p. DOI:

10.4324/9780203078044

[8] Voogt J, Pareja Roblin N. A comparative analysis of international

Survey of prototype design and evaluation. IEEE Transactions on Learning Technologies. 2014;**7**(1):38-56

2014;**73**:178-188

**References**

2013;**22**(4):449-462

2017;**20**:1-11

[32] Tan LWH, Subramanian R. Science and the student entrepreneur. Science. 2002;**298**(5598):1556. DOI: 10.1126/ science.1074497

[33] Ryan R, Deci E, Grolnick W. Autonomy, relatedness, and the self: Their relation to development and psychopathology. In: Cicchetti D, Cohen D, editors. Developmental Psychopathology, Vol. 1, Theory and Methods. New York: John Wiley & Sons, Inc.; 1995. pp. 618-655

[34] Reeve J. Self-determination theory applied to educational settings. In: Deci E, Ryan R, editors. Handbook of Self-Determination. Rochester, NY: The University of Rochester Press; 2002

[35] Ryan R, Deci E. Self-determination theory and the facilation of intrinsic motivation, social development, and well-being. American Psychologist. 2000;**55**(1):68-78

[36] Kolb D. Experiential Learning: Experience as a Source of Learning and Development. Prentice Hall: Englewood Cliffd; 1984

[37] Klausmeier H, Hooper F. Conceptual development and instruction. In: Kerlinger F, Carroll B, editors. Review of Research in Education, Vol. 2. New York: F. E. Peacock; 1974

[38] Klausmeier H. Concept learning and concept teaching. Educational Psychologist. 1992;**27**:267-286

[39] Bressler D, Bodzin A. A mixed methods assessment of students' flow experiences during a mobile augmented reality science game. Journal of Computer Assisted Learning. 2013;**29**(6):505-517

[40] Radu J. Augmented reality in education: A meta-review and cross-media analysis. Personal

and Ubiquitous Computing. 2014;**18**(6):1533-1543

[41] Player-Koro C. The contemporary faith in educational technology—A critical perspective. Tidskrift för Professions Studier. 2016;**23**(2):98-106

**67**

Section 2

Mixed and Virtual Reality

Applications

Section 2
