**5. Results and analysis**

114 Applications of Virtual Reality

identical learning set up for each student, and students were given their assigned region as their home place to start the learning task. Region information is given in the Table 2 and the corresponding 3D MUVE map, and an island map is shown in the Figure 2. The root island (Learn) remained as an empty island with everyone to access as a sandbox so that students can try their desired content creation and other activities without affecting

Fig. 2. Wireless Island overview with layout – left side; right side – an interactive learning

**Island Name (Region) Relative Position Coordinates Number of Students**  Aelous1 X = 1002; Y = 1000 6 Aelous2 X = 1004; Y = 1000 6 Aelous3 X = 1002; Y = 1002 6 Aelous4 X = 1002; Y = 1004 6 Aeluos5 X = 1004; Y = 1002 6/7 Learn (root island) X = 1000; Y = 1000 -

In the Figure 3, a block of square represents an island (256 m x 256 m virtual area), and the islands are distributed as shown in the map, to minimise adjacency problems and to simulate the isolated island look and feel. Tiny green dots indicate the student avatar

The second part of the data gathering based on a questionnaire with 15 questions divided into two sections: Avatar Behaviour – 7 questions, and 3D MULE management – 8 questions. Additionally, five open-ended questions were included to help students to express their opinions openly. The 8 questions in the 3D MULE management section had some relevance on the two factors that we are investigating, self-regulation and environment management; however, the questions did not directly represent the variables. We decided to confirm through the statistical analysis, therefore, treated as 8 related

aid for simulating Exposed Node Problem in Wireless communication

Table 2. Experiment region information and their relative positions on the Grid

the learning environment.

positions when the image taken.

questions.

Based on the experiment design described above, our data gathering was conducted during the laboratory sessions by observing the avatar interactions and then at the end of the session through a questionnaire. Let us first discuss the observed student engagements during the laboratory session and then analyse the collected questionnaire data for the hypotheses testing.

#### **5.1 Observations of student behaviours**

For the preliminary analysis on observed student activities following scenarios are considered.

Avatar appearance change or outfit creation is an entertaining activity for any 3D MUVE user. However, the ease of creating attractive appearances within a limited time can be one of the critical determinants for student motivation on appearance change. Notably, the OpenSim with Hippo viewer gave an additional step on changing user appearance compared to Second Life. Users first have to create body-part, edit it and then wear to change the shape of the default avatar. Without this step, students could not change the gender of their avatars (default avatar has the view of a female user shown in the top left picture of the Figure 4). However, few students spent more time during their lab session and created more sophisticated shapes, clothes and appearance. Postgraduate students showed relatively less enthusiasm on changing their avatar shape, whereas many undergraduates went to a further step by comparing the avatar appearances with their friends'. In overall, students showed different preferences on spending time for appearance change and their commitments towards complex shape creation. However, we believe that the student commitment for making their avatars look good should not be underestimated as it could take substantial portions of their learning time. Constructive arrangements, such as, pre-sessions for familiarity and entertainment before the actual learning engagement, are therefore, highly encouraged.

3D Multi User Learning Environment Management –

An Exploratory Study on Student Engagement with the Learning Environment 117

Fig. 5. Observed student engagements through content creation and content alteration

Fig. 6. A terraformed and altered island (right side) compared to its' original layout

Fig. 4. Varying levels of avatar appearance customisation

Content object creation is one of the fundamental interactivity mechanisms available in 3D MUVE. Second Life and Open Simulator allow its users to create various basic shapes known as prims. Prims can be combined to create complex and compound shapes. The sculptured prims, to represent organic shapes that are difficult to design with basic prims, are also available for advanced content creation. Indeed, allowing students to create these attractive 3D shapes makes them engaged with the environment with passion. As observed, students tried range of constructions as well as editing of the existing objects. Some of these alterations directly affected the learning experience; activities such as wearing the control buttons of the media display, moving and changing the internal arrangement of the lecture theatre, and creating constructs on the simulation area (shown in Figure 5), should have been discouraged through usable policy consideration for a supportive learning. However, in this exploratory study, we wanted to examine such potential actions as empirical evidence to facilitate our future work; hence, students were given unrestricted access to their environments.

Fig. 4. Varying levels of avatar appearance customisation

their environments.

Content object creation is one of the fundamental interactivity mechanisms available in 3D MUVE. Second Life and Open Simulator allow its users to create various basic shapes known as prims. Prims can be combined to create complex and compound shapes. The sculptured prims, to represent organic shapes that are difficult to design with basic prims, are also available for advanced content creation. Indeed, allowing students to create these attractive 3D shapes makes them engaged with the environment with passion. As observed, students tried range of constructions as well as editing of the existing objects. Some of these alterations directly affected the learning experience; activities such as wearing the control buttons of the media display, moving and changing the internal arrangement of the lecture theatre, and creating constructs on the simulation area (shown in Figure 5), should have been discouraged through usable policy consideration for a supportive learning. However, in this exploratory study, we wanted to examine such potential actions as empirical evidence to facilitate our future work; hence, students were given unrestricted access to

Fig. 5. Observed student engagements through content creation and content alteration

Fig. 6. A terraformed and altered island (right side) compared to its' original layout

3D Multi User Learning Environment Management –

sample size and ratio.

suitability for Factor Analysis.

**5.2.2 Exploratory factor analysis – Hypothesis test** 

variation of the aspect 3D MULE Management.

PASW.

consideration.

An Exploratory Study on Student Engagement with the Learning Environment 119

Exploratory Factor Analysis tends to provide reliable factor identifications when the sample size is larger. There are many criteria to decide the appropriate sample size; however, many scholars agree that the suitable determination of the ratio between Minimum Sample Size (N) to Number of Variables (p) i.e., *N:p* should be a more reliable practice. Costello and Osborne (2005) concluded, through empirical tests, that high *N:p* provides higher accuracies compared to lower ratios. In our analysis on variable exploration *N:p* is 16:1 suggesting higher reliability in the solution model; hence validating the Factor Analysis outcome with our sample. Previous research (Moose and Azevedo, 2007) have suggested citing Nunnally (1978) that 10:1 is adequate and has performed Factor Analysis (N=49, p=3) validating our

Moreover, the Bartlett Test of Sphericity, a strict test on sampling and suitable correlations, was performed to test the appropriateness of data for the Factor Analysis, using PASW [version 18.0] statistical software. The Bartlett Test of Sphericity gave χ2= 155.257, p<.001 suggesting that the correlation matrix (R-Matrix) of items shows highly significant relationships and can be clustered based on relationships. Furthermore, the Null Hypothesis of R-Matrix being an Identity Matrix can be rejected with significance, indicating high

Finally, Kaiser-Meyer-Olkin (KMO) test was performed to examine the accuracy of using the data sample for the factor analysis. KMO value obtained was 0.714 (>0.7 threshold for goodness) (Field, 2006) validated the data sample for Factor Analysis. Having all the conditions for Factor Analysis met through the pre-test, we performed Factor Analysis using

With the positive results for the pre-tests performed to examine the appropriateness of using Exploratory Factor Analysis to confirm the research hypotheses (**H1** and **H2**), we used the Factor Analysis provided in PASW [18.0]. As the standard practice, our factor selection was based on the Principal Component Analysis with factors that have higher Eigenvalues (>1). The PASW output, shown in the Table 3, indicates two factors with Eigenvalues over 1. It is a clear separation of the contributing factors against the weak components, as the 3rd highest Eigenvalue is .648. Moreover, the highest two factors contribute nearly 72% of the total

For further analysis, we employed Orthogonal Verimax Rotation with Kaizer Normalization and obtained the rotated factor loadings per items as shown in the Table 3. For the suppression of weak loadings we used Stevens (1992) and Field (2006) suggestions for the cut-off for small samples, and used .6 as the limit, considering the exploratory objectives of the analysis. With the rotated loadings, we obtained improved factor loadings for the identified two factors. We also felt that the Orthogonal Verimax Rotation is accurate as the two factors more or less equally contribute (37.13% and 34.74%) to the underlying aspect of

Component plot in the rotated space is shown in the Figure 7. Furthermore, the Component Transformation Matrix showed symmetry over the diagonal, indicating that our use of

Orthogonal Rotation is accurate, and the rotated factor loadings are correct.

The Figure 6 shows a higher degree of student interaction that caused the learning environment to be significantly altered compared to its original layout and appearance. However, this observation was a one-off incident, as majority of students retrained from changing land settings, although they have tried content creation, often. Moreover, compared to the Masters students, the junior honours students showed high interactivity and environment alteration, resulted in a range of user created objects, altered content and changed land terrain. The undergraduates showed more enthusiasm on exploring potential game-like features, and associate their real-world friends for collaborative activities/plays, although such acts were not necessarily related with the Intended Learning Outcomes (ILOs) or Teaching and Learning Activities (TLAs). In contrast, Postgraduate students showed a higher tendency to complete the given tasks, which may have, in certain instances, resulted in less motivation to entertain themselves by exploring the 3D MUVE features.

It is important to note that, in all of these cases, students were allowed to follow their preferred behaviour to interact with the environment as a mean of learning through exploration (Kolb *et al.*, 2001) without any restriction. An assurance was given that their behaviour or environment interactions do not affect their assessments, but the completion of learning tasks.

#### **5.2 Analysis of questionnaire data**

We received 32 completed questionnaires (54.28%), which consisted of 20 responses from postgraduate students (71.4%) and 12 responses from undergraduate students (38.7%). A preliminary observation was performed to examine the question characteristics and to validate the designed mappings of variables based on the collected responses. Descriptive statistics and the literal meanings of the questions were considered to understand the user responds and possible classifications. Analytical observations of the question and respond behaviour resulted in preliminary clustering of questions into two major categories, as designed in the questionnaire: User Behaviour and 3D MULE Management, which include questions Q1 to Q7 and Q8 to Q15, respectively, supporting the design rational of the questionnaire.

The seven questions in User Engagement section (Q1 to Q7) indicates a high level of internal consistency among the questions with Cronbach's alpha = 0.802 (>0.7) table7. Furthermore, the intended data collection on user behaviour section was mainly based on student behavioural activities within the environment; therefore, the seven questions were designed to cover seven different student engagement aspects. In that respect, statistical observation of internal consistency validated the combined use of questions to represent the associated variable.

#### **5.2.1 Pre-test for factor analysis**

The questionnaire section on 3D MULE management was designed, as explained in the research design, examining two prime variables associating system administration and user self-control by retaining from misconduct. Although, we had this in mind when the questionnaire was designed, we were compelled to analyse the set of questions statistically for an accurate identification of the variables as a method to test the research hypotheses.

The Figure 6 shows a higher degree of student interaction that caused the learning environment to be significantly altered compared to its original layout and appearance. However, this observation was a one-off incident, as majority of students retrained from changing land settings, although they have tried content creation, often. Moreover, compared to the Masters students, the junior honours students showed high interactivity and environment alteration, resulted in a range of user created objects, altered content and changed land terrain. The undergraduates showed more enthusiasm on exploring potential game-like features, and associate their real-world friends for collaborative activities/plays, although such acts were not necessarily related with the Intended Learning Outcomes (ILOs) or Teaching and Learning Activities (TLAs). In contrast, Postgraduate students showed a higher tendency to complete the given tasks, which may have, in certain instances, resulted in less motivation to entertain themselves by exploring the 3D MUVE features.

It is important to note that, in all of these cases, students were allowed to follow their preferred behaviour to interact with the environment as a mean of learning through exploration (Kolb *et al.*, 2001) without any restriction. An assurance was given that their behaviour or environment interactions do not affect their assessments, but the completion of

We received 32 completed questionnaires (54.28%), which consisted of 20 responses from postgraduate students (71.4%) and 12 responses from undergraduate students (38.7%). A preliminary observation was performed to examine the question characteristics and to validate the designed mappings of variables based on the collected responses. Descriptive statistics and the literal meanings of the questions were considered to understand the user responds and possible classifications. Analytical observations of the question and respond behaviour resulted in preliminary clustering of questions into two major categories, as designed in the questionnaire: User Behaviour and 3D MULE Management, which include questions Q1 to Q7 and Q8 to Q15, respectively, supporting the design rational of

The seven questions in User Engagement section (Q1 to Q7) indicates a high level of internal consistency among the questions with Cronbach's alpha = 0.802 (>0.7) table7. Furthermore, the intended data collection on user behaviour section was mainly based on student behavioural activities within the environment; therefore, the seven questions were designed to cover seven different student engagement aspects. In that respect, statistical observation of internal consistency validated the combined use of questions to represent

The questionnaire section on 3D MULE management was designed, as explained in the research design, examining two prime variables associating system administration and user self-control by retaining from misconduct. Although, we had this in mind when the questionnaire was designed, we were compelled to analyse the set of questions statistically for an accurate identification of the variables as a method to test the research hypotheses.

learning tasks.

the questionnaire.

the associated variable.

**5.2.1 Pre-test for factor analysis** 

**5.2 Analysis of questionnaire data** 

Exploratory Factor Analysis tends to provide reliable factor identifications when the sample size is larger. There are many criteria to decide the appropriate sample size; however, many scholars agree that the suitable determination of the ratio between Minimum Sample Size (N) to Number of Variables (p) i.e., *N:p* should be a more reliable practice. Costello and Osborne (2005) concluded, through empirical tests, that high *N:p* provides higher accuracies compared to lower ratios. In our analysis on variable exploration *N:p* is 16:1 suggesting higher reliability in the solution model; hence validating the Factor Analysis outcome with our sample. Previous research (Moose and Azevedo, 2007) have suggested citing Nunnally (1978) that 10:1 is adequate and has performed Factor Analysis (N=49, p=3) validating our sample size and ratio.

Moreover, the Bartlett Test of Sphericity, a strict test on sampling and suitable correlations, was performed to test the appropriateness of data for the Factor Analysis, using PASW [version 18.0] statistical software. The Bartlett Test of Sphericity gave χ2= 155.257, p<.001 suggesting that the correlation matrix (R-Matrix) of items shows highly significant relationships and can be clustered based on relationships. Furthermore, the Null Hypothesis of R-Matrix being an Identity Matrix can be rejected with significance, indicating high suitability for Factor Analysis.

Finally, Kaiser-Meyer-Olkin (KMO) test was performed to examine the accuracy of using the data sample for the factor analysis. KMO value obtained was 0.714 (>0.7 threshold for goodness) (Field, 2006) validated the data sample for Factor Analysis. Having all the conditions for Factor Analysis met through the pre-test, we performed Factor Analysis using PASW.

#### **5.2.2 Exploratory factor analysis – Hypothesis test**

With the positive results for the pre-tests performed to examine the appropriateness of using Exploratory Factor Analysis to confirm the research hypotheses (**H1** and **H2**), we used the Factor Analysis provided in PASW [18.0]. As the standard practice, our factor selection was based on the Principal Component Analysis with factors that have higher Eigenvalues (>1). The PASW output, shown in the Table 3, indicates two factors with Eigenvalues over 1. It is a clear separation of the contributing factors against the weak components, as the 3rd highest Eigenvalue is .648. Moreover, the highest two factors contribute nearly 72% of the total variation of the aspect 3D MULE Management.

For further analysis, we employed Orthogonal Verimax Rotation with Kaizer Normalization and obtained the rotated factor loadings per items as shown in the Table 3. For the suppression of weak loadings we used Stevens (1992) and Field (2006) suggestions for the cut-off for small samples, and used .6 as the limit, considering the exploratory objectives of the analysis. With the rotated loadings, we obtained improved factor loadings for the identified two factors. We also felt that the Orthogonal Verimax Rotation is accurate as the two factors more or less equally contribute (37.13% and 34.74%) to the underlying aspect of consideration.

Component plot in the rotated space is shown in the Figure 7. Furthermore, the Component Transformation Matrix showed symmetry over the diagonal, indicating that our use of Orthogonal Rotation is accurate, and the rotated factor loadings are correct.

3D Multi User Learning Environment Management –

following results that summarises the variable behaviour.

Q3 I tried to change land or content objects in the

Q5 I have followed other avatars collaboratively

Q7 My activities in the environment resulted in a

Table 4. Questions on student behaviour and the descriptive statistics

Q6 I moved to all the places in my island and

Disagree" as their answer (Mode is 3).

**5.2.3 Variable summary** 

**Student engagement** 

environment.

1 summarises the analysis results

An Exploratory Study on Student Engagement with the Learning Environment 121

Including the verified two variables, statistical analyses were performed and obtained the

With suitable naming, we examined the questions associated with this variable for descriptive statistics and overall variable behaviour based on the question scores. The table

**No Question Mean Mode Std. Dev. Std. Error**  Q1 I changed my appearance as I like to appear 3.09 3 0.466 .082 Q2 I created content objects in the environment 3.69 4 0.471 .083

learning environment 3.66 4 0.483 .085 Q4 I communicated with others regularly 3.53 4 0.507 .090

during my learning 3.44 3 0.504 .089

teleported to other islands 3.81 4 0.397 .070

high engagement with my learning tasks 3.97 4 0.40 .071

Additionally, the One-Sample Kolmogorov-Smirnov Test for normality indicated the question means are normally distributed [*X ~ N (3.598, 0.283)*] with significance (α=0.997) to retain the hypothesis of normal distribution. This implies that in general, students showed a positive response indicating that there has been a high degree of engaged with the

However, the questions Q1 and Q5 (Q1 in particular) showed averages around 3 corresponding to the answer: "Neither Agree nor Disagree". If we have a closer look at the questions, the Q1 relates the student engagement through the measure on the level of avatar appearance modification. In fact, few students enjoyed by modifying their avatars to represent, either their real identity through physical appearance or having a fantasy look. However, most of the students were comfortable in using the default avatar or with minimum customisation, such as colour change of clothes, and committed their time more on other engagements and learning tasks, which resulted in this lower mean value. The Q5 asks about the student collaboration, specific on following others behaviour or learning steps. Although, we encourage students to collaboratively work with their peers, traditional learning environments are not supportive of collaborative learning unless the students are asked to do so. In this learning activity, the students were given individual task sheets that they should complete, but not specifically asked to collaborate with other students assigned into the same region. Moreover, the majority of the students may have felt that they should not misuse the flexibility given within the 3D MULE, as their individual performances on the tasks are assessed, taking a more self-restrained behaviour on collaborating with others. As a result in both cases (Q1 and Q5) the majority reported scale 3 – "Neither Agree nor


Table 3. Factor Analysis Output from PASW (including rotated loadings)

As the rotated factor loadings indicated, questions Q9, Q15, Q10 and Q8 were considered as a one variable and examined for suitable naming. When the objectives of questions were analysed, we identified a common parameter that governs all the four questions. We concluded it as the student behaviour with Self-regulation, and defined the new variable as Self-Regulation. The second factor represents Q11, Q14, Q12 and Q13 questions and we observed a strong contextual meaning on system administration and control as the common parameter. The questions strongly discuss the student preference and impact on the system control, administration and management of the 3D MUVE. Therefore, we defined this variable as Environment Management to cover all the aspects that associate with this factor.

Fig. 7. Rotated Component Matrix and the Component Plot in Rotated Space

The identified two variables explain nearly 72% of the behaviour of the aspect 3D MULE Management. Moreover, there aren't any other strong and significant variable revealed through the factor analysis; therefore, we can conclude that the extracted two variables, Self-Regulation and Environment Management, represent the aspect of 3D MULE Management. Hence, we conclude that the research hypotheses **H1** and **H2** of our study is substantiated based on the exploratory factor analysis.

#### **5.2.3 Variable summary**

120 Applications of Virtual Reality

As the rotated factor loadings indicated, questions Q9, Q15, Q10 and Q8 were considered as a one variable and examined for suitable naming. When the objectives of questions were analysed, we identified a common parameter that governs all the four questions. We concluded it as the student behaviour with Self-regulation, and defined the new variable as Self-Regulation. The second factor represents Q11, Q14, Q12 and Q13 questions and we observed a strong contextual meaning on system administration and control as the common parameter. The questions strongly discuss the student preference and impact on the system control, administration and management of the 3D MUVE. Therefore, we defined this variable as Environment Management to cover all the aspects that associate with this factor.

Table 3. Factor Analysis Output from PASW (including rotated loadings)

Fig. 7. Rotated Component Matrix and the Component Plot in Rotated Space

based on the exploratory factor analysis.

The identified two variables explain nearly 72% of the behaviour of the aspect 3D MULE Management. Moreover, there aren't any other strong and significant variable revealed through the factor analysis; therefore, we can conclude that the extracted two variables, Self-Regulation and Environment Management, represent the aspect of 3D MULE Management. Hence, we conclude that the research hypotheses **H1** and **H2** of our study is substantiated Including the verified two variables, statistical analyses were performed and obtained the following results that summarises the variable behaviour.

#### **Student engagement**

With suitable naming, we examined the questions associated with this variable for descriptive statistics and overall variable behaviour based on the question scores. The table 1 summarises the analysis results


Table 4. Questions on student behaviour and the descriptive statistics

Additionally, the One-Sample Kolmogorov-Smirnov Test for normality indicated the question means are normally distributed [*X ~ N (3.598, 0.283)*] with significance (α=0.997) to retain the hypothesis of normal distribution. This implies that in general, students showed a positive response indicating that there has been a high degree of engaged with the environment.

However, the questions Q1 and Q5 (Q1 in particular) showed averages around 3 corresponding to the answer: "Neither Agree nor Disagree". If we have a closer look at the questions, the Q1 relates the student engagement through the measure on the level of avatar appearance modification. In fact, few students enjoyed by modifying their avatars to represent, either their real identity through physical appearance or having a fantasy look. However, most of the students were comfortable in using the default avatar or with minimum customisation, such as colour change of clothes, and committed their time more on other engagements and learning tasks, which resulted in this lower mean value. The Q5 asks about the student collaboration, specific on following others behaviour or learning steps. Although, we encourage students to collaboratively work with their peers, traditional learning environments are not supportive of collaborative learning unless the students are asked to do so. In this learning activity, the students were given individual task sheets that they should complete, but not specifically asked to collaborate with other students assigned into the same region. Moreover, the majority of the students may have felt that they should not misuse the flexibility given within the 3D MULE, as their individual performances on the tasks are assessed, taking a more self-restrained behaviour on collaborating with others. As a result in both cases (Q1 and Q5) the majority reported scale 3 – "Neither Agree nor Disagree" as their answer (Mode is 3).

3D Multi User Learning Environment Management –

**Environment management** 

management

Q11

An Exploratory Study on Student Engagement with the Learning Environment 123

The questions Q9, Q10 and Q15 more or less have recorded nearly same averages (~4 = Agree) while the majority confirms that preference as the Mode is 4 for each question. Q9 was designed associating the privacy concerns of being in an open environment that could be seen by others and with a high probability on simultaneous engagement on same learning activity or content. The association of real-world classroom metaphor reinforces the student comparative observations, resulting in a broader opinion with higher accuracy. Q10 examines the student view on having their real identity (first name and last name) as their avatar username. Avatar anonymity and its impact on student learning has been researched previously on various contexts (Messinger *et al.,* 2008); the majority of students agreed (Mode = 4 and Mean = 4.02) that there is a positive impact of using their real identities on following appropriate behaviour within the environment. The Q15 plays the concluding role for the student's self-regulatory practice with the sense of being a responsible participant in the learning session. Importantly, we did not relate any indirect variable association as the effect of being a responsible student with proper practice; we let the student to self-evaluate the consequences of their practices for the responses. The student responses indicate that majority of the students agreed that students must use the environment responsibly, indicating a positive association of self-regulated interaction as an acceptable practice.

The One-Sample Komogorov-Smirnov Test for normality indicated the question means are normally distributed [*X ~ N (3.95, 0.175)*] with significance (α=0.796) to retain the hypothesis of normal distribution. This implies that in general student showed a positive response

are important for a reliable learning 3.91 4 0.296 .052

reduce the 3D MUVE usability 4.19 4 0.592 .105

ensure a successful learning experience 3.89 4 0.390 .070

These four questions associate the system administration and environment control aspect as students perceive. The question Q11 asks about two major 3D MULE system administration function groups for their significance in using as an environment management mechanism. Land and content management related functions are the most significant activities that an avatar can use to affect the existing learning environment. Therefore, it is important to examine the student preference on constraining them from using these features if needed for environment management. As the results indicate, students agreed on this statement. The question Q12 further examines the student view on having a reliable learning experience through system controls and environment management. This statement associates the aspect that the learning environment is trustful. Student responses indicate that they still welcome

Dev.

3.78 4 0.420 .074

Std. Error

indicating that there has been a high degree of engaged with the environment.

No Question Mean Mode Std.

Table 6. Questions on system environment management and descriptive statistics

Land and Content management controls are important for the environment

Q12 System control and management practices

Q13 System management settings should not

Q14 Appropriate system security and controls

The questions Q2, Q3, Q4 and Q6 were based on the remaining avatar actions that associate with the possible major areas of avatar engagement. Content creation, land and content alteration, and avatar communication activities showed reasonably similar average responds agreeing with the asked questions. Additionally, the most of the students confirmed that response (Mode = 4). The question Q6 had the environment exploration aspect within the assigned island and the teleportation to other islands. Almost all the students indicated higher positive average with most of the students agreeing with the statement.

Q7 played the concluding role for the student behaviour question set. It let the students to think about their activities within the 3D MULE and then evaluate his or her learning engagement as a result of the activities performed. This was a crucial question as it directly inspects about the considered aspect of student engagement as a measure of student activity. Student responses showed a high average, indicating that they agree with the question (Mode =4). This strengthened our research hypotheses as a direct measure, while showing a possible relationship between the environment interactions and the learning engagement, which we recommend for further study.

#### **Self-regulation**

The One-Sample Kolmogorov-Smirnov Test for normality indicated that the question means are normally distributed [*X ~ N (3.876, 0.378)*] with significance (α=0.516) to retain the hypothesis of normal distribution. This implies that, in general, students have shown a positive response indicating that students agree the self-regulatory practice as an important consideration.


Table 5. Questions on student self-regulation and descriptive statistics

The question Q8 tends to be a self-assessing question as students had to think about their behaviour reflectively and critically. This was important to meet the objectives of the question set, as students answer the rest of the questions with a reflective mind associating every little detail that they have experienced or felt during the engagement. In that sense, beyond the literal meaning of the Q8 it helped students to give accurate responses to the rest of the questions as an indirect effect. However, student may have been doubtful about on what degree they consider their behaviour and impact on others' learning; therefore, we observe an average of 3.31 (more towards the respond "*Neither Agree nor Disagree"*) while the majority confirming that (mode = 3).

The questions Q9, Q10 and Q15 more or less have recorded nearly same averages (~4 = Agree) while the majority confirms that preference as the Mode is 4 for each question. Q9 was designed associating the privacy concerns of being in an open environment that could be seen by others and with a high probability on simultaneous engagement on same learning activity or content. The association of real-world classroom metaphor reinforces the student comparative observations, resulting in a broader opinion with higher accuracy. Q10 examines the student view on having their real identity (first name and last name) as their avatar username. Avatar anonymity and its impact on student learning has been researched previously on various contexts (Messinger *et al.,* 2008); the majority of students agreed (Mode = 4 and Mean = 4.02) that there is a positive impact of using their real identities on following appropriate behaviour within the environment. The Q15 plays the concluding role for the student's self-regulatory practice with the sense of being a responsible participant in the learning session. Importantly, we did not relate any indirect variable association as the effect of being a responsible student with proper practice; we let the student to self-evaluate the consequences of their practices for the responses. The student responses indicate that majority of the students agreed that students must use the environment responsibly, indicating a positive association of self-regulated interaction as an acceptable practice.

#### **Environment management**

122 Applications of Virtual Reality

The questions Q2, Q3, Q4 and Q6 were based on the remaining avatar actions that associate with the possible major areas of avatar engagement. Content creation, land and content alteration, and avatar communication activities showed reasonably similar average responds agreeing with the asked questions. Additionally, the most of the students confirmed that response (Mode = 4). The question Q6 had the environment exploration aspect within the assigned island and the teleportation to other islands. Almost all the students indicated higher positive average with most of the students

Q7 played the concluding role for the student behaviour question set. It let the students to think about their activities within the 3D MULE and then evaluate his or her learning engagement as a result of the activities performed. This was a crucial question as it directly inspects about the considered aspect of student engagement as a measure of student activity. Student responses showed a high average, indicating that they agree with the question (Mode =4). This strengthened our research hypotheses as a direct measure, while showing a possible relationship between the environment interactions and the learning

The One-Sample Kolmogorov-Smirnov Test for normality indicated that the question means are normally distributed [*X ~ N (3.876, 0.378)*] with significance (α=0.516) to retain the hypothesis of normal distribution. This implies that, in general, students have shown a positive response indicating that students agree the self-regulatory practice as an important

learning 3.31 3 0.592 .104

behaviour of students 4.02 4 0.309 .056

learning environment 4.10 4 0.296 .051

The question Q8 tends to be a self-assessing question as students had to think about their behaviour reflectively and critically. This was important to meet the objectives of the question set, as students answer the rest of the questions with a reflective mind associating every little detail that they have experienced or felt during the engagement. In that sense, beyond the literal meaning of the Q8 it helped students to give accurate responses to the rest of the questions as an indirect effect. However, student may have been doubtful about on what degree they consider their behaviour and impact on others' learning; therefore, we observe an average of 3.31 (more towards the respond "*Neither Agree nor Disagree"*) while

Dev.

4.05 4 0.354 .064

Std. Error

No Question Mean Mode Std.

agreeing with the statement.

**Self-regulation** 

consideration.

session

Q9

engagement, which we recommend for further study.

Q8 I think my behaviour affected others'

Q10 Use of real identities increases the proper

Q15 Student should responsibly use the

the majority confirming that (mode = 3).

The open space and others avatars made me to interact as in a real-world learning

Table 5. Questions on student self-regulation and descriptive statistics

The One-Sample Komogorov-Smirnov Test for normality indicated the question means are normally distributed [*X ~ N (3.95, 0.175)*] with significance (α=0.796) to retain the hypothesis of normal distribution. This implies that in general student showed a positive response indicating that there has been a high degree of engaged with the environment.


Table 6. Questions on system environment management and descriptive statistics

These four questions associate the system administration and environment control aspect as students perceive. The question Q11 asks about two major 3D MULE system administration function groups for their significance in using as an environment management mechanism. Land and content management related functions are the most significant activities that an avatar can use to affect the existing learning environment. Therefore, it is important to examine the student preference on constraining them from using these features if needed for environment management. As the results indicate, students agreed on this statement. The question Q12 further examines the student view on having a reliable learning experience through system controls and environment management. This statement associates the aspect that the learning environment is trustful. Student responses indicate that they still welcome

3D Multi User Learning Environment Management –

**5.3 Hypothesis testing** 

regression model is reliable.

variables to represent the 3D MULE Management aspect.

An Exploratory Study on Student Engagement with the Learning Environment 125

variables Self-Regulation and Environment Management are sufficiently independent as measuring two different aspects, which further proves the Factor analysis and selection of

To test the research hypotheses **H3** and **H4**, we used regression analysis between the hypotheses variables. Furthermore, this enabled us to examine the variable behaviour and

A sample size test was done for the fitness for regression analysis, as the first step. As defined in Cohen (1988) and further elaborated by Field (2006), for the test statistics of *anticipated large effect* (*F2=0.35*), Number of *predictors* (*n=2*), Probability level of Significance (*α = 0.05*) with the desired *statistical Power level* of (*1-β = 0.8*), the minimum required sample size was 31. Therefore, our sample size *N* = 32 (*>31*) suites well for the analysis and the

the exact relationships among the dependent and independent variables.

Fig. 9. Regression analysis output with model test using ANOVA (PASW)

PASW (18.0) linear regression analysis model summary and the model fit (ANOVA) is shown in the Fig.9. R2 = 0.759, indicates that about 75.9% of the variation in the student engagement is determined by the environment management and student self-regulation with the learning activities in 3D MULE, by means of a combined effect. As the model fit test results in ANOVA shows, the regression model significantly explains the Student

the controlling measures on the 3D MULE to improve the trust, knowing the importance of the reliability of a learning activity for formal education. The Q13 discusses the associated concern on 3D MULE usability, if the management controls are restrictive and unsupportive for attractive learning engagement. Interestingly, the average feedback was 4.19, the highest value for all the questions, showed the student concern on losing the usability and attractiveness of learning environment. This delivers a very important message to the 3D MULE designers and module administrators, as their environment management policies should always prioritise the usability of the environment. Finally, the question Q14 brings a conclusive statement for the environment management concerns. This, in fact, summarises the student's overall opinion on the use of system environment controls and management practices to ensure successful learning. Students on average agreed to the statement showing their positive attitude for a supportive environment management for their learning facilitation.

#### **5.2.4 Variable analysis**

The internal consistency was measured among the question items within the three variables, using Cronbach Alpha. Results, summarised in Table 7, are higher than the recommended thresholds (>0.7 for exploratory studies), indicating higher internal consistency within variable items. As discussed previously, similar to the variable student engagement, the question items that associate Self-Regulation and Environment Management also meet the requirements to be considered collectively to represent the relevant variable.


Table 7. Summary of the analysis variables and their internal consistency measure


Fig. 8. Correlation between the two identified variables self-regulation and environment management

Finally, the Spearman Correlation analysis between the derived variables, Self-Regulation and Environment Management was performed (shown in Figure 8) to examine their relationship. The Spearman Correlation Coefficient (*rho*) was 0.398 indicating a weak positive relationship with significance (*p<0.05*). Furthermore, the weak relationship between the two variables was further indicated as only about 15% of one's variance could be explained by the other. This is important to proceed with our analysis as we have to be certain about the model that we are testing; i.e., the two variables are measuring sufficiently different parameters and the inter-relationship between the two is insignificant to affect the hypothesis test. Therefore, we conclude that, through all these analyses that the two variables Self-Regulation and Environment Management are sufficiently independent as measuring two different aspects, which further proves the Factor analysis and selection of variables to represent the 3D MULE Management aspect.

#### **5.3 Hypothesis testing**

124 Applications of Virtual Reality

the controlling measures on the 3D MULE to improve the trust, knowing the importance of the reliability of a learning activity for formal education. The Q13 discusses the associated concern on 3D MULE usability, if the management controls are restrictive and unsupportive for attractive learning engagement. Interestingly, the average feedback was 4.19, the highest value for all the questions, showed the student concern on losing the usability and attractiveness of learning environment. This delivers a very important message to the 3D MULE designers and module administrators, as their environment management policies should always prioritise the usability of the environment. Finally, the question Q14 brings a conclusive statement for the environment management concerns. This, in fact, summarises the student's overall opinion on the use of system environment controls and management practices to ensure successful learning. Students on average agreed to the statement showing their positive attitude for a supportive environment management for their learning

The internal consistency was measured among the question items within the three variables, using Cronbach Alpha. Results, summarised in Table 7, are higher than the recommended thresholds (>0.7 for exploratory studies), indicating higher internal consistency within variable items. As discussed previously, similar to the variable student engagement, the question items that associate Self-Regulation and Environment Management also meet the

Variable Name (Factor) Mean µ Standard Deviation σ Cronbach α Number of Items

requirements to be considered collectively to represent the relevant variable.

Student Engagement 3.598 0.283 .802 7 Self-Regulation 3.876 0.378 .829 4 Environment Management 3.950 0.175 .80 4

Table 7. Summary of the analysis variables and their internal consistency measure

Fig. 8. Correlation between the two identified variables self-regulation and environment

Finally, the Spearman Correlation analysis between the derived variables, Self-Regulation and Environment Management was performed (shown in Figure 8) to examine their relationship. The Spearman Correlation Coefficient (*rho*) was 0.398 indicating a weak positive relationship with significance (*p<0.05*). Furthermore, the weak relationship between the two variables was further indicated as only about 15% of one's variance could be explained by the other. This is important to proceed with our analysis as we have to be certain about the model that we are testing; i.e., the two variables are measuring sufficiently different parameters and the inter-relationship between the two is insignificant to affect the hypothesis test. Therefore, we conclude that, through all these analyses that the two

facilitation.

management

**5.2.4 Variable analysis** 

To test the research hypotheses **H3** and **H4**, we used regression analysis between the hypotheses variables. Furthermore, this enabled us to examine the variable behaviour and the exact relationships among the dependent and independent variables.

A sample size test was done for the fitness for regression analysis, as the first step. As defined in Cohen (1988) and further elaborated by Field (2006), for the test statistics of *anticipated large effect* (*F2=0.35*), Number of *predictors* (*n=2*), Probability level of Significance (*α = 0.05*) with the desired *statistical Power level* of (*1-β = 0.8*), the minimum required sample size was 31. Therefore, our sample size *N* = 32 (*>31*) suites well for the analysis and the regression model is reliable.




Fig. 9. Regression analysis output with model test using ANOVA (PASW)

PASW (18.0) linear regression analysis model summary and the model fit (ANOVA) is shown in the Fig.9. R2 = 0.759, indicates that about 75.9% of the variation in the student engagement is determined by the environment management and student self-regulation with the learning activities in 3D MULE, by means of a combined effect. As the model fit test results in ANOVA shows, the regression model significantly explains the Student

3D Multi User Learning Environment Management –

**5.4 Summary of open feedback** 

brackets, for each question.

Environments for learning activities?

is an open research possibility for further study.

*[No privacy with my activities] [It is open, so privacy is a big concern]* 

*[I believe it is safe and secure]* 

*[I'm not sure]* 

overall opinion?

An Exploratory Study on Student Engagement with the Learning Environment 127

The given questionnaire had a section with five open-ended questions, mentioned below, to capture student opinions in general. The most important objective was to identify any concerns or feedback that we cannot capture through the first 15 questions. Therefore, we associated the common parameters such as security and privacy concerns to facilitate the Environment Management policy considerations. We like to mention the fact that not all participants have responded to these questions. However, a wider range of responses was observed with a reasonably good turnout. Few selected student expressions are shown in

**OFQ1** – Are there any privacy concerns you have regarding the use of 3D Multi User Virtual

This question aims at capturing privacy concerns that the students have or felt while engaging the learning activities. Importantly, this was taken as a measure for both Environment Management and Self-Regulation. Majority of the students felt that their activities are visible to others so the actions were committed consciously knowing that they are in an open environment. It is interesting to see that students felt the immersion at most and felt that their work could be monitored, which is also possible with other information systems as a forensic measure, if they commit misconduct. However, students feeling of being a part of open community may have resulted in shaping their behaviour. However, it

**OFQ2** - Are there any security challenges on using 3D MUVE for your learning, as an

This question let the students to express their opinions on learning environment security. Importantly, the security enforcement of a learning environment is well researched (Weippl, 2005) and a prominent concern for present technology supported learning solutions (Perera et al., 2011a). With respect to the study, this question had the objective of identifying students' trust on existing 3D MUVE security implementations and policies employed. Moreover, students must have a confidence on the learning environment reliability to engage comfortably with the assessed and serious learning tasks. Overall, students showed their lack of awareness on the security and administrative measures available in 3D MULE.

*[I feel psychologically aware about my progression being observed by others]* 

This was an important observation, and we consider it for our future research.

**OFQ3** - What are the areas of 3D MULE that should be addressed or improved?

This was a general question to see whether students have different thoughts for improving the learning experience with 3D MULE. There were few interesting thoughts expressed, although following two ideas appear to be undoubtedly correct. First, some students expressed their need on having a private space if the learning activities are not designed for collaboration. Secondly, various concerns were raised on familiarising the 3D MUVE. It is a known fact that 3D MUVE have a steep learning curve (OpenSim, 2007). Importantly, some

*[There can be, but I'm not so sure, I need to learn more]* 

Engagement from the two variables Environment Management and Self-Regulation (*p<0.001*).

The individual relationships of the independent variables Self-Regulation and Environment Management with the dependent variable Student Engagement were obtained through the regression test coefficients, and the summary is mentioned in the Table 8.


*\* p < 0.05* 

*\*\* p < 0.01* 

Table 8. Regression analysis summary

As the variable relationship with predictor parameters of the model shown in table 6, the model path coefficients are .240 for Self-Regulation, which is significant (*p*<0.05) and .657 for Environment Management with significance (*p<.001*). The resulted research model outcome can be shown as follows (Figure 10).

Fig. 10. Hypotheses model summary with path coefficients

Therefore, the research model substantiates our research hypotheses **H3** and **H4**. With reference to **H3**, we can say that the student *Self-Regulation* on learning activities in 3D MULE result in a significant positive effect on the *Student Engagement* with the learning environment and learning tasks. Therefore, we conclude that, for constructive and successful 3D MULE student engagement, we have to consider and promote student selfregulation as a main policy consideration area for 3D MULE management. Thus, we validate our research hypothesis on considering student self-regulation as a prime factor for successful management of 3D MULE and associated learning activities.

Regarding to the hypothesis **H4**, we can say that the *Environment Management* practices, as described previously, result in a significant positive effect on Student Engagement with the 3D MULE and learning tasks. Therefore, we conclude that, for constructive and successful 3D MULE student engagement, we have to identify and implement Environment Management policies as a main policy consideration area for 3D MULE management. Thus, we validate our research hypothesis on considering environment management of 3D MULE as a prime factor for successful management of 3D MULE and associated learning activities.

#### **5.4 Summary of open feedback**

126 Applications of Virtual Reality

Engagement from the two variables Environment Management and Self-Regulation

The individual relationships of the independent variables Self-Regulation and Environment Management with the dependent variable Student Engagement were obtained through the

**Variable** *β* **Standard Error t-value Significance**  Self-Regulation .240 .097 2.482 0.019\* Environment Management .657 .092 7.312 0.000\*\*

As the variable relationship with predictor parameters of the model shown in table 6, the model path coefficients are .240 for Self-Regulation, which is significant (*p*<0.05) and .657 for Environment Management with significance (*p<.001*). The resulted research model outcome

Therefore, the research model substantiates our research hypotheses **H3** and **H4**. With reference to **H3**, we can say that the student *Self-Regulation* on learning activities in 3D MULE result in a significant positive effect on the *Student Engagement* with the learning environment and learning tasks. Therefore, we conclude that, for constructive and successful 3D MULE student engagement, we have to consider and promote student selfregulation as a main policy consideration area for 3D MULE management. Thus, we validate our research hypothesis on considering student self-regulation as a prime factor for

Regarding to the hypothesis **H4**, we can say that the *Environment Management* practices, as described previously, result in a significant positive effect on Student Engagement with the 3D MULE and learning tasks. Therefore, we conclude that, for constructive and successful 3D MULE student engagement, we have to identify and implement Environment Management policies as a main policy consideration area for 3D MULE management. Thus, we validate our research hypothesis on considering environment management of 3D MULE as a prime factor for successful management of 3D MULE and associated learning activities.

regression test coefficients, and the summary is mentioned in the Table 8.

(*p<0.001*).

*\* p < 0.05 \*\* p < 0.01* 

Table 8. Regression analysis summary

can be shown as follows (Figure 10).

Fig. 10. Hypotheses model summary with path coefficients

successful management of 3D MULE and associated learning activities.

The given questionnaire had a section with five open-ended questions, mentioned below, to capture student opinions in general. The most important objective was to identify any concerns or feedback that we cannot capture through the first 15 questions. Therefore, we associated the common parameters such as security and privacy concerns to facilitate the Environment Management policy considerations. We like to mention the fact that not all participants have responded to these questions. However, a wider range of responses was observed with a reasonably good turnout. Few selected student expressions are shown in brackets, for each question.

**OFQ1** – Are there any privacy concerns you have regarding the use of 3D Multi User Virtual Environments for learning activities?

This question aims at capturing privacy concerns that the students have or felt while engaging the learning activities. Importantly, this was taken as a measure for both Environment Management and Self-Regulation. Majority of the students felt that their activities are visible to others so the actions were committed consciously knowing that they are in an open environment. It is interesting to see that students felt the immersion at most and felt that their work could be monitored, which is also possible with other information systems as a forensic measure, if they commit misconduct. However, students feeling of being a part of open community may have resulted in shaping their behaviour. However, it is an open research possibility for further study.

*[I feel psychologically aware about my progression being observed by others] [No privacy with my activities] [It is open, so privacy is a big concern]* 

**OFQ2** - Are there any security challenges on using 3D MUVE for your learning, as an overall opinion?

This question let the students to express their opinions on learning environment security. Importantly, the security enforcement of a learning environment is well researched (Weippl, 2005) and a prominent concern for present technology supported learning solutions (Perera et al., 2011a). With respect to the study, this question had the objective of identifying students' trust on existing 3D MUVE security implementations and policies employed. Moreover, students must have a confidence on the learning environment reliability to engage comfortably with the assessed and serious learning tasks. Overall, students showed their lack of awareness on the security and administrative measures available in 3D MULE. This was an important observation, and we consider it for our future research.

*[I believe it is safe and secure] [There can be, but I'm not so sure, I need to learn more] [I'm not sure]* 

**OFQ3** - What are the areas of 3D MULE that should be addressed or improved?

This was a general question to see whether students have different thoughts for improving the learning experience with 3D MULE. There were few interesting thoughts expressed, although following two ideas appear to be undoubtedly correct. First, some students expressed their need on having a private space if the learning activities are not designed for collaboration. Secondly, various concerns were raised on familiarising the 3D MUVE. It is a known fact that 3D MUVE have a steep learning curve (OpenSim, 2007). Importantly, some

3D Multi User Learning Environment Management –

learning tasks.

An Exploratory Study on Student Engagement with the Learning Environment 129

sessions. We observed varying types of student engagements with the environment. As we have explained previously, this indicates student motivation to perform their preferred tasks comfortably. Essentially, such an interaction would be welcomed by any teacher as we all agree that making students being engaged with the learning tasks is a prime challenge. However, we suggest that teachers and learning activity designers should constructively associate these student-preferred 3D MULE functions for making effective and attractive

With the exploratory factor analysis, we have identified that the aspect of managing the 3D MUVE supported learning environments can be achieved through its components: students' self-regulation and system environment management. In fact, the self-regulatory learning practice is a widely researched and well established practice of learning. Pintrich, (2000) defines self-regulation as "an active, constructive process whereby learners set goals for their learning and then attempt to monitor, regulate, and control their cognition, motivation, and behaviour, guided and constrained by their goals and the learning environment contextual features". Students with better self-regulatory skills tend to be more academically motivated and display better learning (Pintrich, 2003). However, most of the previous studies are based on either traditional learning or e-Learning activities; our study brings a new dimension to the norms of self-regulatory learning. Schunk (2005) has suggested that there is a need for more research aimed at improving students' self-regulatory skills as they are engaged in learning and to examine how learning environment contexts affect the amount and type of self-regulation displayed. In our study, we have specifically identified this challenge and worked to overcome it aiming a general scenario of learning with 3D support. Importantly, previous studies on self-regulatory learning mainly considered student practice that helps them achieving their learning objectives with minimum support or drive from others. When it comes to 3D MULE, students face a new set of challenges and opportunities to explore while they are engaged in learning. Prominently, students have to engage actively with the learning environment and progress through various tasks, in contrast to passive interaction on 2D or traditional learning setup. As we suggest, for success, students should not feel their learning engagement and the environment engagement as two different activities; if we can constructively align the learning objectives, learning tasks and assessment practices with the available 3D MULE functions, we can see a significant success on student learning through self-regulation. Furthermore, such alignment would help students to perceive the environment management controls as a

positive construct that enables their preferred learning methods with reliability.

Finally, through the hypothesis testing, we observed that there is a positive influence on student engagement with the learning environment due to the association of self-regulatory and environment management practices. As the obtained values suggest, environment management shows the highest positive impact on increasing the student engagement. This can be mainly due to the students' perception on possible environment and system control mechanisms that relate with their activities. Although, students entertain themselves by various environment engagements such as, content creation, editing land and terraform, and 3D presence with associated actions, they also felt the difficulty of task coordination and unsupportive avatar behaviour during a crucial learning engagement. For example, lecture displays reset whenever an avatar hits the play button, disturbing the other viewers that have been watching. In another instance, if a student's simulation arrangement is too close to another's setup, there can be interferences on communication display spheres. These

students worried on certain challenges they faced due to the 3D MUVE at the early stages of the learning activity. We also realised the importance of providing necessary user guidance and consider it for future work.

*[There should be availability of private space for some activities with data which are secure] [Students need to be trained before they use the virtual world for assessment and laboratory work]* 

**OFQ4** - If the 3D MULE security enhanced, can it affect the rich features and the usability of the system?

This question helped us to observe the student view on implementing various environment management strategies to enhance the system security. As 3D MUVE are designed mainly for entertainment and gameplay, mapping those use cases for a formal learning engagement can be a challenge and requires additional level of security management (Perera et al, 2010, Perera et al., 2011a). However, there is a growing concern on 3D MUVE control as it can affect the intrinsic characteristics and usability of the environment. Students came up with vibrant answers, both supporting and against. We suggest a contextual approach for deciding the required level of security management for the learning needs.

*[Yes, because it might hinder the entire idea of collaborative learning in a virtual environment] [I don't see an issue of security] [It can be!] [May be; but should not implement such measures]* 

**OFQ5** - Any other comment/concern/suggestion about using 3D MUVE for learning

As a final thought, all students who expressed their answers indicated the benefits of using 3D MULE as a teaching and learner support tool. Importantly, their positive comments are highly encouraging and supportive of the future studies on 3D MULE while strengthening our prime objective of facilitating student learning with technology support.

*[I believe it should be handled on a wider scale and it is a great tool for interactive and collaborative learning] [Generally, I like it as an educational methodology] [It is interesting to use these stuffs]* 

In general, this open ended feedback showed some of the significant concerns that students have, which we could not relate directly with the questionnaires. As explained in each of these questions, careful analyses on the student answers helped us to associate those with the statistically identified variables. It was observed that the student expressions were scattered on different functional and systems properties of the 3D MUVE when considered in isolation; however, in the general context, we have been able to incorporate those with the identified variables self-regulation and environment management with suitable adaptations for the future steps of this research.

#### **6. Discussion**

#### **6.1 Study contributions**

We have shown the analysis results of this study previously with reasonable explanations. The first contribution we have is the student engagement observations during the laboratory

students worried on certain challenges they faced due to the 3D MUVE at the early stages of the learning activity. We also realised the importance of providing necessary user guidance

*[There should be availability of private space for some activities with data which are secure] [Students need to be trained before they use the virtual world for assessment and laboratory* 

**OFQ4** - If the 3D MULE security enhanced, can it affect the rich features and the usability of

This question helped us to observe the student view on implementing various environment management strategies to enhance the system security. As 3D MUVE are designed mainly for entertainment and gameplay, mapping those use cases for a formal learning engagement can be a challenge and requires additional level of security management (Perera et al, 2010, Perera et al., 2011a). However, there is a growing concern on 3D MUVE control as it can affect the intrinsic characteristics and usability of the environment. Students came up with vibrant answers, both supporting and against. We suggest a contextual approach for

*[Yes, because it might hinder the entire idea of collaborative learning in a virtual environment]* 

**OFQ5** - Any other comment/concern/suggestion about using 3D MUVE for learning

As a final thought, all students who expressed their answers indicated the benefits of using 3D MULE as a teaching and learner support tool. Importantly, their positive comments are highly encouraging and supportive of the future studies on 3D MULE while strengthening

*[I believe it should be handled on a wider scale and it is a great tool for interactive and* 

In general, this open ended feedback showed some of the significant concerns that students have, which we could not relate directly with the questionnaires. As explained in each of these questions, careful analyses on the student answers helped us to associate those with the statistically identified variables. It was observed that the student expressions were scattered on different functional and systems properties of the 3D MUVE when considered in isolation; however, in the general context, we have been able to incorporate those with the identified variables self-regulation and environment management with suitable adaptations

We have shown the analysis results of this study previously with reasonable explanations. The first contribution we have is the student engagement observations during the laboratory

deciding the required level of security management for the learning needs.

our prime objective of facilitating student learning with technology support.

and consider it for future work.

*[I don't see an issue of security]* 

*collaborative learning]* 

*[It is interesting to use these stuffs]* 

for the future steps of this research.

**6. Discussion** 

**6.1 Study contributions** 

*[May be; but should not implement such measures]* 

*[Generally, I like it as an educational methodology]* 

*[It can be!]* 

*work]* 

the system?

sessions. We observed varying types of student engagements with the environment. As we have explained previously, this indicates student motivation to perform their preferred tasks comfortably. Essentially, such an interaction would be welcomed by any teacher as we all agree that making students being engaged with the learning tasks is a prime challenge. However, we suggest that teachers and learning activity designers should constructively associate these student-preferred 3D MULE functions for making effective and attractive learning tasks.

With the exploratory factor analysis, we have identified that the aspect of managing the 3D MUVE supported learning environments can be achieved through its components: students' self-regulation and system environment management. In fact, the self-regulatory learning practice is a widely researched and well established practice of learning. Pintrich, (2000) defines self-regulation as "an active, constructive process whereby learners set goals for their learning and then attempt to monitor, regulate, and control their cognition, motivation, and behaviour, guided and constrained by their goals and the learning environment contextual features". Students with better self-regulatory skills tend to be more academically motivated and display better learning (Pintrich, 2003). However, most of the previous studies are based on either traditional learning or e-Learning activities; our study brings a new dimension to the norms of self-regulatory learning. Schunk (2005) has suggested that there is a need for more research aimed at improving students' self-regulatory skills as they are engaged in learning and to examine how learning environment contexts affect the amount and type of self-regulation displayed. In our study, we have specifically identified this challenge and worked to overcome it aiming a general scenario of learning with 3D support. Importantly, previous studies on self-regulatory learning mainly considered student practice that helps them achieving their learning objectives with minimum support or drive from others. When it comes to 3D MULE, students face a new set of challenges and opportunities to explore while they are engaged in learning. Prominently, students have to engage actively with the learning environment and progress through various tasks, in contrast to passive interaction on 2D or traditional learning setup. As we suggest, for success, students should not feel their learning engagement and the environment engagement as two different activities; if we can constructively align the learning objectives, learning tasks and assessment practices with the available 3D MULE functions, we can see a significant success on student learning through self-regulation. Furthermore, such alignment would help students to perceive the environment management controls as a positive construct that enables their preferred learning methods with reliability.

Finally, through the hypothesis testing, we observed that there is a positive influence on student engagement with the learning environment due to the association of self-regulatory and environment management practices. As the obtained values suggest, environment management shows the highest positive impact on increasing the student engagement. This can be mainly due to the students' perception on possible environment and system control mechanisms that relate with their activities. Although, students entertain themselves by various environment engagements such as, content creation, editing land and terraform, and 3D presence with associated actions, they also felt the difficulty of task coordination and unsupportive avatar behaviour during a crucial learning engagement. For example, lecture displays reset whenever an avatar hits the play button, disturbing the other viewers that have been watching. In another instance, if a student's simulation arrangement is too close to another's setup, there can be interferences on communication display spheres. These

3D Multi User Learning Environment Management –

Intended Learning Outcomes (ILOs), instead.

for further study considering the outcomes of this research.

candidates for formal and serious educational needs.

practices with high success.

**7. Conclusion** 

**6.3 Future work** 

An Exploratory Study on Student Engagement with the Learning Environment 131

this challenge. Although, we cannot use the student inputs for these questions with the

This research and its findings suggest a number of potential future studies. Some of these can be considered as further extensions of our work. As mentioned above, provision of user guidance on 3D MULE management can be a very valuable future work, since such facility would enable 3D MULE users to experience their learning activities with high confidence and reliability. Moreover, it would reduce the steepness of the 3D MUVE learning curve. This will attract many potential educators to conduct their teaching and learning support activities with 3D MULE. On the other hand, when students and teachers exactly know 3D MULE limitations and suitable functions for their work, they could save a significant amount of their time, which then can be used for achieving productive tasks such as

Developing 3D MULE policy consideration based on the research findings will be the most important further extension of this study. The identified two variables, Self-Regulation and Environment Management, sufficiently influence on the user behaviour and their engagement with the learning environment, as the results indicate. Therefore, the required policy consideration for the 3D MULE Management can be designed based on these two variables. Furthermore, as this study reveals, policy considerations based on student selfregulation and environment management may tend to provide effective policy-based

Various views/concerns mentioned by the students associating privacy and collaboration needs with 3D MULE can be considered for future studies that specialise those areas. Various learning aids and tool support with 3D MUVE can be seen in these days expecting the learning benefits. However, we believe that there is a strong need of empirical evidence on student interaction with those learning aids in 3D MULE, which we strongly recommend

This research has presented several important contributions for the development of 3D Multi User Learning Environments as an effective means of providing engaged student learning. The results and the analysis confirmed that the student self-regulation and system environment management are the two important factors that contribute for the 3D MULE management. Therefore, our research towards developing policy considerations for 3D MULE management will be based on these two factors for its future work. Additionally, we have seen through the analysis that the student self-regulation and the system environment management have positive and significant influence on student engagement with the 3D MUVE. This was an important observation for the researchers who are keen on managing their 3D MUVE supported learning environments. Importantly, this result encourages the policy based management of 3D MULE while enabling those to be considered as successful

Our contribution, through research findings and the experiment observations, would help the other researchers by providing empirical evidence on student engagement with 3D

statistical analysis, the comments provided a significant contribution, qualitatively.

types of problems can be easily solved if we implement appropriate policy considerations through system environment control.

#### **6.2 Study limitations**

Despite all the efforts to make this study accurate as possible, it was affected by the following limitations. Extra efforts and careful analyses were employed to minimise the impact while increasing the accuracy of research models, observed.

Due to the nature of the research objectives, our study sample was limited for a selected set of students. This specialised group of students was chosen, considering their participation in the learning activity designed with 3D MUVE. These students, therefore, have provided their feedback and answers based on their real experiences, which we could validate through the observation of their interaction and the environment changes. In that respect, the mere increase of the sample size from a random student population for the sake of numbers would create a negative influence on the accuracy of results, than having a relatively small, yet accurately representative group of participants. Therefore, we consider the data received as accurate and conclusive though the study had a relatively small sample.

Sample size for the Factor Analysis was another challenge we encountered due to the use of a specialised user group. As we have expressed above, our *N:p* ratio was sufficient to give high reliability on the solution factor model (Costello and Osborne, 2005). Recent studies also suggest that the effect of sample size becomes insignificant if the solution model provides a *stronger* outcome due to the strength of data; in which the strength of data is defined as having 3 or more items that show over 0.7 loadings per factor, and smaller number of factors (MacCallum, *et al.*, 1999). Through a rigorous empirical study on top ranked journal articles that had factor analysis, Lingard and Rowlinson (2005) have proven this observation and recommend a new strategy to validate factor results for smaller sample sizes; higher N:p ratio with stronger data. Our exploratory analysis indicates only 2 factors with each having more than 3 items over .7 loadings. This was suggested in Osborne and Costello (2004) as well. More importantly, irrespective of the sample size impact on the results, the Kaiser-Meyer-Olkin (KMO) and Bartlett Test of Sphericity for the data set showed a very strong and significant (p<.001) result, indicating the correctness of Factor Analysis with PCA over the data sample. This further validates the Factor Analysis results and the stronger outcome.

The questions used in the questionnaire were appropriately designed to capture student views, although they have yet to be examined for psychometric measures. Unfortunately, it is a challenge to find a widely accepted standard set of psychometric measures, in particular, for 3D MULE user evaluation and learning activities, as the field of study is still growing. We would welcome the researchers to consider this aspect in their future research.

Another challenge we had was the accurate capturing of the user opinions. Likert scales have been widely used and are the standard mechanism for these research needs. However, users only answer the asked questions or consider the given statements; there can be different opinions with individual users, which we may ignore if we confined the user input to the asked questions. Since this was an exploratory study, we welcomed different aspects that student wish to provide; hence, included a set of open-ended questions to overcome this challenge. Although, we cannot use the student inputs for these questions with the statistical analysis, the comments provided a significant contribution, qualitatively.

#### **6.3 Future work**

130 Applications of Virtual Reality

types of problems can be easily solved if we implement appropriate policy considerations

Despite all the efforts to make this study accurate as possible, it was affected by the following limitations. Extra efforts and careful analyses were employed to minimise the

Due to the nature of the research objectives, our study sample was limited for a selected set of students. This specialised group of students was chosen, considering their participation in the learning activity designed with 3D MUVE. These students, therefore, have provided their feedback and answers based on their real experiences, which we could validate through the observation of their interaction and the environment changes. In that respect, the mere increase of the sample size from a random student population for the sake of numbers would create a negative influence on the accuracy of results, than having a relatively small, yet accurately representative group of participants. Therefore, we consider the data received as accurate and conclusive though the study had a relatively small sample. Sample size for the Factor Analysis was another challenge we encountered due to the use of a specialised user group. As we have expressed above, our *N:p* ratio was sufficient to give high reliability on the solution factor model (Costello and Osborne, 2005). Recent studies also suggest that the effect of sample size becomes insignificant if the solution model provides a *stronger* outcome due to the strength of data; in which the strength of data is defined as having 3 or more items that show over 0.7 loadings per factor, and smaller number of factors (MacCallum, *et al.*, 1999). Through a rigorous empirical study on top ranked journal articles that had factor analysis, Lingard and Rowlinson (2005) have proven this observation and recommend a new strategy to validate factor results for smaller sample sizes; higher N:p ratio with stronger data. Our exploratory analysis indicates only 2 factors with each having more than 3 items over .7 loadings. This was suggested in Osborne and Costello (2004) as well. More importantly, irrespective of the sample size impact on the results, the Kaiser-Meyer-Olkin (KMO) and Bartlett Test of Sphericity for the data set showed a very strong and significant (p<.001) result, indicating the correctness of Factor Analysis with PCA over the data sample. This further validates the Factor Analysis results

The questions used in the questionnaire were appropriately designed to capture student views, although they have yet to be examined for psychometric measures. Unfortunately, it is a challenge to find a widely accepted standard set of psychometric measures, in particular, for 3D MULE user evaluation and learning activities, as the field of study is still growing. We would welcome the researchers to consider this aspect in their future research. Another challenge we had was the accurate capturing of the user opinions. Likert scales have been widely used and are the standard mechanism for these research needs. However, users only answer the asked questions or consider the given statements; there can be different opinions with individual users, which we may ignore if we confined the user input to the asked questions. Since this was an exploratory study, we welcomed different aspects that student wish to provide; hence, included a set of open-ended questions to overcome

impact while increasing the accuracy of research models, observed.

through system environment control.

**6.2 Study limitations** 

and the stronger outcome.

This research and its findings suggest a number of potential future studies. Some of these can be considered as further extensions of our work. As mentioned above, provision of user guidance on 3D MULE management can be a very valuable future work, since such facility would enable 3D MULE users to experience their learning activities with high confidence and reliability. Moreover, it would reduce the steepness of the 3D MUVE learning curve. This will attract many potential educators to conduct their teaching and learning support activities with 3D MULE. On the other hand, when students and teachers exactly know 3D MULE limitations and suitable functions for their work, they could save a significant amount of their time, which then can be used for achieving productive tasks such as Intended Learning Outcomes (ILOs), instead.

Developing 3D MULE policy consideration based on the research findings will be the most important further extension of this study. The identified two variables, Self-Regulation and Environment Management, sufficiently influence on the user behaviour and their engagement with the learning environment, as the results indicate. Therefore, the required policy consideration for the 3D MULE Management can be designed based on these two variables. Furthermore, as this study reveals, policy considerations based on student selfregulation and environment management may tend to provide effective policy-based practices with high success.

Various views/concerns mentioned by the students associating privacy and collaboration needs with 3D MULE can be considered for future studies that specialise those areas. Various learning aids and tool support with 3D MUVE can be seen in these days expecting the learning benefits. However, we believe that there is a strong need of empirical evidence on student interaction with those learning aids in 3D MULE, which we strongly recommend for further study considering the outcomes of this research.

#### **7. Conclusion**

This research has presented several important contributions for the development of 3D Multi User Learning Environments as an effective means of providing engaged student learning. The results and the analysis confirmed that the student self-regulation and system environment management are the two important factors that contribute for the 3D MULE management. Therefore, our research towards developing policy considerations for 3D MULE management will be based on these two factors for its future work. Additionally, we have seen through the analysis that the student self-regulation and the system environment management have positive and significant influence on student engagement with the 3D MUVE. This was an important observation for the researchers who are keen on managing their 3D MUVE supported learning environments. Importantly, this result encourages the policy based management of 3D MULE while enabling those to be considered as successful candidates for formal and serious educational needs.

Our contribution, through research findings and the experiment observations, would help the other researchers by providing empirical evidence on student engagement with 3D

3D Multi User Learning Environment Management –

An Exploratory Study on Student Engagement with the Learning Environment 133

Keskitalo, T., Pyykkö, E., & Ruokamo, H. (2011), Exploring the Meaningful Learning of Students in Second Life, *Educational Technology & Society*, 14(1), 16–26 Kirriemuir J., (2010), An autumn 2010 "snapshot" of UK Higher and Further Education

Kolb, D. A., Boyatzis, R.E. and Mainemelis, C., (2001), Experiential Learning Theory:

Lingard, H.C. and Rowlinson, S. (2005) Sample size in factor analysis: why size matters,

McCaffery J., Miller A., and Allison C., (2011), Extending the use of virtual worlds as an

Messinger P., Xin G., Stroulia E., Lyons K., Smirnov K., and Bone M. (2008), On the

Moos, D.C., and Azevedo, R. (2007), Self-regulated learning with hypermedia: The role of prior domain knowledge, *Contemporary Educational Psychology*, 33(2):270-298

Oliver, I.A., Miller, A.H.D. and Allison, C. (2010), Virtual worlds, real traffic: interaction and

Osborne, J.W. and Costello, A.B., (2004), Sample size and subject to item ratio in principal components analysis, *Practical Assessment, Research & Evaluation*, 9(11) Perera, I., Allison C., Nicoll, J. R. and T. Sturgeon (2009), Towards Successful 3D Virtual

Perera, I., Allison C. and Miller A. (2010), A Use Case Analysis for Learning in 3D MUVE: A Model based on Key e-Learning Activities, *5th ICVL*, Romania, 114-120 Perera, I., Allison, C., McCaffery J., and Miller A. (2011a), Towards Effective Blended

Learning, *3rd Computer Supported Education - CSEDU2011*, INSTICC, 2, 46-55 Perera, I., Allison, C., and Miller A. (2011b), Policy Considerations for Managing 3D Multi

Pintrich, P. (2000), The role of goal orientation in self-regulated learning. In M. Boekaerts, P.

Pintrich P. (2003), A motivational science perspective on the role of student motivation in

Schunk D.H. (2005), Commentary on self-regulation in school contexts, *Learning and* 

Pintrich, & M. Zeidner (Eds.), *Handbook of self-regulation*, 452–502

learning and teaching contexts, *Educational Psychology*, 95, 667–686.

SCQF (2007), Scottish Credit and Qualifications Framework, [available at]

educational platform - Network Island: An advanced learning environment for

Relationship between My Avatar and Myself, *Journal of Virtual Worlds Research*, 1(2),

Learning – A Case Study on Teaching Human Computer Interaction, *In Proceedings* 

Learning with 3D MUVE - An Analysis of Use Case Implementations for 3D MUVE

User Learning Environments – Achieving Usability and Trust for Learning, *6th*

 http://rec.hku.hk/steve/MSc/factoranalysisnoteforstudentresourcepage.pdf MacCallum, R.C., Widaman, K.F., Zhang, S. & Hong, S., (1999), Sample size in factor

teaching Internet routing algorithms, *3rd CSEDU*, INSTICC, 1

Nunnally, J.C., (1978), Psychometric Theory (2nd ed.) McGraw Hill, New York

adaptation, 1st *ACM SIGMM - MMSys'10*, 306-316

*of 4th ICITST-2009, IEEE*, 159-164

*ICVL*, 106-112

*Instruction*, 15(2), 173-177

http://www.scqf.org.uk/The%20Framework/

Previous Research and New Directions, J. Sternberg and L. Zhang, (Eds.): *In Perspectives on Thinking, Learning and Cognitive Styles*, Lawrence Erlbaum, 227

developments in Second Life, *Virtual World Watch, Eduserv*,

Linden Labs (2003), Second Life, http://www.secondlife.com

analysis, *Psychological Methods*, 4, 84-99

1-17

pp.1–6, [accessed in] December 2011, [available at]

MULE. As the empirical evidence from research on 3D MULE is a growing need, this would open a new research dialogue among researchers who partake in the development of technology supported learning. We are committed to extending this work to enhance the policy based management of 3D MULE and evaluate its impact with a broader perspective. With that we invite frontiers in educational technologies to further research and strategically associate the study outcomes with existing learning infrastructures, in a blended manner.

#### **8. Acknowledgment**

This research is supported by the UK Commonwealth Scholarships and the Scottish Informatics and Computer Science Alliance (SICSA). Second Life region rental was supported in part by the University of St Andrews Fund for Innovations in Learning, Teaching and Assessment (FILTA). The Higher Education Academy (HEA) UK supported part of the work on OpenSim. A special thank goes to the INTECH Open for their generous scholarship support to meet the costs associated with the publication of this article.

## **9. References**


MULE. As the empirical evidence from research on 3D MULE is a growing need, this would open a new research dialogue among researchers who partake in the development of technology supported learning. We are committed to extending this work to enhance the policy based management of 3D MULE and evaluate its impact with a broader perspective. With that we invite frontiers in educational technologies to further research and strategically associate the study outcomes with existing learning infrastructures, in a blended manner.

This research is supported by the UK Commonwealth Scholarships and the Scottish Informatics and Computer Science Alliance (SICSA). Second Life region rental was supported in part by the University of St Andrews Fund for Innovations in Learning, Teaching and Assessment (FILTA). The Higher Education Academy (HEA) UK supported part of the work on OpenSim. A special thank goes to the INTECH Open for their generous

Allison, C., Miller, A., Getchell, K., and Sturgeon, T., (2008): Exploratory Learning for Computer Networking, *Advances in Web Based Learning – ICWL*, 331-342, Allison, C., Miller A., Sturgeon T., Perera I. and McCaffery J., (2011), The Third Dimension

Biggs, J. (1996), Enhancing Teaching through Constructive Alignment, *Higher Education*,

Bronack, S., Sanders R., Cheney A., Riedl R., Tashner J., and Matzen N. (2008), Presence

Burkle, M. and Kinshuk (2009), Learning in Virtual Worlds: The Challenges and Opportunities, *International Conference on Cyber Worlds, CW'09, IEEE*, pp.320-327 Cohen J. (1988), *Statistical Power Analysis for the Behavioural Sciences* (2nd Ed.), Academic Press Costello, A.B. & Osborne, J.W., (2005), Best practices in exploratory factor analysis: four

Dabbagh, N. and Kitsantas A. (2004), Supporting Self-Regulation in Student-Centred Webbased Learning Environments, *International Journal of e-Learning*, 2(4), 40-47 Dalgarno, B., Bishop, A., Adlong, W. and Bedgood Jr., D., (2009), Effectiveness of a Virtual

de Freitas S., Rebolledo-Mendez G., Liarokapis F., Magoulas G., and Poulovassilis A. (2010),

Getchell, K., Miller, A., Nicoll, R., Sweetman, R., and Allison, C. (2010), Games

*Journal of Teaching and Learning in Higher Education*, 20(1), 59-69

in Open Learning, 41st ASEE/IEEE Frontiers in Education, IEEE Press, T2E-1 –

Pedagogy: Teaching and Learning in a 3D Virtual Immersive World, *International* 

recommendations for getting the most from your analysis, *Practical Assessment,* 

Laboratory as a preparatory resource for Distance Education chemistry students,

Learning as immersive experiences: Using the four-dimensional framework for designing and evaluating immersive learning experiences in a virtual world, *British* 

Methodologies and Immersive Environments for Virtual Fieldwork, *IEEE* 

scholarship support to meet the costs associated with the publication of this article.

**8. Acknowledgment** 

**9. References** 

T2E-6

32(3), 347-364

*Research & Evaluation*, 10(7):1-9

*Computers & Education*, 53(3), 863-865,

*Journal of Educational Technology*, 41(1), 69-85

Field A., (2006), *Discovering Statistics using SPSS*, 2nd Ed., SAGE, London

*Transactions on Learning Technologies*, 3(4), 281-293


http://rec.hku.hk/steve/MSc/factoranalysisnoteforstudentresourcepage.pdf


**7** 

*Brasil* 

**Methodology for the Construction of a** 

 **Virtual Environment for the Simulation** 

Tadeu Augusto de Almeida Silva and Oscar Luiz Monteiro de Farias

In the so-called real world, any given process occurs simultaneously with a myriad of other processes. All these processes take place on a *continuum* of mass, energy and time *(the world)*. All things in this *continuum* are interlinked in some way. Defining a process as such is in itself already an act of abstraction. One way to study a process is to split it into three entities: the observer, the process itself and the rest of the continuum*.* The observer usually simplifies a process in his or her attempt to understand it. The observer considers some factors and variables as taking part in the process while ignoring or excluding others. All the variables involved in a given process form a type of microcosm. Using the scientific method, researchers aim to determine laws that govern the interplay among certain entities. To this

A virtual environment is a computer environment. It represents a subset of the real world, where models of real world variables, processes and events are projected onto a threedimensional space. The creation of a virtual environment is an important tool for simulating certain critical processes, especially those in which human beings or things are likely to suffer irreversible or long term damage. This methodology combines a representation of the three-dimensional space with geographic references, the agent-based models, and an adequate representation of time, and aims to construct virtual environments able to simulate phenomena or processes. The three-dimensional geo-referenced representations of space can either be the spatial representation of traditional geographical information systems (GIS), or the representation adopted by Google MapsTM (Google, 2011). Adding autonomous agents to these spatial representations allows us to simulate events, measure any variable, obtain a possible spatial distribution of people and objects, estimate any environmental impacts,

build alternative scenarios and train staff to deal with these critical processes.

**(ABS) and a geographical information system (GIS)** 

**2. The architecture of a virtual environment using an agent-based system** 

Simulation is the process of building a model of a real system and conducting experiments with this model, in order to understand the behavior of the system and / or to evaluate how

**1. Introduction** 

it operates.

end, different models and techniques may be used.

 **of Critical Processes** 

*Instituto de Radioproteção e Dosimetria(IRD), Universidade do Estado do Rio de Janeiro(UERJ)* 

