**3.1.1 Method**

*Subjects*. Fifteen undergraduate students, 14 males and 1 female from the Informatics School at the Universidad de Guadalajara were invited to participate. Five groups of triads were formed voluntarily.

*Materials and Task*. The task consisted on the re-arrange of furniture on an apartment sketch to make room for a billiard or a ping-pong table; the group decided which one of them.

*Procedure*. A number of rules with punctuation were given regarding on how to place furniture such as the required space for the playing table, spaces between furniture and restriction on the number of times they could move each piece of furniture. The instructions were given both verbally and in written form.

Participants were allowed to try the application for a while before starting the task in order to get comfortable with its functionality. The time to accomplish the task was restricted to 15 minutes. Sessions were audio recorded.

*Data*. Every student intervention within the environment was recorded in a text log file. The logs content is the user identification; the type of action , i.e. move furniture, point furniture, a change in the point of view of the environment, when speaking to others; and the time the intervention was made in minutes and seconds. Data was manipulated to identify discussion periods and the session stages.

#### **Discussion periods**

78 Virtual Reality and Environments

The avatars' head is another entity that can take four positions to change the user field of view; to the front where the other two peers can be seen, to the right or left to see directly one of the peers, or down to see the workspace –see Figure 3, for that the arrow keys are

And, when the user is speaking a dialogue globe appears near his/her right hand as showed

This first trial was conducted with the aim of modeling a virtual tutor to facilitate

*Subjects*. Fifteen undergraduate students, 14 males and 1 female from the Informatics School at the Universidad de Guadalajara were invited to participate. Five groups of triads were

*Materials and Task*. The task consisted on the re-arrange of furniture on an apartment sketch to make room for a billiard or a ping-pong table; the group decided which one of them.

*Procedure*. A number of rules with punctuation were given regarding on how to place furniture such as the required space for the playing table, spaces between furniture and

in Figure 4., when the user wants to speak he/she needs to press the spacebar key.

collaboration. In the next session, the tutor implementation is discussed.

Fig. 3. The avatar head movements

Fig. 4. The dialogue globe

**3.1.1 Method** 

formed voluntarily.

used.

Discussion periods are important in a collaborative session because when they occur, planes, evaluation and agreements are settled. A number of talking-turns involving most of the group members seems to be an appropriate method for distinguishing them from situations like a simple question-answer interchange, or the statements people working in a group produce alongside their action directed to no one in particular (Heath et al., 1995).

A talking turn, as defined by Jaffe and Feldstein (1970), begins when a person starts to speak alone, and it is kept while nobody else interrupts him/her. For practical effects, in a computer environment with written text communication, the talking turn can be understood as a posted message, and in oral communication as a vocalization.

Discussion periods for these trials were established as when each one of the three group members had at least one talking-turn. Because for automatic speech recognition the end of an utterance is usually measured when a silence pause occurs in the range of 500 to 2000 ms (Brdiczka, Maisonnasse, & Reignier, 2005), and the answer to a question usually goes in a smaller range, around 500 ms (A. Johnson & Leigh, 2001); to determine the end of a discussion period, pauses of silences were considered in the range of three seconds.

#### **Initial-Planning-Implementation-Reviewing stages**

The collaborative stages can be established by nonverbal cues in different ways, although it also has to relay on the specifications of the task and the instructor strategy for the session. For example, the initial phase could be the introduction to the problem within the environment.

In this case, because the task was explained in person, instruction were delivered to participants in written paper and they had an initial session to try the application, the initial stage was expected to be brief, more likely to be used to get the initiative to start. Then, the Planning stage was expected to start almost immediately; to identify it, the first discussion period was used.

The restrictions posted for the objects manipulation makes to expect that participants will not move objects if they have no implementation intention; therefore, the initiation of the stage was determined with the first movement of an object.

Once the group starts to implement, the discussions periods should mean that they are making new plans or changing them, because there is no way to differentiate new plans

The Users' Avatars Nonverbal Interaction in Collaborative Virtual Environments for Learning 81

The team got points when its participants followed the given instructions, but if they did not, their punctuation was decremented. Other way to measure the group effectiveness was by comparing the number of movements required to get each piece of furniture where it was last placed, against the number of movements the team actually made; the more effective use of movements is then represented by a 0 difference. The game table was not taken into account for this effectiveness score because a group could not get to that point. In Table 5, the first and second rows show the score given for correctly following instructions as "Instructions Score", and the effectiveness in moving furniture as "Movements score".

The percentage of time each group spent in each session stage is also presented in Table 5, the row that follows the stage is the discussion time the group spent in that specific stage, and the final row presents the total percentage of time they used for discussion periods.

Regarding a collaborative intelligent tutor, a clear opportunity to intervene is the fourth team presented in Table 5. This team started almost immediately –after 2 seconds, with the implementation and then they had very short discussion periods; through the data, it seems that they worked almost in silence. In the audio tape at some point they commented "– remember that we are not supposed to talk" with apparently no reason and they worked to the end of the task in silence. However, they faked talking, that is, they pressed the talking-

Stages / Group 1 2 3 4 5 Instructions score 39 93 22 -15 -17 Movements score 112 61 33 747 49 Initial 0.012 0.009 0.039 0.002 0.009 Planning 0.227 0.119 0.048 - 0.084 Discussion 0.194 0.115 0.025 - 0.071 Implementation 0.553 0.776 0.854 0.998 0.676 Discussion 0.366 0.574 0.277 0.043 0.178 Reviewing 0.209 0.097 0.058 0.002 0.231 Discussion 0.092 0.097 - - 0.110 Total Discussion 0.651 0.786 0.303 0.043 0.360

Fig. 5. Seeing down to the workspace

turn key probably to bring the others attention.

Table 5. Percentage of time elapsed in session stages

from reviewing those already made through the available nonverbal communication, the discussion periods within the Implementation stage were considered as Reviewing stages.

By the end of the Implementation, a final Reviewing stage to agree on the final details of the task was expected. The collaborative stages were then determined based on data logs as follows:


## **3.1.2 Results**

At a first glance to the data it could be overseen that the pointing mechanism was barely used; the speech content revealed that the users' had to make oral references to areas where there were no furniture because they could not point them. Due to this misconception in the design of the environment, pointing gestures were left out.

The changes in gazes were expected to be used to manage talking-turns. The number of times subjects directed their gaze to their peers while they were talking or listening, was relatively small compared to the number of times they were gazing to the workspace as shown in Table 4. A first attempt to understand gazes was to identify possible problems for the participants not using the mechanism as expected.

The possible identified problems in the experimental application were that when the user was viewing the workspace area, he/she did not receive enough awareness about the other users' gazes −see Figure 5. Users had sometimes to specify verbally whom they were addressing if not to both members. Also, sometimes even if they knew their peers names, they did not know which of the two avatars represented each of them.

An external person was asked to determine through the audio recorders, for each talkingturns interchange whether the students were having an episode in which they were taking decisions, making plans or reviewing one of those, that is, discussion periods. Only two interchanges involving two of the three members had these characteristics and the rest of them included the 43 discussion periods identified following the specifications. That is, almost 96% of the talking-turn interchanges with the three members involved were discussion periods.


Table 4. Number of gazes to the workspace or peers while talking or listening

Fig. 5. Seeing down to the workspace

80 Virtual Reality and Environments

from reviewing those already made through the available nonverbal communication, the discussion periods within the Implementation stage were considered as Reviewing stages. By the end of the Implementation, a final Reviewing stage to agree on the final details of the task was expected. The collaborative stages were then determined based on data logs as

*Planning stage* – starts with the first discussion period and ends when the

*Reviewing stage –* when discussion periods occur during the Implementation stage, and

At a first glance to the data it could be overseen that the pointing mechanism was barely used; the speech content revealed that the users' had to make oral references to areas where there were no furniture because they could not point them. Due to this misconception in the

The changes in gazes were expected to be used to manage talking-turns. The number of times subjects directed their gaze to their peers while they were talking or listening, was relatively small compared to the number of times they were gazing to the workspace as shown in Table 4. A first attempt to understand gazes was to identify possible problems for

The possible identified problems in the experimental application were that when the user was viewing the workspace area, he/she did not receive enough awareness about the other users' gazes −see Figure 5. Users had sometimes to specify verbally whom they were addressing if not to both members. Also, sometimes even if they knew their peers names,

An external person was asked to determine through the audio recorders, for each talkingturns interchange whether the students were having an episode in which they were taking decisions, making plans or reviewing one of those, that is, discussion periods. Only two interchanges involving two of the three members had these characteristics and the rest of them included the 43 discussion periods identified following the specifications. That is, almost 96% of the talking-turn interchanges with the three members involved were

**Gazes While Talking While Listening**  Group Workspace Peers Workspace Peers 1 93 29 172 89 2 270 19 474 36 3 108 4 217 10 5 188 45 369 68

Table 4. Number of gazes to the workspace or peers while talking or listening

*Initial stage* – starts with the session and ends when the Planning stage starts.

*Implementation stage* – starts when participants move the first piece of furniture.

follows:

Implementation stage starts.

design of the environment, pointing gestures were left out.

the participants not using the mechanism as expected.

they did not know which of the two avatars represented each of them.

at the end of it.

**3.1.2 Results** 

discussion periods.

The team got points when its participants followed the given instructions, but if they did not, their punctuation was decremented. Other way to measure the group effectiveness was by comparing the number of movements required to get each piece of furniture where it was last placed, against the number of movements the team actually made; the more effective use of movements is then represented by a 0 difference. The game table was not taken into account for this effectiveness score because a group could not get to that point. In Table 5, the first and second rows show the score given for correctly following instructions as "Instructions Score", and the effectiveness in moving furniture as "Movements score".

The percentage of time each group spent in each session stage is also presented in Table 5, the row that follows the stage is the discussion time the group spent in that specific stage, and the final row presents the total percentage of time they used for discussion periods.

Regarding a collaborative intelligent tutor, a clear opportunity to intervene is the fourth team presented in Table 5. This team started almost immediately –after 2 seconds, with the implementation and then they had very short discussion periods; through the data, it seems that they worked almost in silence. In the audio tape at some point they commented "– remember that we are not supposed to talk" with apparently no reason and they worked to the end of the task in silence. However, they faked talking, that is, they pressed the talkingturn key probably to bring the others attention.


Table 5. Percentage of time elapsed in session stages

The Users' Avatars Nonverbal Interaction in Collaborative Virtual Environments for Learning 83

Based on the participants' nonverbal behavior, the tutor makes recommendations regarding their participation and the expected stage sequence (i.e. Initial–Planning–Implement– Reviewing). The "F" key was activated, and when the participants end the session, one of them has to press it. The PVA messages regarding the stages are five and they are triggered

The facilitator also displays other six messages regarding participation, two for each group member. The PVA verifies the percentage of participation from each member compared to the whole group participation, and applies a formula that implies a tolerance range to an

The applied formulas were extracted from a real life situation (Peña & de Antonio, 2009) by a regression model. When the participants are in the Planning or a Reviewing stage the

Elapsed time *A* (3000 ms) from the beginning of the session in which participants do not initiate either a discussion period or implementation

If they start implementation without having at least one discussion period, which implies they did not make a plan

Elapsed time *B* (3000 ms) without a

When the three of them were doing implementation at the same time

Participants finish the session without having at least one discussion period after they finished the implementation.

discussion period

**Stage Number / Message Triggered when** 

Fig. 7. A message from the IVA

exact equal participation.

under the circumstances described in Table 6.

Initial 0. "First step should be to talk about

Implement 2. "A review of what you have until now is advisable"

Implement 3. "You should try to work as a

Implement 4. "Before leaving the session you

team"

starting the implementation"

should review your outcome"

Table 6. Messages from the facilitator to foster collaboration

formula is based only on the subject talking time as follows:

what to do"

Initial 1. "You should discuss before

This is a very small sample of data, and then it was decided not to treat it statistically. Even though it is worth mentioning, that the best scored groups in following the instructions, were those with the highest percentage of time in discussion periods.

The group 5 low score in following instructions was due to a misunderstanding about the task, they tried to put both gaming tables. With a PVA regarding the task during the session, it is probable that the collaborative process could be more attached to the task results.

This trial was meant to understand, the users' nonverbal behavior in order to model a PVA to facilitate collaboration. In the next session, how the tutor was modeled in this same application is presented.

### **3.2 Modeling the virtual tutor**

The PVA model here propose, as already mentioned, aims to facilitate in time, understanding facilitation as guiding the group process, a collaborative 3D virtual session of a small group of participants. While they synchronously accomplish a task with an openended solution that implies the manipulation of objects, through monitoring their users' avatars NVC cues displayed during their collaborative interaction.

The experimental CVE was modified to implement the PVA and to correct some identified problems (see section 3.1.2). To solve the abovementioned misconception about the pointing mechanism, in this version, the arrow can be placed at some parts of the table, see Figure 6.

Then, to solve the awareness of others change of view when the user was viewing to the workspace, the avatars were shrunk in order to show a biggest part of their faces. For the participants to know whom to address verbally when talking, the name of the participant is now displayed in blue letters near his/her avatar – see Figure 6. Finally, the log files now include milliseconds in the timing.

Fig. 6. Experimental application, pointing to the table

#### **Modeling the facilitator**

The virtual facilitator has no graphical representation within the environment. Because the PVA is not meant to give feedback but in the collaborative process, it was considered that it might not need a body. The PVA advices are delivered via text messages posted at the bottom of the screen in black letters –see Figure 7.

Fig. 7. A message from the IVA

This is a very small sample of data, and then it was decided not to treat it statistically. Even though it is worth mentioning, that the best scored groups in following the instructions,

The group 5 low score in following instructions was due to a misunderstanding about the task, they tried to put both gaming tables. With a PVA regarding the task during the session, it is probable that the collaborative process could be more attached to the task results.

This trial was meant to understand, the users' nonverbal behavior in order to model a PVA to facilitate collaboration. In the next session, how the tutor was modeled in this same

The PVA model here propose, as already mentioned, aims to facilitate in time, understanding facilitation as guiding the group process, a collaborative 3D virtual session of a small group of participants. While they synchronously accomplish a task with an openended solution that implies the manipulation of objects, through monitoring their users'

The experimental CVE was modified to implement the PVA and to correct some identified problems (see section 3.1.2). To solve the abovementioned misconception about the pointing mechanism, in this version, the arrow can be placed at some parts of the table, see Figure 6. Then, to solve the awareness of others change of view when the user was viewing to the workspace, the avatars were shrunk in order to show a biggest part of their faces. For the participants to know whom to address verbally when talking, the name of the participant is now displayed in blue letters near his/her avatar – see Figure 6. Finally, the log files now

The virtual facilitator has no graphical representation within the environment. Because the PVA is not meant to give feedback but in the collaborative process, it was considered that it might not need a body. The PVA advices are delivered via text messages posted at the

were those with the highest percentage of time in discussion periods.

avatars NVC cues displayed during their collaborative interaction.

application is presented.

**3.2 Modeling the virtual tutor** 

include milliseconds in the timing.

**Modeling the facilitator** 

Fig. 6. Experimental application, pointing to the table

bottom of the screen in black letters –see Figure 7.

Based on the participants' nonverbal behavior, the tutor makes recommendations regarding their participation and the expected stage sequence (i.e. Initial–Planning–Implement– Reviewing). The "F" key was activated, and when the participants end the session, one of them has to press it. The PVA messages regarding the stages are five and they are triggered under the circumstances described in Table 6.

The facilitator also displays other six messages regarding participation, two for each group member. The PVA verifies the percentage of participation from each member compared to the whole group participation, and applies a formula that implies a tolerance range to an exact equal participation.


Table 6. Messages from the facilitator to foster collaboration

The applied formulas were extracted from a real life situation (Peña & de Antonio, 2009) by a regression model. When the participants are in the Planning or a Reviewing stage the formula is based only on the subject talking time as follows:

The Users' Avatars Nonverbal Interaction in Collaborative Virtual Environments for Learning 85

About pointing, in Table 8 is presented the number of times the group pointed a piece of furniture or to the table during each stage, the third row correspond to the Reviewing periods during the Implementation stage. Groups 2 and 3, same with the better performance in both collaborative process and the scores about the task, used the pointing mechanism during the Planning and the Reviewing stages, but Group 2, the highest score, pointed

> **Group 1 2 3 4**  Planning 0 6 2 0 Implementation 13 8 10 13 Reviewing 2 14 0 0 Reviewing 0 2 1 0

Gazes were now observed under a different perspective. It was detected that the change of view mechanism was being used, although as mentioned not while the user was talking or speaking, the users usually change their viewpoint repeatedly before doing something else,

Because in real life people change the direction of gaze in a regular fashion notably associated with utterances (Kendon, 1990), it was decide to observe what the users were doing after they changed their viewpoint. For that, only the groups that went through the four stages were included. Results are shown in Table 9 for Groups 1 and 2 of the first trial; Table 10 for Groups 3 and 4 of the first trial; and Table 11 for Groups 2 and 3 of the

Group 1 without IVA Group 2 without IVA Plan Implement Review Plan Implement Review # NA GT # NA GT DP # NA GT # NA GT # NA GT DP # NA GT 4 T P 6 T W 4 T W 17 T W 1 T P 1 T P 5 T W D 6 T W D 9 T P

8 T P 7 T W D 1 T P 1 M D 11 T P 4 T P 1 T W 14 T P 3 T P 7 T W 3 T W D 7 T p 2 T P 1 T W 5 T W 19 -

5 T W 2 T W D 11 T P 2 T W D 13 - 1 T P D 4 T P 2 M D 4 T W D 1 T W 2 T W D 2 T W 3 T W D

Table 9. Groups 1 and 2, of the first trial, gazes and the followed action

5 M 18 T W D 12 T P 2 M 1 T W D 6 T P

during Implementations most of the time during Reviewing periods.

Table 8. Pointing to the furniture and the workspace.

an average of 4.02 movements.

second trial.

1 T P 1 K W

**3.2.1 Results** 

1. subject's participation = −0.711 + ( 7.990 \* percentage of talking time)

When the group is at the Implementation stage the applied formula includes the subject implementation as follows:

2. subject's participation = −1.759 + (6.742 \* percentage of talking time) + (4.392 \* percentage of manipulation time)

If the participant has an under participation (according to the formulas 1 or 2), that is when the result is under 1, this message encouraging him/her to increase his/her participation is sent: "<<participantName>>, you should try to increase your participation".

If the participant has an over participation, when the formula result is more than 3, the sent message is: "<<participantName>>, you should try to involve more your peers".


Table 7. Messages from the facilitator regarding participation

The number of the triggered message corresponds to the user to be encouraged for participation or to diminish it as shown in Table 7; the three users were denominated SS, SA and SB. All the messages, although they have a participant target, are sent to the three group members.

When a message appears on the screen, the group members can agree or disagree with it by pressing the keys "O" for OK to agree or "N" for NO to disagree. Although users are not forced to answer the messages, when they do, a number of actions are taken: the message disappears from the screen; the chronograph of elapsed times is set to 0; the participant times are also set to 0; the participants' answers are included in the log file; and, if at least two of the three participants disagree with the message, it is deactivated, that means it will not appear again.

The log file has now information regarding the facilitator as another actor in the scenario, when the PVA sends a message and which one is sent by its number. There is an 11th message just to say "Good bye".

This second trial followed the same method as the first one, with the next differences:

*Subjects*. Twelve undergraduate students, 13 males from the Informatics School and a female from the Chemistry School at the Universidad de Guadalajara participated. Four groups of triads were formed voluntarily.

*Procedure.* Participants filled out a questionnaire at the end of the session.

*Data*. The starting time is now considered when the three computers are connected and not when the first action takes place as in the first trial.

*Discussion periods* were identified same as in the first trial, when the three group members have had at least one talking-turn, but now the pauses were considered in the range of 2000 ms since the system registers now the milliseconds.

#### **3.2.1 Results**

84 Virtual Reality and Environments

When the group is at the Implementation stage the applied formula includes the subject

2. subject's participation = −1.759 + (6.742 \* percentage of talking time) + (4.392 \* percentage

If the participant has an under participation (according to the formulas 1 or 2), that is when the result is under 1, this message encouraging him/her to increase his/her participation is

If the participant has an over participation, when the formula result is more than 3, the sent

The number of the triggered message corresponds to the user to be encouraged for participation or to diminish it as shown in Table 7; the three users were denominated SS, SA and SB. All the messages, although they have a participant target, are sent to the three group

When a message appears on the screen, the group members can agree or disagree with it by pressing the keys "O" for OK to agree or "N" for NO to disagree. Although users are not forced to answer the messages, when they do, a number of actions are taken: the message disappears from the screen; the chronograph of elapsed times is set to 0; the participant times are also set to 0; the participants' answers are included in the log file; and, if at least two of the three participants disagree with the message, it is deactivated, that means it will

The log file has now information regarding the facilitator as another actor in the scenario, when the PVA sends a message and which one is sent by its number. There is an 11th

*Subjects*. Twelve undergraduate students, 13 males from the Informatics School and a female from the Chemistry School at the Universidad de Guadalajara participated. Four groups of

*Data*. The starting time is now considered when the three computers are connected and not

*Discussion periods* were identified same as in the first trial, when the three group members have had at least one talking-turn, but now the pauses were considered in the range of 2000

This second trial followed the same method as the first one, with the next differences:

*Procedure.* Participants filled out a questionnaire at the end of the session.

1. subject's participation = −0.711 + ( 7.990 \* percentage of talking time)

sent: "<<participantName>>, you should try to increase your participation".

**Message No addressed to with the intention to** 

Table 7. Messages from the facilitator regarding participation

message is: "<<participantName>>, you should try to involve more your peers".

**5 / 6 / 7** SS / SA / SB encourage his/her participation **8 / 9 / 10** SS / SAB/ SB diminish his/her participation

implementation as follows:

members.

not appear again.

message just to say "Good bye".

triads were formed voluntarily.

when the first action takes place as in the first trial.

ms since the system registers now the milliseconds.

of manipulation time)

About pointing, in Table 8 is presented the number of times the group pointed a piece of furniture or to the table during each stage, the third row correspond to the Reviewing periods during the Implementation stage. Groups 2 and 3, same with the better performance in both collaborative process and the scores about the task, used the pointing mechanism during the Planning and the Reviewing stages, but Group 2, the highest score, pointed during Implementations most of the time during Reviewing periods.


Table 8. Pointing to the furniture and the workspace.

Gazes were now observed under a different perspective. It was detected that the change of view mechanism was being used, although as mentioned not while the user was talking or speaking, the users usually change their viewpoint repeatedly before doing something else, an average of 4.02 movements.

Because in real life people change the direction of gaze in a regular fashion notably associated with utterances (Kendon, 1990), it was decide to observe what the users were doing after they changed their viewpoint. For that, only the groups that went through the four stages were included. Results are shown in Table 9 for Groups 1 and 2 of the first trial; Table 10 for Groups 3 and 4 of the first trial; and Table 11 for Groups 2 and 3 of the second trial.


Table 9. Groups 1 and 2, of the first trial, gazes and the followed action

The Users' Avatars Nonverbal Interaction in Collaborative Virtual Environments for Learning 87

In these Tables (9, 10 and 11), for each group the first, second and third columns correspond to the Planning stage, columns 4, 5, 6 and 7th are for the Implementation stage, and the last three columns (8 to 10) are for the Reviewing stage. The first columns (#) of each group correspond to the number of times the user changed his/her point of view. The second column represents the next action (NA) after the viewpoint was changed as follows: "T" for talking, "K" for taking and "M" for moving a piece of furniture. In the third column is the final gaze target (GT) with a "P" when it was a peer and a "W" when it was the workspace. The same description can be applied for the 4, 5 and 6th columns, but the 7th column (DP) contains a "D" when the viewpoint changes were made during a discussion period. In the Reviewing stage, when the next action (NA) is a dash, it means that the user did nothing

Table 12 shows the messages sent by the PVA during the four group sessions. The number identifying the message is in the first column of each group. In the second column, the elapsed time of the session when it was sent, then the stage of the session in which it was sent. Columns 3, 4 and 5th contain the answer of the participant (SS, SA and SB). In the 6th column the tutor assumption regarding to the sent message, if two participants agreed with

1 01:24.428 Imp OK OK NO accepted 9 01:00.824 Pl OK OK OK accepted 8 02:00.292 Imp OK OK OK accepted 8 03:33.946 Imp OK OK - not answered 8 03:04.611 Imp - OK - not answered 5 05:08.022 Imp OK - - not answered 8 04:05.592 Imp OK OK - not answered 5 05:17.445 Imp - - - not answered 8 05:14.902 Imp OK OK - not answered 5 05:24.917 Imp - OK - not answered 8 06:00.018 Imp OK OK - not answered 6 06:04.775 Imp - OK - not answered 8 07:03.245 Imp OK OK - not answered 5 07:00.779 Imp - - - not answered 8 08:01.651 Imp OK OK - not answered 5 08:00.091 Imp - - - not answered 8 09:03.039 Imp OK OK - not answered 6 09:01.539 Imp - OK - not answered 8 10:02.895 Imp OK OK - not answered 6 10:02.972 Imp - - - not answered 8 11:00.661 Imp OK OK - not answered 5 11:23.765 Imp OK NO OK accepted 8 12:00.051 Imp OK OK - not answered 5 12:00.893 Imp NO OK - not answered 8 13:00.719 Imp OK OK - not answered 5 13:05.846 Imp OK - - not answered 8 14:00.299 Imp OK OK - not answered 6 14:00.467 Imp OK OK OK accepted

**Elapsed** 

**Elapsed** 

5 01:00.075 Pl OK NO OK accepted 0 01:19.576 Ini OK OK - not answered 5 03:11.334 Imp OK NO OK accepted 0 03:17.621 Ini OK OK OK accepted 6 04:06.059 Imp - OK OK not answered 1 03:48.618 Imp OK OK OK accepted 6 05:01.626 Imp - NO OK not answered 5 05:02.890 Imp OK OK - not answered 6 06:02.076 Imp OK OK OK accepted 5 06:20.266 Imp - - - not answered 9 07:04.133 Imp - OK OK not answered 9 07:00.936 Imp OK - NO not answered 9 08:04.147 Imp OK OK OK accepted 8 08:03.414 Imp OK OK OK accepted 5 09:08.122 Imp - OK OK not answered 7 10:00.648 Imp OK OK NO accepted 5 10:01.707 Imp OK OK OK accepted 5 11:02.028 Imp OK OK OK accepted 5 12:05.268 Imp OK OK OK accepted 5 13:04.828 Imp OK - - not answered 9 14:02.870 Imp OK OK NO accepted

**Time stg SS SA SB participants** 

**Time stg SS SA SB participants** 

else afterwards. Each user has a different tone of gray.

**Group 1 Group 3** 

**Time stg SS SA SB participants Msg**

**Group 2 Group 4** 

**Time stg SS SA SB participants Msg**

Table 12. Facilitator messages sent during the sessions

**Msg** 

**Msg** 

**Elapsed** 

**Elapsed** 


Table 10. Groups 3 and 4, of the first trial, gazes and the followed action


Table 11. Groups 2 and 3, of the second trial, gazes and the followed action

Group 3 without IVA Group 4 without IVA Plan Impement Review Plan Impement Review # NA GT # NA GT DP # NA GT # NA GT # NA GT DP # NA GT 1 T W 20 K W D 14 - 2 T W 3 T p 15 T P 6 T W 16 T W 25 T p 8 T w 3 M P 3 T W

20 T W 1 T W 1 T P D 6 T P 1 T W 1 T W 2 T P

Group 2 with IVA Group 3 with IVA Plan Impement Review Plan Impement Review # NA GT # NA GT DP # NA GT # NA GT # NA GT DP # NA GT 8 T W 2 M W 3 T w 2 -

> M W 2 T W D K W D 1 T W 3 M W K W D 3 M W K W D 2 M w D

Table 10. Groups 3 and 4, of the first trial, gazes and the followed action

1 T P 2 K W 3 T w 1 T W 2 T W 2 T W

13 T P 2 T P 3 T W 1 M P 2 T W 3 K 1 T W D 1 T P 2 M W 3 T P 1 T p D 2 T W 1 T w D

Table 11. Groups 2 and 3, of the second trial, gazes and the followed action

2 T W 3 M w 2 T W

2 T P 2 T P 3 T W

1 T W

3 T P 3 M w

4 T W 1 -

1 T P D 1 T W 1 T W D 3 T P 7 T W 1 T P

 M P 2 T P M W D 6 T P T W 1 T P T W 5 T P T W 3 T P

> 4 T P 8 T P 4 T W 6 T P 10 T P 4 -

1 T W 1 M W 2 T P

In these Tables (9, 10 and 11), for each group the first, second and third columns correspond to the Planning stage, columns 4, 5, 6 and 7th are for the Implementation stage, and the last three columns (8 to 10) are for the Reviewing stage. The first columns (#) of each group correspond to the number of times the user changed his/her point of view. The second column represents the next action (NA) after the viewpoint was changed as follows: "T" for talking, "K" for taking and "M" for moving a piece of furniture. In the third column is the final gaze target (GT) with a "P" when it was a peer and a "W" when it was the workspace. The same description can be applied for the 4, 5 and 6th columns, but the 7th column (DP) contains a "D" when the viewpoint changes were made during a discussion period. In the Reviewing stage, when the next action (NA) is a dash, it means that the user did nothing else afterwards. Each user has a different tone of gray.

Table 12 shows the messages sent by the PVA during the four group sessions. The number identifying the message is in the first column of each group. In the second column, the elapsed time of the session when it was sent, then the stage of the session in which it was sent. Columns 3, 4 and 5th contain the answer of the participant (SS, SA and SB). In the 6th column the tutor assumption regarding to the sent message, if two participants agreed with


Table 12. Facilitator messages sent during the sessions

The Users' Avatars Nonverbal Interaction in Collaborative Virtual Environments for Learning 89

Related to a balanced participation some observations can be done. Based in the personal messages responded by the target member, some members corrected their participation rate. For example, in Group 2, SA received three messages encouraging his participation, one of them was rejected, and then he received two messages asking him to involve more his peers, a clear change in his rate of participation. This type of change can be also observed, in Groups 3 and 4. Unfortunately, not all members corrected their participation rate, in that same Group 2, SS received two messages encouraging his participation and by the end

A more concerning part could be regarding to the stages. Only four messages were sent, one to Group 1 and three to Group 4. The Group 1 received a message suggesting them to settle a plan before starting the implementation, members SS and SA agreed, but member SB who was the one who started the implementation, disagreed with the message and continued

Group 4 received two messages suggesting to agree on decisions as a starting point. The group answered until the second one and agreed with it, but instead of having a discussion period as expected, they started the implementation. Thus, the facilitator sent a third message encouraging them to get an agreement before implementing, and even that the group agreed with the message, they kept moving furniture instead of having a discussion

As mentioned, the pointing mechanism can be used to monitor whether participants are talking about the task, to understand their involvement on it. In this case, the participants were video recorded and physically observed while they were carrying out the sessions; therefore, they were involved in the task no doubt. In these trials, the intention was its observation in order to incorporate data coming from it to the facilitator in the future, same

In the accomplishment of a task like the one presented here, a mechanism to point objects or some areas is an advantage to identify them quickly; deictic gestures during collaboration in real life are more likely to appear during Planning and Reviewing phases. Although, we are completely aware that this is a very small data sample, in Table 13 can be observed that the Group 2 used it the most during these stages, and it is the one with the better scores regarding the task accomplishment and a more proper use of the collaborative stages. A first thought is to suggest its use during the collaborative session, especially during the Planning

The change of gaze direction was used in a very particular way, see Tables 9, 10 and 11, a number of them (4.02 in average) usually preceded a talking turn, 81% of the times. This behavior may correspond to the fact that people try to get feedback from facial expressions when speaking or listening, but the avatars in the environment do not have them, then the users' change of gazes seem to be as an "announcement" that they were

**Did the facilitator affect the collaborative process?** 

received other two with the same advice.

with it.

as gazes.

period as expected.

**3.2.3 Learned lessons** 

and Reviewing stages.

about to speak.

it, the message is considered as "accepted". When one of the three members did not answer a message, it was considered as "not answered".

The resending of the same message responds, at some extent, to the fact that if the participants did not answer the message, the numbers were not cleared. None message was rejected, but there were many messages that were not answered by the whole group, and some participants ignored most of them.

Table 13 shows the scores each group got for following the instructions, the effectiveness in their furniture movements and the time each group elapsed in the stages.

Two of the four groups did no go through a Planning stage, Group 1 and 4. As can be seen in Table 13, the Group 4 had a very small amount of discussion time; this group did not finish the task.


Table 13. Percentage of time elapsed in session stages with the facilitator

#### **3.2.2 Discussion**

Most of the messages from the facilitator were to try to balance participation, 45 of 49 as can be seen in Table 12. From the 135 expected answers, almost a third (28.8%) did not arrive.

In Group 1, participant SB accepted the first two messages and then ignored the rest of them, but they were all the same message addressed to SS asking to diminish his participation. A similar condition can be found in Group 3 where participant SB ignored 11 messages, 7 of them addressed to encourage SS participation and 3 to encourage SA for participation, while in this same Group 3, SS ignored 7 messages from which 4 were addressed to him.

In summary, and although the answer to the specific question about the PVA that was "How proper do you consider the posted messages were?" in a scale of 1 to 4 got a 2.57 average score, there is not a clear perception from the participants of the PVA.

#### **Did the facilitator affect the collaborative process?**

Related to a balanced participation some observations can be done. Based in the personal messages responded by the target member, some members corrected their participation rate. For example, in Group 2, SA received three messages encouraging his participation, one of them was rejected, and then he received two messages asking him to involve more his peers, a clear change in his rate of participation. This type of change can be also observed, in Groups 3 and 4. Unfortunately, not all members corrected their participation rate, in that same Group 2, SS received two messages encouraging his participation and by the end received other two with the same advice.

A more concerning part could be regarding to the stages. Only four messages were sent, one to Group 1 and three to Group 4. The Group 1 received a message suggesting them to settle a plan before starting the implementation, members SS and SA agreed, but member SB who was the one who started the implementation, disagreed with the message and continued with it.

Group 4 received two messages suggesting to agree on decisions as a starting point. The group answered until the second one and agreed with it, but instead of having a discussion period as expected, they started the implementation. Thus, the facilitator sent a third message encouraging them to get an agreement before implementing, and even that the group agreed with the message, they kept moving furniture instead of having a discussion period as expected.

#### **3.2.3 Learned lessons**

88 Virtual Reality and Environments

it, the message is considered as "accepted". When one of the three members did not answer

The resending of the same message responds, at some extent, to the fact that if the participants did not answer the message, the numbers were not cleared. None message was rejected, but there were many messages that were not answered by the whole group, and

Table 13 shows the scores each group got for following the instructions, the effectiveness in

Two of the four groups did no go through a Planning stage, Group 1 and 4. As can be seen in Table 13, the Group 4 had a very small amount of discussion time; this group did not

their furniture movements and the time each group elapsed in the stages.

Table 13. Percentage of time elapsed in session stages with the facilitator

Stages / Group 1 2 3 4 Score -47 97 41 -18 Effectiveness 288 160 55 100

Initial 0.098 0.066 0.065 0.254 Planning - 0.219 0.126 - Discussion - 0.090 0.081 - Implementation 0.872 0.630 0.753 0.746 Discussion 0.064 0.282 0.230 0.077 Reviewing 0.030 0.085 0.056 - Discussion 0.002 0.064 0.056 - Total Discussion 0.066 0.397 0.367 0.077

Most of the messages from the facilitator were to try to balance participation, 45 of 49 as can be seen in Table 12. From the 135 expected answers, almost a third (28.8%) did not arrive.

In Group 1, participant SB accepted the first two messages and then ignored the rest of them, but they were all the same message addressed to SS asking to diminish his participation. A similar condition can be found in Group 3 where participant SB ignored 11 messages, 7 of them addressed to encourage SS participation and 3 to encourage SA for participation, while in this same Group 3, SS ignored 7 messages from which 4 were

In summary, and although the answer to the specific question about the PVA that was "How proper do you consider the posted messages were?" in a scale of 1 to 4 got a 2.57

average score, there is not a clear perception from the participants of the PVA.

a message, it was considered as "not answered".

some participants ignored most of them.

finish the task.

**3.2.2 Discussion** 

addressed to him.

As mentioned, the pointing mechanism can be used to monitor whether participants are talking about the task, to understand their involvement on it. In this case, the participants were video recorded and physically observed while they were carrying out the sessions; therefore, they were involved in the task no doubt. In these trials, the intention was its observation in order to incorporate data coming from it to the facilitator in the future, same as gazes.

In the accomplishment of a task like the one presented here, a mechanism to point objects or some areas is an advantage to identify them quickly; deictic gestures during collaboration in real life are more likely to appear during Planning and Reviewing phases. Although, we are completely aware that this is a very small data sample, in Table 13 can be observed that the Group 2 used it the most during these stages, and it is the one with the better scores regarding the task accomplishment and a more proper use of the collaborative stages. A first thought is to suggest its use during the collaborative session, especially during the Planning and Reviewing stages.

The change of gaze direction was used in a very particular way, see Tables 9, 10 and 11, a number of them (4.02 in average) usually preceded a talking turn, 81% of the times. This behavior may correspond to the fact that people try to get feedback from facial expressions when speaking or listening, but the avatars in the environment do not have them, then the users' change of gazes seem to be as an "announcement" that they were about to speak.

The Users' Avatars Nonverbal Interaction in Collaborative Virtual Environments for Learning 91

This information could be used in a number of ways such as to establish nets of communication or to oversee some participants' collaborative characteristics such as being implementers or leaders. For example, it has been found that talkative group members are more likely to be task leaders (Stein & Heller, 1979), and to receive more gazes and send less

Base on the nonverbal behavior of the users' avatars in a CVE for learning, an IVA was modeled within an experimental application with the intent to scaffold the collaborative process. The model used only two NVC cues, talking turns and artifacts manipulation, to give two types of advices: one regarding a balance in the group members' participation rates in both talk and implementation; and, the other regarding an expected sequence in the Plan-

Two trials were presented, the first without the facilitator or IVA and the second one with it. In the second trial, the observation of other two NVC cues was conducted, deictic gestures

Although in this chapter only nonverbal behavior took part in the facilitator modeling, our final intention is to incorporate the scheme to a verbal analysis, an example, can be found in (Peña, Aguilar, & de Antonio, 2010). In trying to avoid a restricted interface like the Sentence Opener approach, the analysis in (Casillas & Daradoumis, 2009) will be adapted to the

How people nonverbally behave in graphical environment through their avatars and how they will adapt the CVE facilitations for that, are big open issues. The analysis in here was narrowed to collaborative interaction during the accomplishment of a task in a small group through only a few nonverbal communication cues, barely a small brushstroke of what is

Aguilar, R., Gómez, G., Moo, F., Uicab, R., & Tuyub, R. (2010). A model based on intelligent

Argyle, M. (1990). *Bodily communication* (2nd ed.). Madison, Conn.: International Universities

Bailenson, J. N., Blascovich, J., Beall, A. C., & Loomis, J. (2003). Interpersonal distance in immersive virtual environments. *Personality and Social Psychology, 29*, 819-833.

Bergstrom, T., & Karahalios, K. (2007). Conversation clock: Visualizing audio patterns in co-

Bales, R. F. (1970). *Personality and interpersonal behavior*. New York: Holt.

virtual environment to assist the learning of the programming and mathematics. Proceedings of the International Conference on Education and New Learning Technologies (Edulearn 2010). Barcelona, Spain. Gómez, L., Martí, D. & Cande, I.

and gazes, while some indications on this regard were pointed out.

than their peers (Peña & de Antonio, 2009).

suggested as a complete area for research.

(Eds.). pp. 2564-2571.

located groups. *HICSS,* 78.

**4. Conclusions and future work** 

Implement-Review stages.

model.

**5. References** 

Press.

In the Planning stage, only one time the action following to the gaze changes was pointing. The final gaze target during the Planning stage to the peers was 43%, in the Implementation stage only 15%, and in the Reviewing stage, it was the 71 %. In the Reviewing stage, the followed action to the changes in the viewpoint, in 5 of the 34 times was the end of the session, this is consistent with a last look to verify what had been done. The change of gaze that ended in a moving action may be because of the user trying to see the workspace from other perspectives. Curiously, Group 2 from the first trial without facilitator, used these changes of gaze by the end of the session during the Reviewing stage, which imply head movements on their avatars, to say yes or not to the others.

In reserve of confirming these observations with a biggest sample of data, they may represent an adaptation of the users' nonverbal behavior facilities in a CVE, as awareness of their interaction with others.

Regarding the facilitator, it seems that the messages were sent very frequently, see column 2 in Table 12, it might be better to spread them, especially those regarding participation rates; also, statistics should be cleaned once a message is triggered regardless the users' answer to avoid repetitions. This, as an attempt of improving the number of answered messages, without forcing the participants to do so.

Other method to make the facilitator advices more acceptable could be to present them in a more attractive way like changing the color of the letters, or maybe in an imperative form, to give them via voice, or to give the facilitator a body.

Finally, following the logs files a kind of story can be tell about what is going on in the environment, the next lines were composed based on them and it correspond to a SS user:

Results from Loading: keyStrokeLog - 1.txt SS(1) Look at Both at 9:40:29 Look at Nobody at 9:40:29 Look at SB at 9:40:30 Look at SA at 9:40:31 Talk to SA at 9:40:31 Look at SB at 9:40:32 Talk to SB at 9:40:32 Talk to SB at 9:40:35 Talk to SB at 9:40:42 ... Take Dining Table at 9:44:14 Talk to SB at 9:44:32 Move Dining Table at 9:44:43 Move Dining Table at 9:44:44 Move Dining Table at 9:44:45 Move Dining Table at 9:44:46 … Talk to SB at 9:44:52 Talk to SB at 9:44:58...

This information could be used in a number of ways such as to establish nets of communication or to oversee some participants' collaborative characteristics such as being implementers or leaders. For example, it has been found that talkative group members are more likely to be task leaders (Stein & Heller, 1979), and to receive more gazes and send less than their peers (Peña & de Antonio, 2009).
