**4. Results**

38 Applications of Virtual Reality

a task A11 Outlines and describes the plan/brief for a design

completed

members

acting

information B21 Gives updates and reports key events

B24 Clearly documents design

Assessing capabilities B41 Asks for assistance

B43

B53 Debriefs

information

justification B34 Gives clear instructions

standard

B52 Reassures / encourages

information

Supporting others B51 Acknowledges concerns of others

Prioritising tasks A21 Assigns priority to design tasks to be completed

A12 Reviews a design after changes are made A13 Describes actions required once design is

A22 Prioritises the segments within design tasks

A32 Cross checks the completion of design tasks

B11 Confirms roles and responsibilities of team

B13 Considers requirements of others before

B14 Co-operates with others to achieve goals

B23 Communicates design plans and relevant

B31 Is appropriately and necessarily assertive

A31 Follows design protocols and briefs

A41 Identifies and allocates resources A42 Allocates tasks to team members A43 Requests additional resources

B12 Discusses design with others

B22 Confirms shared understanding

B32 Takes appropriate leadership

B33 States case for instruction and gives

B42 Asks team member about experience

B54 Anticipates when others will need

Notices that a team member does not complete a task to an appropriate

Generic skills Sub-skills Code Observable Behaviour

Planning or preparing

Providing direction and maintaining standards for the task

Identifying and utilising resources

Co-ordinating activities with team

members

Exchanging

Using authority and assertiveness

Task Management

Team Working

#### **4.1 Generic skills**

The generic skill *Shared Situational Awareness* increased significantly (*F*(2, 8) = 4.903, *p* < .05). The Within-Subject Contrasts test indicated a significant difference between face-to-face and whiteboard conditions (*F*(1, 4) = 19.478, *p* < .05).

For the skill of *Decision Making*, there was a significant decrease (*F*(2, 8) = 42.431, *p* < .001) in frequency as the design conditions moved from low to high bandwidth conditions. The Within-Subject Contrasts test demonstrated a significant difference between both the faceto-face to whiteboard and whiteboard to 3D virtual world (*F*(1, 4) = 120.274, *p* < .001 and *F*(1, 4) = 8.685, *p* < .05 respectively).

For the skill of *Task Management*, the decrease in frequency from face-to-face to whiteboard approached significance (*F*(1, 4) = 4.799, *p* > .1).

#### **4.2 Observable behaviours**

The following five observable skills (see Figure 2) were significantly affected by the experimental conditions:

Changing Skills in Changing Environments: Skills Needed in Virtual Construction Teams 41

In addition, two other statistical results from the generic skill Team Working are worth

 B23 ("Communicates design plans and relevant information" to relevant members). The Within-Subjects Contrasts test indicates that the mean frequency of B23 reduced significantly (*F*(1, 4) = 23.774 *p* < .05) between the *face-to-face* and *whiteboard* conditions. B52 ("Reassures/Encourages") was the only observable behaviour that approached significance (*F*(2, 8) = 3.462 *p* < .1). The decrease in frequency of this behaviour between *whiteboard* and *3D virtual world* (*F*(1, 4) = 5.956 *p* < .1) is also approaching significance. Changes in the incidence of the remaining observable behaviours were non-significant, in

This study examined the generic skills of five design teams in three settings: face-to-face and two levels of virtual technology (viz. whiteboard and 3D virtual world). The behaviours underpinning the generic skills designers use during the conceptual stages of a variety of projects were recorded and analysed. The major findings were a significant increase in the frequency of Shared Situational Awareness and a significant decrease in Decision Making as

There was a significant and consistent increase in *Shared Situational Awareness* as the design process moved from low to high bandwidth as well as a significant increase between faceto-face and whiteboard conditions. This generic skill incorporates the sub-skills of gathering information, recognising and understanding as well as anticipating. One of this skill's observable behaviours (C11 "asks for documents and/or information regarding an idea or design") increased significantly as bandwidth increased and also between face-to face and whiteboard conditions. This behaviour is associated with information gathering and involves designers asking questions about a design, a site, an idea or an artefact. Shared goals and shared understandings are considered to be an intrinsic part of the team-building process (Berry, 2011). An increase in the frequency of this behaviour may indicate escalating levels of uncertainty (Gay & Lentini, 1995; Kayworth & Leidner, 2000) and it is conceivable that moving to unfamiliar design environments may have engendered such concerns. Furthermore, designing in 2D is markedly different from designing in 3D. Many designers traditionally work in 2D, and this approach is conveniently facilitated by whiteboard technologies. 3D virtual environments provide additional challenges because few designers have worked in them before. So, not only do 3D environments require designers to exercise their visualisation skills in a more complex way, they require them to use new tools (e.g. avatars, 3D geometric modelling tools etc) to express their conceptual designs. An increase in the incidence of C11 is therefore understandable. From this result, it could be extrapolated that design teams comprised of members from a variety of disciplines would have even greater difficulty in *Shared Situational Awareness*. Berry (2011) has suggested that increased requests for information may be the norm for virtual teamwork, i.e. it may be the normal pattern of communication, reflecting how different the virtual process is from the

noting:-

**5. Discussion** 

some cases due to limited or non-existent data

bandwidth conditions increased.

**5.1 Shared situational awareness** 

 A11 ("Outlines and describes the plan/brief for the design" indicative of *Task Management*). There was a significant decrease (*F*(2, 8) = 9.021, *p* < .05) in the incidence of this behaviour from low to high bandwidth levels. The Within-Subjects Contrasts test indicates that the move from face-to-face to whiteboard was significant (*F*(1, 4) = 7.943, *p* < .05).

Fig. 2. Frequency of significant observable behaviours A11, B21, B33, C11, D11, in 3 conditions


In addition, two other statistical results from the generic skill Team Working are worth noting:-


Changes in the incidence of the remaining observable behaviours were non-significant, in some cases due to limited or non-existent data

#### **5. Discussion**

40 Applications of Virtual Reality

 A11 ("Outlines and describes the plan/brief for the design" indicative of *Task Management*). There was a significant decrease (*F*(2, 8) = 9.021, *p* < .05) in the incidence of this behaviour from low to high bandwidth levels. The Within-Subjects Contrasts test indicates that the move from face-to-face to whiteboard was significant (*F*(1, 4) =

Fig. 2. Frequency of significant observable behaviours A11, B21, B33, C11, D11, in 3 conditions

 B21 ("Gives updates and reports key events", demonstrating *Team Working*). This behaviour increased significantly (*F*(2, 6) = 6.343, *p* < .05) as the design process moved from low to high bandwidth. Furthermore, the difference between *face-to-face* and

 B33 ("States case for instruction and gives justification", also demonstrating *Team Working*). The movement from low to high bandwidth demonstrated a significant decrease in this behaviour (*F*(2, 6) = 5.362, *p* < .05). A significant difference between *whiteboard* and *3D virtual world* was found to be approaching significance (*F*(1, 3) =

 C11 ("Asks for documents and/or information regarding a design" indicating *Shared Situational Awareness*). This increased significantly as the design process moved from low to high bandwidth (*F*(2, 8) = 5.526, *p* < .05). The Within-Subjects Contrasts test showed a significant change (*F*(1, 4) = 15.751, *p* < .05) for the shift from *face-to-face* to

 D11 ("Discusses design options with clients/other designers" demonstrating *Decision-Making*). As the design collaborators shifted from low to high bandwidth, the frequency of the behaviour decreased significantly (*F*(2, 8) = 25.383, *p* < .001). In addition, significant differences were also found between *face-to-face* and *whiteboard* and *whiteboard* and *3D virtual world* (*F*(1, 4) = 46.24, *p* < .05 and *F*(1, 4) = 8.095, *p* < .05,

*whiteboard* conditions was significant (*F*(1, 3) = 16.734, *p* < .05).

7.943, *p* < .05).

5.642, *p* = .098).

respectively).

*whiteboard* conditions.

This study examined the generic skills of five design teams in three settings: face-to-face and two levels of virtual technology (viz. whiteboard and 3D virtual world). The behaviours underpinning the generic skills designers use during the conceptual stages of a variety of projects were recorded and analysed. The major findings were a significant increase in the frequency of Shared Situational Awareness and a significant decrease in Decision Making as bandwidth conditions increased.

#### **5.1 Shared situational awareness**

There was a significant and consistent increase in *Shared Situational Awareness* as the design process moved from low to high bandwidth as well as a significant increase between faceto-face and whiteboard conditions. This generic skill incorporates the sub-skills of gathering information, recognising and understanding as well as anticipating. One of this skill's observable behaviours (C11 "asks for documents and/or information regarding an idea or design") increased significantly as bandwidth increased and also between face-to face and whiteboard conditions. This behaviour is associated with information gathering and involves designers asking questions about a design, a site, an idea or an artefact. Shared goals and shared understandings are considered to be an intrinsic part of the team-building process (Berry, 2011). An increase in the frequency of this behaviour may indicate escalating levels of uncertainty (Gay & Lentini, 1995; Kayworth & Leidner, 2000) and it is conceivable that moving to unfamiliar design environments may have engendered such concerns. Furthermore, designing in 2D is markedly different from designing in 3D. Many designers traditionally work in 2D, and this approach is conveniently facilitated by whiteboard technologies. 3D virtual environments provide additional challenges because few designers have worked in them before. So, not only do 3D environments require designers to exercise their visualisation skills in a more complex way, they require them to use new tools (e.g. avatars, 3D geometric modelling tools etc) to express their conceptual designs. An increase in the incidence of C11 is therefore understandable. From this result, it could be extrapolated that design teams comprised of members from a variety of disciplines would have even greater difficulty in *Shared Situational Awareness*. Berry (2011) has suggested that increased requests for information may be the norm for virtual teamwork, i.e. it may be the normal pattern of communication, reflecting how different the virtual process is from the

Changing Skills in Changing Environments: Skills Needed in Virtual Construction Teams 43

not be acknowledged and / or explored, and that as a consequence, the quality of their solutions may suffer. It is therefore important for designers working in virtual contexts to recognise the potential limitations of their solutions, and to challenge the proposals of their

The generic skill of Task Management demonstrated a decrease between face-to-face and whiteboard conditions that approached significance whilst there was a significant decrease in the behaviour of outlining and describing the plan/brief for the design/s. *Task Management* incorporates the sub-skills of planning or preparing a task, prioritising tasks, providing direction and maintaining standards for the task, and identifying and utilising resources. The management of virtual teams is acknowledged as being challenging (Kayworth & Leidner, 2000) and it may well be that the results apparent here relate to the

*Team Working* skills incorporate co-ordinating activities with team members, exchanging information, using authority and assertiveness, assessing capabilities and supporting others. In demonstrating this generic skill, team members increased the frequency of giving updates and reporting key events significantly; however they significantly less frequently stated the case for instruction or gave justification as they worked in higher bandwidth conditions. The change in condition from face-to-face to whiteboard resulted in significantly more updates and reports of key events, as well as significantly fewer design plans and relevant information being communicated to relevant members. In contrast, the move from whiteboard to 3D virtual environments resulted in some changes that approached significance (fewer reassuring or encouraging comments, and more stating of the case for instruction and giving justification). This skill thus appears to present opportunities for further investigations. There are clearly many factors influencing designers' behaviours and further investigations to distil participants' contributions and interactions should provide

 Whilst the number of interactions analysed was large, the number of design teams analysed was relatively small (5). Each set of design tasks took 3.5 to 4 hours (including training and preparation) and proved challenging to organise. The fact that only five design teams took part is indicative of the difficulties involved in arranging the sessions. Although the number of teams was relatively small, the use of purposive

sampling has permitted an exploration of the diverse nature of design teams. The data were collected under laboratory conditions. Because of confidentiality and logistics, it was not possible to video designers working at their normal place of work, nor was it possible to record their work on real-life design projects. Although the designs the participants were asked to work on were fictitious, they represented realistic design projects. It is difficult to determine the relative differences in complexity

small team size and personal management style of those involved.

The following are the main limitations of this study:

between the five projects provided.

colleagues.

**5.3 Other notable results** 

interesting insights.

**6. Limitations** 

process of working in a face-to-face team. This increase may be due to the challenge of deciphering the ambiguity of remote communication (Nunamaker, et al., 2009). In addition, Berry (2011) reported that social communication in virtual environments tends to occur more slowly at first. Therefore, even if the amount of communication is similar, the rate may be different. Further research into team communication in these environments may elucidate this issue.

Virtual teams have a greater risk of communication breakdown due to the difficulties of establishing shared context of meaning (Bjørn & Ngwenyama, 2009). This breakdown can cause substantial difficulties as team members struggle to communicate and work with each other. This may also increase project delivery risks (Daim, et al., 2012). This increased need to establish a shared awareness suggests that design collaborators became unsure of their interpretation of communication and so requested additional confirmation. We suggest that design collaborators need to supply more detailed descriptions of what they are proposing or attempting to do and continually relate this to the specific task at hand. Additionally, Nunamaker et al (2009) have also recommended having clear rules and expectations when using certain types of technology and also having a clear definition of effective work completion. Virtual environments make it possible to communicate but the efficiency of such interactions and the level of shared understanding between individuals is not always assured. A way to enrich such communications is to use multiple communication channels or modes simultaneously (Gay & Lentini, 1995; Kayworth & Leidner, 2000). Instead of relying on a single mode of communication it is advantageous to support such communication with artefacts, such as sketches, as designers do in face-to-face situations. Verbal commentary is another way to enhance virtual communication. Where these environments support audio communication, verbal commentary and / or explanation provides valuable supplementary support. Berry (2011) suggests that virtual team members should be encouraged to seek out information when misunderstandings occur. We also recommend that multiple modes of communication be used concurrently to increase shared understanding between design team members in virtual conditions.

#### **5.2 Decision making**

There was a significant and consistent decrease in the frequency of *Decision Making* as design processes moved from low to high bandwidth and also between face to face and whiteboard, and between whiteboard and 3D. The sub-skills associated with this generic skill are identifying options, balancing risks and selecting options, and re-evaluating. The behaviour "discusses design options with clients/other designers" demonstrated a similarly significant decrease. The reduced frequency of such interactions suggests that designers using virtual environments more readily accept design proposals as solutions and do not explore as many alternatives as they would have had they been communicating face-to-face. Research on group style characteristics has reported similar findings; virtual teams were described to be less effective at team work, with decision-making being more difficult, resulting in poor decisions (Branson, Clausen, & Sung, 2008). It would seem that, because of the sometimes cumbersome nature of virtual communication, designers working in virtual environments find it more convenient to accept ideas rather than engage in discussions to explore alternative solutions. We therefore speculate that in virtual environments some designers' perspectives may not be offered for discussion, that when they do their ideas may not be acknowledged and / or explored, and that as a consequence, the quality of their solutions may suffer. It is therefore important for designers working in virtual contexts to recognise the potential limitations of their solutions, and to challenge the proposals of their colleagues.

## **5.3 Other notable results**

42 Applications of Virtual Reality

process of working in a face-to-face team. This increase may be due to the challenge of deciphering the ambiguity of remote communication (Nunamaker, et al., 2009). In addition, Berry (2011) reported that social communication in virtual environments tends to occur more slowly at first. Therefore, even if the amount of communication is similar, the rate may be different. Further research into team communication in these environments may

Virtual teams have a greater risk of communication breakdown due to the difficulties of establishing shared context of meaning (Bjørn & Ngwenyama, 2009). This breakdown can cause substantial difficulties as team members struggle to communicate and work with each other. This may also increase project delivery risks (Daim, et al., 2012). This increased need to establish a shared awareness suggests that design collaborators became unsure of their interpretation of communication and so requested additional confirmation. We suggest that design collaborators need to supply more detailed descriptions of what they are proposing or attempting to do and continually relate this to the specific task at hand. Additionally, Nunamaker et al (2009) have also recommended having clear rules and expectations when using certain types of technology and also having a clear definition of effective work completion. Virtual environments make it possible to communicate but the efficiency of such interactions and the level of shared understanding between individuals is not always assured. A way to enrich such communications is to use multiple communication channels or modes simultaneously (Gay & Lentini, 1995; Kayworth & Leidner, 2000). Instead of relying on a single mode of communication it is advantageous to support such communication with artefacts, such as sketches, as designers do in face-to-face situations. Verbal commentary is another way to enhance virtual communication. Where these environments support audio communication, verbal commentary and / or explanation provides valuable supplementary support. Berry (2011) suggests that virtual team members should be encouraged to seek out information when misunderstandings occur. We also recommend that multiple modes of communication be used concurrently to increase shared

There was a significant and consistent decrease in the frequency of *Decision Making* as design processes moved from low to high bandwidth and also between face to face and whiteboard, and between whiteboard and 3D. The sub-skills associated with this generic skill are identifying options, balancing risks and selecting options, and re-evaluating. The behaviour "discusses design options with clients/other designers" demonstrated a similarly significant decrease. The reduced frequency of such interactions suggests that designers using virtual environments more readily accept design proposals as solutions and do not explore as many alternatives as they would have had they been communicating face-to-face. Research on group style characteristics has reported similar findings; virtual teams were described to be less effective at team work, with decision-making being more difficult, resulting in poor decisions (Branson, Clausen, & Sung, 2008). It would seem that, because of the sometimes cumbersome nature of virtual communication, designers working in virtual environments find it more convenient to accept ideas rather than engage in discussions to explore alternative solutions. We therefore speculate that in virtual environments some designers' perspectives may not be offered for discussion, that when they do their ideas may

understanding between design team members in virtual conditions.

elucidate this issue.

**5.2 Decision making** 

The generic skill of Task Management demonstrated a decrease between face-to-face and whiteboard conditions that approached significance whilst there was a significant decrease in the behaviour of outlining and describing the plan/brief for the design/s. *Task Management* incorporates the sub-skills of planning or preparing a task, prioritising tasks, providing direction and maintaining standards for the task, and identifying and utilising resources. The management of virtual teams is acknowledged as being challenging (Kayworth & Leidner, 2000) and it may well be that the results apparent here relate to the small team size and personal management style of those involved.

*Team Working* skills incorporate co-ordinating activities with team members, exchanging information, using authority and assertiveness, assessing capabilities and supporting others. In demonstrating this generic skill, team members increased the frequency of giving updates and reporting key events significantly; however they significantly less frequently stated the case for instruction or gave justification as they worked in higher bandwidth conditions. The change in condition from face-to-face to whiteboard resulted in significantly more updates and reports of key events, as well as significantly fewer design plans and relevant information being communicated to relevant members. In contrast, the move from whiteboard to 3D virtual environments resulted in some changes that approached significance (fewer reassuring or encouraging comments, and more stating of the case for instruction and giving justification). This skill thus appears to present opportunities for further investigations. There are clearly many factors influencing designers' behaviours and further investigations to distil participants' contributions and interactions should provide interesting insights.
