**3. Generic skills**

32 Applications of Virtual Reality

Our studies were part of a project which examined the use of information and computer technologies (ICTs) to facilitate design / construction team interactions. They were funded by the Australian Cooperative Research Centre for Construction Innovation (Maher, 2002) and focused on the early stages of design / construction collaboration where designs for a building are created, developed and revised. Three aspects of collaboration in virtual environments were investigated: (i) the technological processes that enable effective collaboration using these technologies; (ii) the models that allow disciplines to share their views in a synchronous virtual environment; (iii) the generic skills used by individuals and teams when engaging with high bandwidth ICT. The last strand of these investigations was investigated by the authors and is reported on here. Details of the other strands of this project may be found at the project website (Maher, 2002) and other publications (Bellamy, Williams, Sher, Sherratt, & Gameson, (2005) and Sherratt, Sher, Williams, & Gameson, 2010).

There are numerous definitions of teams. For this paper teams are defined as a cluster of two or more people usually occupying different roles and skill levels that interact "adaptively, interdependently, and dynamically towards a common and valued goal" (Salas, Shawn Burke, & Cannon-Bowers, 2000, p. 341). At present the term "virtual teams" is used by different authors to mean different things. A more detailed exploration of the various facets of virtual teams is provided by Dubé and Paré (2004) and is summarised in Table 1. A number of other researchers have outlined the characteristics of or factors relating to virtual teams e.g. Berry (2011); Schumacher, Cardinal and Bocquet (2009) as well as the

reliance

variety

experience

membership

interdependence

Members' ICT Proficiency High Low

Team size Small Large

Task or project duration Long term Short term

Members' assignments Full-time Part-time

LOW <> HIGH

Local Global

High reliance

Low variety

No

Fluid membership

High

Homogeneous Heterogeneous

experience

interdependence

CHARACTERISTICS DEGREE OF COMPLEXITY

Degree of reliance on ICT Low

ICT availability High

Prior shared work experience Extensive

Membership stability Stable

Task interdependence Low

Geographic dispersion (physical proximity)

Cultural diversity (national, organizational,

professional)

Table 1. Key Characteristics of Virtual Teams

**2. Virtual teamwork** 

…related to the basics of virtual teamwork

…related to

complexity of virtual teamwork

the

There is still much discussion about the core set of knowledge, skills and attitudes that constitute teamwork (Salas, et al., 2000). We sought to contribute to this debate by identifying the skills that transferred from a traditional face to face (F2F) environment and the ones that required refining for virtual environments. Furthermore, we wished to identify if virtual teamworkers needed any new skills. As a starting point, we investigated the generic skills workers acquire and use on a daily basis. Generic skills are defined by Salas et al (2000: p, 344) as "the knowledge, skills and attitudes that a team member possesses when completing a task or communicating with fellow members, whether in a co-located or virtual environment". Generic skills influence both individuals and teams; they are skills which are "…transportable and applicable across teams" (Salas, et al., 2000, p. 344). A review of generic skills (Cannon-Bowers, Tannenbaum, Salas, & Volpe, 1995) was used to identify those which are used by design team members and is summarised in Table 2.

To examine the skills designers use it is necessary to understand the content of their interactions. A number of techniques facilitate such insights including Protocol Analysis and Content Analysis. Protocol Analysis attempts to infer cognitive processes by examining verbal interactions (Ericsson & Simon, 1993) but has been found to be a limited means of identifying non-verbal design cognition. Even where some comparisons are discovered, a large degree of interpretation is required (Cross, Christiaans, & K., 1996). The subjectivity of analysis and the length of time required to complete analysis also call into question the appropriateness of this method.

Content Analysis, according to (Wallace, 1987), involves coding transcripts of communications in terms of frequency analyses because the underlying assumption is that "the verbal content produced by the individual is representative of the thought processes at work in his or her mind" (p. 121).

Several content analysis techniques were used to identify and interpret these thought processes and thereby to investigate the generic skills our participants used. We explored micro-level communication processes because these "can provide valuable insights to managers and researchers alike about how to 'read' the health of teams" (Kanawattanachai & Yoo, 2002: p. 210). We identified quantitative content analysis as an effective means of identifying the generic skills of designers. This necessitated the development of a framework by which our data could be coded. Behavioural marker studies (Klampfer, et al., 2001, Carthey, de Leval, Wright, Farewell, & Reason, 2003) provided a template for our generic skills coding framework. Behavioural markers are observable non-technical "aspects of individual and team performance" (Carthey et al, 2003: p. 411) which are related to the effectiveness of an individual and team. The methods for creating behavioural markers informed the development of our framework. In accordance with Klampfer et al's (2001) recommendations, we devised a system that provided simple, clear markers, used appropriate professional terminology, and emphasised observable behaviours rather than

Changing Skills in Changing Environments: Skills Needed in Virtual Construction Teams 35

In addition to the Generic Skills analysis presented here, three other techniques were used to

1. **Bales's Interaction Process Analysis** (Bales, 1951) - to analyse the interactions between design team members, so that aspects such as decision-making, communication and

2. A **Communication Technique Framework** (Williams & Cowdroy, 2002) – to investigate

3. **Linguistic analyses** - to evaluate the communication occurring in teamwork. The approach adopted was derived from systemic functional linguistic theory (Halliday &

The aims of this study were to identity and examine the generic skills which facilitate teamwork in three settings, ranging from face-to-face to 3D virtual environments. The teamwork which we studied occurred during the conceptual stages of designing

Video and audio recordings of designers collaborating in teams were collected using Noldus Observer Pro (Burfield, Cadee, Grieco, Mayton, & Spink, 2003) to store, code and analyse our data. Noldus is ethnographic video analysis software which facilitates the collection, management, analysis and presentation of observational data. It allows researchers to view video footage and score the frequency of specific behaviours, and to note how these behaviours interact with each other or with independent variables. The advantages of such recordings include: being able to review interactions and behaviours as well as being able to compare different coders' or viewers' interpretations (Guerlain, Turrentine, Adams, &

It is often the case that design team members are drawn from different backgrounds/cultures, ages, and experience (Marchman III, 1998), especially in multidisciplinary design teams collaborating on an entire project. Stratified purposive sampling (Rice & Ezzy, 1999) was therefore used to select a heterogenous group of ten participants. This method of sampling ensured that the diversity of the participants was reflective, as far as possible, of the actuality of design teams in the real world. Participants were both male and female, of varying ages, cultures and had differing levels of experience and influence, ranging from higher management to junior staff. Due to constraints imposed by the funding body, recruitment of participants was limited to organisations within the Cooperative Research Centre for Construction Innovation (CRC-CI). The pool of eligible participants was further constrained by work pressures eventually resulting in participants being recruited

**Traditional face-to-face** collaborative design between the design team members

the techniques which the designers used to communicate.

analyse the data:

control could be examined.

Matthiessen, 2004).

construction projects.

**3.1 Data collection** 

Forrest Calland, 2004).

solely from the discipline of architecture.

Data were collected in three experimental conditions:

(including interactions such as talking and sketching).

**3.2 Participants** 

**3.3 Task** 

ambiguous attitudes or opinions. The Anaesthetists' Non-Technical Skills (ANTS) (Fletcher, et al., 2003) system was informative and helped shape our coding system. Using the ANTS system allowed us to incorporate the skills in Table 2 into the Generic Skills coding scheme shown in Table 4.


Table 2. Integrated skills (as adapted from Cannon-Bowers et al 1995)

In addition to the Generic Skills analysis presented here, three other techniques were used to analyse the data:


The aims of this study were to identity and examine the generic skills which facilitate teamwork in three settings, ranging from face-to-face to 3D virtual environments. The teamwork which we studied occurred during the conceptual stages of designing construction projects.

#### **3.1 Data collection**

34 Applications of Virtual Reality

ambiguous attitudes or opinions. The Anaesthetists' Non-Technical Skills (ANTS) (Fletcher, et al., 2003) system was informative and helped shape our coding system. Using the ANTS system allowed us to incorporate the skills in Table 2 into the Generic Skills coding scheme

> behaviour and reallocation of resources to adjust strategies

When team members have compatible mental models of the environment within and

Ability of team members to give, seek and receive task clarifying feedback.

Ability to direct and coordinate the activities of other team members particularly pertaining to performance, tasks, motivation, and creation of a positive environment.

of team members' interactions.

integrated, synchronized and completed within established

members using the prescribed manner and terminology.

resources, activities and responses are organized to ensure that tasks are

temporal constraints.

information, use sound judgment, identify

alternatives, select the most appropriate solution, and evaluate the consequences.

Flexibility

functions

Orientation Team awareness System awareness

Acceptance

monitoring

Goal setting Goal orientation

Task structuring Motivation of others

Conflict resolution Assertiveness Morale building

Task organisation Task interaction

Information exchange Consulting with others

Problem assessment Problem solving Planning Implementation

Timing

Compensatory behaviour Dynamic reallocation of

Performance feedback

Mutual performance

Procedure maintenance

Core Generic Skills Definition Sub-skills

based on feedback

outside of the team.

Adaptability The use of compensatory

Interpersonal relations Ability to optimise the quality

Communication Information exchange between

Decision making Ability to gather and integrate

Table 2. Integrated skills (as adapted from Cannon-Bowers et al 1995)

Co-ordination Process by which team

shown in Table 4.

Shared situational

Performance monitoring

Team management: Project management/leadership

awareness

and feedback

Video and audio recordings of designers collaborating in teams were collected using Noldus Observer Pro (Burfield, Cadee, Grieco, Mayton, & Spink, 2003) to store, code and analyse our data. Noldus is ethnographic video analysis software which facilitates the collection, management, analysis and presentation of observational data. It allows researchers to view video footage and score the frequency of specific behaviours, and to note how these behaviours interact with each other or with independent variables. The advantages of such recordings include: being able to review interactions and behaviours as well as being able to compare different coders' or viewers' interpretations (Guerlain, Turrentine, Adams, & Forrest Calland, 2004).

#### **3.2 Participants**

It is often the case that design team members are drawn from different backgrounds/cultures, ages, and experience (Marchman III, 1998), especially in multidisciplinary design teams collaborating on an entire project. Stratified purposive sampling (Rice & Ezzy, 1999) was therefore used to select a heterogenous group of ten participants. This method of sampling ensured that the diversity of the participants was reflective, as far as possible, of the actuality of design teams in the real world. Participants were both male and female, of varying ages, cultures and had differing levels of experience and influence, ranging from higher management to junior staff. Due to constraints imposed by the funding body, recruitment of participants was limited to organisations within the Cooperative Research Centre for Construction Innovation (CRC-CI). The pool of eligible participants was further constrained by work pressures eventually resulting in participants being recruited solely from the discipline of architecture.

#### **3.3 Task**

Data were collected in three experimental conditions:

 **Traditional face-to-face** collaborative design between the design team members (including interactions such as talking and sketching).

Changing Skills in Changing Environments: Skills Needed in Virtual Construction Teams 37

and this would have occurred when they were designing in the 3D virtual world. Conversely, if they had become fatigued or bored, their last task performed would have been the one most affected. It is thus not possible to determine whether sequence affected

All interactions were coded using the framework shown in Table 4 resulting in 4611 entries. Noldus provided each entry with a time stamp, and allowed entry of a subject code, an observable behaviour and a non-technical skill representative of that observable behaviour

Fig. 1. Screen showing coding of video data in Noldus Observer Pro

used for all statistical analyses.

The resulting scores were statistically analysed using a repeated measures ANOVA parametric test to establish the differences between participants' performance on the three tasks (traditional face-to-face design, virtual design using a electronic whiteboard, and virtual design using a high bandwidth 3D virtual world) (Riedlinger, Gallois, McKay, & Pittam, 2004). The results of the ANOVA tests were interpreted using Mauchly's Test of Sphericity which examines the covariance of the dependent samples. The data were also examined to determine which shift in condition (i.e. face-to-face to whiteboard or whiteboard to 3D virtual world) was responsible for any significance. SPSS Version 12 was

the outcomes of this research.

**3.4 Coding of data** 

(see Figure 1).


Designers were grouped into five teams of two and asked to prepare conceptual designs that responded to various briefs. These briefs related to fictional projects on an actual site at Sydney University, Australia. Depending on the session, designers were asked to design an art gallery, a hostel, a library or a dance school for the site. The participants were then given 30 minutes to prepare their designs using one of the three experimental conditions. Prior to each design session the research team spent one to two hours coaching the designers in the capabilities of the whiteboard and 3D virtual world technologies. Once designers were familiar with the hardware and software, they were asked to prepare their designs. Typical characteristics of the virtual teamwork involved in these tasks are presented in Table 3 using Dubé & Paré's (2004) framework.


Table 3. Typical characteristics of the virtual teams engaged in this project (adapted from Dubé & Paré, 2004)

All tasks were conducted in an identical sequence (i.e. participants first worked "face-toface", then used a "whiteboard" and finally designed in the "3D virtual world"). This procedure was prescribed by our research directorate and was designed primarily for the first two strands of our overall research project (Maher, 2002). We are conscious that participants may have become familiar with aspects of the tasks that they were asked to complete, and may also have become fatigued (Pring, 2005). As the designers gained experience of working together, one would assume they would be able to work more effectively over time. If this is so, their final collaboration would have been the optimal one and this would have occurred when they were designing in the 3D virtual world. Conversely, if they had become fatigued or bored, their last task performed would have been the one most affected. It is thus not possible to determine whether sequence affected the outcomes of this research.

#### **3.4 Coding of data**

36 Applications of Virtual Reality

 **Virtual design using a shared electronic whiteboard** (incorporating synchronous audio and visual communication) which allowed drawings, images and text to be

 **Virtual design using a high bandwidth 3D virtual world** (Activeworlds-Corporation, 2008) (incorporating synchronous audio communication) which allowed drawings, images and text to be shared. This tool represents team members as "avatars" and allowed them to manipulate 3D representations of a design and to communicate using

Designers were grouped into five teams of two and asked to prepare conceptual designs that responded to various briefs. These briefs related to fictional projects on an actual site at Sydney University, Australia. Depending on the session, designers were asked to design an art gallery, a hostel, a library or a dance school for the site. The participants were then given 30 minutes to prepare their designs using one of the three experimental conditions. Prior to each design session the research team spent one to two hours coaching the designers in the capabilities of the whiteboard and 3D virtual world technologies. Once designers were familiar with the hardware and software, they were asked to prepare their designs. Typical characteristics of the virtual teamwork involved in these tasks are presented in Table 3 using

Degree of reliance on ICT Low Varies High ICT availability High X Low Members' ICT Proficiency High X Low Team size Small X Large Geographic dispersion Local X Global Task or project duration Long term X Short term

Members' assignments Full-time X Part-time Membership stability Stable X Fluid Task interdependence Low X High

Table 3. Typical characteristics of the virtual teams engaged in this project (adapted from

All tasks were conducted in an identical sequence (i.e. participants first worked "face-toface", then used a "whiteboard" and finally designed in the "3D virtual world"). This procedure was prescribed by our research directorate and was designed primarily for the first two strands of our overall research project (Maher, 2002). We are conscious that participants may have become familiar with aspects of the tasks that they were asked to complete, and may also have become fatigued (Pring, 2005). As the designers gained experience of working together, one would assume they would be able to work more effectively over time. If this is so, their final collaboration would have been the optimal one

geneous

DEGREE OF COMPLEXITY LOW <> HIGH

Extensive X None

Varies Hetero-

geneous

shared.

audio as well as text "chat" facilities.

Dubé & Paré's (2004) framework.

Cultural diversity Homo-

Prior shared work experience

Dubé & Paré, 2004)

All interactions were coded using the framework shown in Table 4 resulting in 4611 entries. Noldus provided each entry with a time stamp, and allowed entry of a subject code, an observable behaviour and a non-technical skill representative of that observable behaviour (see Figure 1).

Fig. 1. Screen showing coding of video data in Noldus Observer Pro

The resulting scores were statistically analysed using a repeated measures ANOVA parametric test to establish the differences between participants' performance on the three tasks (traditional face-to-face design, virtual design using a electronic whiteboard, and virtual design using a high bandwidth 3D virtual world) (Riedlinger, Gallois, McKay, & Pittam, 2004). The results of the ANOVA tests were interpreted using Mauchly's Test of Sphericity which examines the covariance of the dependent samples. The data were also examined to determine which shift in condition (i.e. face-to-face to whiteboard or whiteboard to 3D virtual world) was responsible for any significance. SPSS Version 12 was used for all statistical analyses.

Changing Skills in Changing Environments: Skills Needed in Virtual Construction Teams 39

design

understanding C21 Describes seriousness or urgency of task C22 Pays close attention to advice of fellow member Anticipating C31 Takes action to avoid future problems

C32 Reviews effects of a change

D22 Implements chosen design Re-evaluating D31 Re-evaluates chosen design technique after it has been chosen

Identifying options D11 Discusses design options with clients/other designers

selecting options D21 Weighs up risks associated with different designs

Intra-reliability between two raters was established for the generic skills coding scheme on a 35-minute session using Noldus Observer Pro. Point-by-point agreement was 81% and 80% on the frequency of coding strings and frequency and sequence of the coding strings, respectively.

The generic skill *Shared Situational Awareness* increased significantly (*F*(2, 8) = 4.903, *p* < .05). The Within-Subject Contrasts test indicated a significant difference between face-to-face and

For the skill of *Decision Making*, there was a significant decrease (*F*(2, 8) = 42.431, *p* < .001) in frequency as the design conditions moved from low to high bandwidth conditions. The Within-Subject Contrasts test demonstrated a significant difference between both the faceto-face to whiteboard and whiteboard to 3D virtual world (*F*(1, 4) = 120.274, *p* < .001 and

For the skill of *Task Management*, the decrease in frequency from face-to-face to whiteboard

The following five observable skills (see Figure 2) were significantly affected by the

These were both at or above the minimum acceptable level of 80% (Kazdin, 1982).

C11 Asks for information or artefacts relating to a

C12 Checks on the status of a project and tasks C13 Collects information regarding a problem C14 Cross checks and double checks information

D12 Discusses various techniques for the design

Generic skills Sub-skills Code Observable Behaviour

Shared Situational Awareness

Decision Making

**4. Results** 

**4.1 Generic skills** 

Gathering information

Recognising and

Balancing risks and

whiteboard conditions (*F*(1, 4) = 19.478, *p* < .05).

approached significance (*F*(1, 4) = 4.799, *p* > .1).

*F*(1, 4) = 8.685, *p* < .05 respectively).

**4.2 Observable behaviours** 

experimental conditions:

Table 4. Generic Skills Coding Scheme



Table 4. Generic Skills Coding Scheme

Intra-reliability between two raters was established for the generic skills coding scheme on a 35-minute session using Noldus Observer Pro. Point-by-point agreement was 81% and 80% on the frequency of coding strings and frequency and sequence of the coding strings, respectively. These were both at or above the minimum acceptable level of 80% (Kazdin, 1982).
