**3. Theoretical framework**

#### **3.1. General**

positions of president, team manager, team leader as well as team member, as shown in Figure 7. The structure of the team is in the form of a pyramid or an inverted tree representing an organizational chart in which each branch is a team which is a team member aggregate. The process of hierarchical aggregate assessment is the action of grouping team members together in a hierarchical organizational structure on several levels and then make an assessment process for each member of the team that is a leaf of the tree or a node of the organizational structure. The Internet application "Cluster" has implemented this data structure located in its MySQL [22] database, and its complex assessment task presentation engine in collaborative mode can perform assessment procedures for each team member or each node of the tree. So in one assessment task, the application can assess different objectives, skills, abilities and knowledge. This feature has not been implemented completely in other distance learning applications such as Moodle [23], Blackboard [24] and WebCT [24], and this statement defines

**Figure 7.** Hierarchical aggregate assessment process capabilities for simultaneous assessment of multiple skills

the fundamentals of the problematics of this research.

90 E-Learning - Instructional Design, Organizational Strategy and Management

The theories and research produced by the actual project are an extension of previous work made by Nance [25] that is using a similar aggregation process as the "Cluster" application to form teams with several levels of hierarchy for educational purposes to manage project teams in software engineering courses and also the work of Freeman and McKenzie [14, 15] on the development of the "SPARK" software application that is an Internet distance assessment system managing self-assessment and peer assessment made with assessment grids. Peer assessment is in level 5 of Krathwohl's affective domain taxonomy (Legendre [1]; Lavallée [26]; Krathwohl, Bloom and Masia [27]). Competency assessment in the field of hierarchical aggregate assessment can be made with observation grids or competencies assessment grids (Hubert & Denis [28]; Jeunesse [29]) and also with portfolio (Allal [4]) that usually contains self-assessments (Endrizzi and Ray [30]).

The actual research is based on the development of the "Cluster" Internet application which implements the process of hierarchical aggregate assessment. This application is a presentation engine of collaborative mode complex assessment tasks in an authentic context. The develop‐ ment of the "Cluster" Internet application finds its theoretical foundations in (1) the complex assessment tasks (Louis & Bernard [31]; Tardif [32]), (2) authentic context assessment (Palm [33], p. 6; Louis & Bernard [31]; Wiggins [34, 35]; Hart [36]; Allal [4]; Rennert-Ariev [37]), (3) teamwork assessment (Baker & Salas [9]; Marin-Garcia & Lloret [38]), (4) collaborative work assessment (Swan, Shen & Hiltz [39]; Volkov & Volkov [8]; Boud, Cohen & Sampson [40]; MacDonald [41]; Swan, Shen & Hiltz [39]; Worcester Polytechnic Institute[42]), and (5) assessment grids (Durham, Knight & Locke [43]; Marin-Gracia & Lloret [38]) as well as selfassessment and peer assessment (Lingard [44]; Goldfinch [45]; Goldfinch & Raeside [46]; Northrup & Northrup [47]).

#### **3.2. Definition of hierarchical aggregate assessment process in general terms**

The hierarchical aggregate assessment is defined in general terms as a process that groups teams as well as a subfield of teamwork assessment in which teams have several levels of hierarchy and supervision (Lesage, Raîche Riopel & Sebkhi [18]; Lesage, Raîche Riopel, Fortin & Sebkhi [19, 20]; Sebkhi, Raîche Riopel & Lesage [21]). This assessment process with several levels of hierarchy and supervision in the field of education, that is one of the main theoretical contributions of this research project, has been named "hierarchical aggregate assessment". This process includes the formation of teams with several levels of hierarchy, the display of exams or complex assessment tasks to the teams and also the dismantling of the teams for the next assessment task in teams, as shown in Figure 1.

#### **3.3. Definition of hierarchical aggregate assessment process in education**

In the education field, the process of hierarchical aggregate assessment is defined as a team grouping process and a teamwork assessment subfield. In this subfield, teams have several levels of hierarchy and supervision where team leaders that could be students are assessed by one or many group managers that could be other students, teachers or professors (Lesage, Raîche, Riopel & Sebkhi [18]; Lesage, Raîche, Riopel, Fortin & Sebkhi [19, 20]; Sebkhi, Raîche, Riopel & Lesage [21]).

#### **3.4. Situation of the field of hierarchical aggregate assessment process in the mastery learning paradigm**

The assessment process in the mastery learning paradigm wants to determine the level at which the educational objectives are mastered or attained (Legendre [1]). Bloom's [48] cognitive level taxonomy of educational objectives allows to determine educational objectives by a statement describing knowledge, skill or performance and a description concerning the application of this knowledge, skill or performance. Bloom's cognitive level taxonomy of comprehension, application, analysis and synthesis is considered to represent the most important goals of the education field. This constatation has provided a foundation to raise the complexity level of tests and teaching programs towards educational objectives that could be in the higher levels of Bloom's taxonomy (Krathwohl [49]). According to some authors as Wiggins [34], traditional tests based on educational objectives are using out-of-context rote learning or open questions needing a few words for answers as an exam on multiplication tables. Those type of tests or exams are verifying if the students meet the criteria mentioned in the course curriculum.

The hierarchical assessment process is based on teamwork assessment. According to the mastery learning paradigm, the assessment process is realized by tests or exams that could contain items [49], questions and tests (De Ketele & Gérard [2]) and also work to accomplish [48]. As stated by the mastery learning paradigm, an exam or a test done in teams needs an accurate work or performance accomplished by a team at the end of a course or a study program [1]. Exam questions and learning objectives, concerning work or team performances, are included in the levels of Bloom's cognitive level taxonomy. In the hierarchical assessment process, the tests and exams are done in teams, so the persons taking part in the team exam can assess quantitatively and qualitatively the work done in teams to determine if the pro‐ duction or the performance meets the determined criteria; this type of assessment being part of level 6 of Bloom's [48] cognitive level taxonomy is named "evaluation". In some exams taken in teams, the persons taking part in the exam could do self-assessment and peer assessment. The peers' assessment process is part of level 5 of Krathwohl's [27] affective level taxonomy which interprets value or belief system classification [01, 26, 27].

[33], p. 6; Louis & Bernard [31]; Wiggins [34, 35]; Hart [36]; Allal [4]; Rennert-Ariev [37]), (3) teamwork assessment (Baker & Salas [9]; Marin-Garcia & Lloret [38]), (4) collaborative work assessment (Swan, Shen & Hiltz [39]; Volkov & Volkov [8]; Boud, Cohen & Sampson [40]; MacDonald [41]; Swan, Shen & Hiltz [39]; Worcester Polytechnic Institute[42]), and (5) assessment grids (Durham, Knight & Locke [43]; Marin-Gracia & Lloret [38]) as well as selfassessment and peer assessment (Lingard [44]; Goldfinch [45]; Goldfinch & Raeside [46];

The hierarchical aggregate assessment is defined in general terms as a process that groups teams as well as a subfield of teamwork assessment in which teams have several levels of hierarchy and supervision (Lesage, Raîche Riopel & Sebkhi [18]; Lesage, Raîche Riopel, Fortin & Sebkhi [19, 20]; Sebkhi, Raîche Riopel & Lesage [21]). This assessment process with several levels of hierarchy and supervision in the field of education, that is one of the main theoretical contributions of this research project, has been named "hierarchical aggregate assessment". This process includes the formation of teams with several levels of hierarchy, the display of exams or complex assessment tasks to the teams and also the dismantling of the teams for the

In the education field, the process of hierarchical aggregate assessment is defined as a team grouping process and a teamwork assessment subfield. In this subfield, teams have several levels of hierarchy and supervision where team leaders that could be students are assessed by one or many group managers that could be other students, teachers or professors (Lesage, Raîche, Riopel & Sebkhi [18]; Lesage, Raîche, Riopel, Fortin & Sebkhi [19, 20]; Sebkhi, Raîche,

**3.4. Situation of the field of hierarchical aggregate assessment process in the mastery**

The assessment process in the mastery learning paradigm wants to determine the level at which the educational objectives are mastered or attained (Legendre [1]). Bloom's [48] cognitive level taxonomy of educational objectives allows to determine educational objectives by a statement describing knowledge, skill or performance and a description concerning the application of this knowledge, skill or performance. Bloom's cognitive level taxonomy of comprehension, application, analysis and synthesis is considered to represent the most important goals of the education field. This constatation has provided a foundation to raise the complexity level of tests and teaching programs towards educational objectives that could be in the higher levels of Bloom's taxonomy (Krathwohl [49]). According to some authors as Wiggins [34], traditional tests based on educational objectives are using out-of-context rote learning or open questions needing a few words for answers as an exam on multiplication tables. Those type of tests or exams are verifying if the students meet the criteria mentioned

**3.2. Definition of hierarchical aggregate assessment process in general terms**

**3.3. Definition of hierarchical aggregate assessment process in education**

next assessment task in teams, as shown in Figure 1.

92 E-Learning - Instructional Design, Organizational Strategy and Management

Northrup & Northrup [47]).

Riopel & Lesage [21]).

**learning paradigm**

in the course curriculum.

#### **3.5. Situation of the field of hierarchical aggregate assessment process in the competencybased approach paradigm**

In the competency-based paradigm, the execution of a competency is based on resource mobilization to solve a complex situation (Van Kempen [3]). Competencies include the grouping of skills, attitudes and knowledge allowing a person to perform tasks (Bas‐ tiaens [50]). The competency-based approach paradigm replaces classical tests based on objectives by assessment tasks or situations that include social interaction (Allal [4]). Assessment tasks are evaluation tools that use or mobilize resources to solve a problemat‐ ic situation or to perform a complex task. These tools are used to develop competencies with complex tasks allowing knowledge synthesis (Saskatchewan Professional Develop‐ ment Unit [51]; Olivier [52]; Louis and Bernard [31]; Tardif [32]; Van Kempen [3]; De Ketele & Gérard [2]).

The objectives of learning and assessment situations are to develop disciplinary and transver‐ sal competencies and to assess all students that must prove that they can resolve a problematic situation with their knowledge and skills (Bibeau [53]). The aim of competency assessment is to verify if the student has well used all available resources to accomplish a task successfully. During this process, students should be involved in their own assessment and perform their self-assessment (Jeunesse [29]). The competency formative assessment process is based on interactive regulation that comes with student-teacher interaction, interactions with peers and learning tools. The learner can imply himself in the assessment process with self-assessment, peer assessment and co-evaluation (Allal [4]). The hierarchical assessment process in the competency-based approach paradigm is the implementation of complex assessment tasks in teams. These tasks could include summative assessment that are performance or tasks to accomplish either individually or in teams and also includes formative assessment that is produced by self-assessment and peer assessment of team members. The competency-based approach in the hierarchical aggregate assessment field could be performed with observation grids or competency assessment grids as shown in Figures 21, 22 and 23 (Hubert & Denis [28]; Jeunesse [29]) and also with portfolio assessment (Allal [4]) that usually contains self-assess‐ ment (Endrizzi & Ray [30]).

#### **3.6. Previous work and similar available existing internet applications**

The current research project finds its origins and its theoretical framework in other previous research and through other distance assessment Internet applications that have been developed with a research and development methodology (R & D). These applications are SPARK developed by Freeman and McKenzie [14, 15] and Willey and Freeman [54, 55]; MLE developed by Marshall-Mies, Fleischman, Martin, Zaccaro, Baughman and McGee [56]; Mega Code developed by Kaye and Mancini [57]; and the application that is most similar to the current research project is a collaborative work management Internet application developed by Nance [25].

SPARK [14, 15, 54, 55] is a remote rating system that calculates the results of self-assessment and peer assessment grids to determine the final grade of engineering students on projects during practical work in engineering. This primarily detects the team members who have not done their fair share of work by giving poor performance in their team by letting others do their work for them.

MLE [56] is an application that predicts and assesses the leadership potential of high-level military managers such as colonels and generals with complex assessment tasks that are case studies and resolution of war scenarios.

Mega Code [57] is a software application used in the field of medicine and that is a cardiac arrest simulator. This application is used to assess the performance of resident doctors and nurses when they hold the role of leader of a resuscitation team who treated the case of patients who suffered a cardiac arrest according to the five main roles that are (1) the doctor who is in charge of the team, (2) the controller of respiration, (3) the head of the defibrillator, (4) the head of chest compressions and (5) the head of injections and intravenous infusions. The assessment of the team leader is made using an assessment grid that checks the two main aspects of cardiac resuscitation that are the team effort and the process and directions given to the members of the team by the team leader to resuscitate the patient.

The collaborative work management Web application developed by Nance [25] is used by students of engineering and computer science faculties. This application uses a multiple-level aggregation process for the grouping of teams that is similar to the aggregation process implemented in this research and in the "Cluster" Internet application. Nance's research [25] consists of the implementation of an Internet-based collaborative work application that is used to manage and assess the projects and the productions of engineering and computer science students. This application has the features needed to group students in teams that have multiple levels of hierarchy and supervision including team leaders and project managers (bosses) and project administrators (bosses of bosses (BOB)) supervising several project managers in the field of engineering and computer science. Nance's application collaborative work implementation is based on electronic mail (E-mail) and a discussion forum website.

#### **3.7. Link between hierarchical aggregate assessment applications and Management Information Systems (MIS)**

Jeunesse [29]) and also with portfolio assessment (Allal [4]) that usually contains self-assess‐

The current research project finds its origins and its theoretical framework in other previous research and through other distance assessment Internet applications that have been developed with a research and development methodology (R & D). These applications are SPARK developed by Freeman and McKenzie [14, 15] and Willey and Freeman [54, 55]; MLE developed by Marshall-Mies, Fleischman, Martin, Zaccaro, Baughman and McGee [56]; Mega Code developed by Kaye and Mancini [57]; and the application that is most similar to the current research project is a collaborative work management Internet application

SPARK [14, 15, 54, 55] is a remote rating system that calculates the results of self-assessment and peer assessment grids to determine the final grade of engineering students on projects during practical work in engineering. This primarily detects the team members who have not done their fair share of work by giving poor performance in their team by letting others do

MLE [56] is an application that predicts and assesses the leadership potential of high-level military managers such as colonels and generals with complex assessment tasks that are case

Mega Code [57] is a software application used in the field of medicine and that is a cardiac arrest simulator. This application is used to assess the performance of resident doctors and nurses when they hold the role of leader of a resuscitation team who treated the case of patients who suffered a cardiac arrest according to the five main roles that are (1) the doctor who is in charge of the team, (2) the controller of respiration, (3) the head of the defibrillator, (4) the head of chest compressions and (5) the head of injections and intravenous infusions. The assessment of the team leader is made using an assessment grid that checks the two main aspects of cardiac resuscitation that are the team effort and the process and directions given to the members of

The collaborative work management Web application developed by Nance [25] is used by students of engineering and computer science faculties. This application uses a multiple-level aggregation process for the grouping of teams that is similar to the aggregation process implemented in this research and in the "Cluster" Internet application. Nance's research [25] consists of the implementation of an Internet-based collaborative work application that is used to manage and assess the projects and the productions of engineering and computer science students. This application has the features needed to group students in teams that have multiple levels of hierarchy and supervision including team leaders and project managers (bosses) and project administrators (bosses of bosses (BOB)) supervising several project managers in the field of engineering and computer science. Nance's application collaborative work implementation is based on electronic mail (E-mail) and a discussion forum website.

**3.6. Previous work and similar available existing internet applications**

94 E-Learning - Instructional Design, Organizational Strategy and Management

ment (Endrizzi & Ray [30]).

developed by Nance [25].

their work for them.

studies and resolution of war scenarios.

the team by the team leader to resuscitate the patient.

A management information system (MIS) software application "uses computer equipment and software, databases, manual procedures, models for analysis, planning, control and decisionmaking" (Davis, Olson, Ajenstat & Peaucelle [58]). These systems may contain information about the function, department and the hierarchical position of the members of the organiza‐ tion that are stored in hierarchical databases (Burch & Grudnitski [59]; Davis & Olson [60]; Davis, Olson, Ajenstat & Peaucelle [58]; Laudon & Laudon [61]; Laudon, Laudon & Brabston [62]). Some authors such as Kanter [63] indicate that the employee file can be sorted by order of position or assignment to identify employees who have the same hierarchical position. A database diagram illustrating an employee's position is shown in Figure 8. A hierarchical aggregate assessment software application is therefore a management information system where the employees to manage are students who have a hierarchical position.

**Figure 8.** The record of an employee in a management information system database [62]

In the actual paradigm, there is a major difference between distance assessment systems and management information system software applications. A distance assessment system software application is a question bank repository stored in a database that usually presents the same questions or the same assessment tasks to all the students to assess the same skills and knowledge and there is no hierarchical relationship or hierarchy levels between the students. A management information system (MIS) is a software application that stores and processes management data and information on employees to produce information used for decision-making. The assessment data that a management information system produces and computes for the employees are usually sales data and production performance. Management information systems are able to record the hierarchical relations and positions of the employ‐ ees, while distance assessment applications cannot.

In the hierarchical aggregate assessment paradigm, there is only a slight difference between hierarchical aggregate assessment applications and management information systems because both records the hierarchical relations and positions of the employees. The only difference is that the management information system processes management data, while the hierarchical aggregate assessment software application processes assessment data, course material, question banks and complex assessment tasks with several levels of hierarchy. Hence, any management information system could be modified to record course material and question banks to present complex assessment tasks with several levels of hierarchy. So the modified management information system has now been added hierarchical aggregate assessment capability and is equally now a hierarchical aggregate assessment software application, as shown in Figure 9.

**Figure 9.** Link between hierarchical aggregate assessment applications and management information systems (MIS)
