**5. Empirical support for e-assessment based on the human lexicon**

To train a classifier, hundreds of successful and unsuccessful learners´ schema-related wordrecognition patterns are presented to it. Achieving this requires first obtaining schema-related

**Figure 3.** Concepts related to schema course content are selected during a constructive evaluation (A). These concepts can be used to assess whether a student integrated new information into her/his long-term memory using digitized cognitive semantic priming techniques (chronometric evaluation; B). Word-recognition latency patterns are used by the

**Figure 2.** Diagram of how continuous assessment of student knowledge acquisition affects various levels of processing

**Figure 4A** shows a computer system for obtaining students' and teachers' concept definers to target schema-related concepts using a technique called natural semantic net mapping. This technique produces definitions (using single concept definers such as nouns and adjectives) for represented objects based on their meanings and not on free associations or pure semantic

In this technique, the 10 highest-ranked definers of each target concept (SAM group) can be used to draw a semantic net, if desired. Some concepts serve as definers for more than one target concept. These are common definers, and other definers and target concepts are interconnected through them. Numerous common definers tend to emerge whenever there

concepts from students before and after a course (after learning).

neural net to discriminate between successful and unsuccessful learners.

74 From Natural to Artificial Intelligence - Algorithms and Applications

category memberships [40, 41].

in an AHS learning session.

are close links among target concepts (schemata).

To better describe these concepts, we will describe data resulting from application of constructive-chronometric assessment in an undergraduate psychology course on the computational mind. **Figure 5A** and **B** shows partial instances of definitions obtained from a set of 10 schema-related target concepts relevant to this course before learning (**Figure 5**, top panels) and after learning (**Figure 5**, bottom panels). The following target concepts were provided by the teacher of the course: mind, computation, von Neumann, Turing machine, connectionism, memory, computational mind, working memory, long-term memory, and HPI (human information processing).

In developing a natural semantic net, participants are allowed 60 seconds to provide concept definers. Then, following each definition task, they rank each concept definer (between 1 and 10) in terms of how well they define the target concept. After the system has randomly presented target concepts, it calculates the 10 highest-ranking definers for each target (SAM group; **Figure 5A**). For later consideration in building an expert system, note that **Figure 5A** shows that the M value corresponds to the sum of ranks assigned by all the participants to each definer concept. This value is a measure of the definition relevance for the target concept. Other values, such as the density of the net (G value) and the richness of the definers for each target (J value), are also calculated [40].

their meaning formation around the core concepts of the computational mind: symbol; mind and brain; and the leading figures in this academic field, Turing and von Neumann. The

Formative E-Assessment of Schema Acquisition in the Human Lexicon as a Tool in Adaptive…

http://dx.doi.org/10.5772/intechopen.81623

77

The weight matrix is used by a CSNN to simulate schema behavior, as shown in **Figure 6**. Here, there is 100% activation of the SYMBOLS input. As a result, MIND and DURATION were the only output activated concepts. When the students were asked about this result, they argued that according to what they learned from the course, a core concept in cognitive theory is that all mind activities occur in time, even symbol processing and construction. This schema acquisition was also intended by the teacher. In addition, note from the surface plot in **Figure 6** that balanced positivity and negativity of weight association values (from +10 to −60)

By selecting schema-related concepts from the computer models and semantic definers relevant to meaning formation (e.g., emergence of common definers in SAM groups or concepts relevant to a schema), schema word pairs can be selected to perform a semantic word priming study.

Schema-related concepts following the course involve longer word-recognition times since a

To illustrate this point, **Figure 7** shows interaction graphs describing a frequent result on schemata word-related time recognition. **Figure 7A** shows that at the beginning of the course, schemata related are not significantly differentiated from other semantically related word pairs. This is not the case at the end of the course where students required significant higher

**Figure 6.** User interface for modeling schema-based behavior, and a surface plot of its underlying weight association

enhanced correct discrimination among the schema-related concepts.

whole schema is activated (not simply a lexical association).

processing time to recognize schemata words (schemata priming).

matrix (bottom center).

teacher confirmed that this was the intent.

Note also that in **Figure 5B**, before learning (top panel), some of the targets lacked complete definitions. Moreover, lower common definers are obtained before learning. This lack of connectivity is reflected when a weight association matrix among concepts is calculated using Eq. (1) (**Figure 5C**, top). This is not the case for the weight association symmetric matrix obtained after learning, shown in **Figure 5C** (bottom). In turn, these connectivity matrixes can be used as an input matrix to many visualization tools, as shown in **Figure 5D**. Before learning, the visual concept organization allows one to immediately note that all the definers were arranged in two main groups connected by a single central one (PROCESSES). In contrast, at the end of the course, the net consists of a more sophisticated concept organization resembling a small world structure characterized by a set of highly clustered neighborhoods and a short average path length in which a small number of well-connected nodes serve as hubs. This net is a normal result of learning when using this technique [41].

This approach to evaluating learning emphasizes two aspects. First, the semantic net focuses on identifying meaning formation. For instance, at the end of the course, students centered

**Figure 5.** Ten relevant concept definers (SAM group) used to define schema concepts (A) are obtained by a computer system to obtain natural semantic nets (B) before and after a course on the computational mind. Cooccurrence weight associations among concepts (C) and GEPHI analysis (using the Yifan Hu algorithm and (D) can be produced using this semantic mapping technique.

their meaning formation around the core concepts of the computational mind: symbol; mind and brain; and the leading figures in this academic field, Turing and von Neumann. The teacher confirmed that this was the intent.

In developing a natural semantic net, participants are allowed 60 seconds to provide concept definers. Then, following each definition task, they rank each concept definer (between 1 and 10) in terms of how well they define the target concept. After the system has randomly presented target concepts, it calculates the 10 highest-ranking definers for each target (SAM group; **Figure 5A**). For later consideration in building an expert system, note that **Figure 5A** shows that the M value corresponds to the sum of ranks assigned by all the participants to each definer concept. This value is a measure of the definition relevance for the target concept. Other values, such as the density of the net (G value) and the richness of the definers for each

Note also that in **Figure 5B**, before learning (top panel), some of the targets lacked complete definitions. Moreover, lower common definers are obtained before learning. This lack of connectivity is reflected when a weight association matrix among concepts is calculated using Eq. (1) (**Figure 5C**, top). This is not the case for the weight association symmetric matrix obtained after learning, shown in **Figure 5C** (bottom). In turn, these connectivity matrixes can be used as an input matrix to many visualization tools, as shown in **Figure 5D**. Before learning, the visual concept organization allows one to immediately note that all the definers were arranged in two main groups connected by a single central one (PROCESSES). In contrast, at the end of the course, the net consists of a more sophisticated concept organization resembling a small world structure characterized by a set of highly clustered neighborhoods and a short average path length in which a small number of well-connected nodes serve as hubs. This net is a normal result of

This approach to evaluating learning emphasizes two aspects. First, the semantic net focuses on identifying meaning formation. For instance, at the end of the course, students centered

**Figure 5.** Ten relevant concept definers (SAM group) used to define schema concepts (A) are obtained by a computer system to obtain natural semantic nets (B) before and after a course on the computational mind. Cooccurrence weight associations among concepts (C) and GEPHI analysis (using the Yifan Hu algorithm and (D) can be produced using this

target (J value), are also calculated [40].

76 From Natural to Artificial Intelligence - Algorithms and Applications

learning when using this technique [41].

semantic mapping technique.

The weight matrix is used by a CSNN to simulate schema behavior, as shown in **Figure 6**. Here, there is 100% activation of the SYMBOLS input. As a result, MIND and DURATION were the only output activated concepts. When the students were asked about this result, they argued that according to what they learned from the course, a core concept in cognitive theory is that all mind activities occur in time, even symbol processing and construction. This schema acquisition was also intended by the teacher. In addition, note from the surface plot in **Figure 6** that balanced positivity and negativity of weight association values (from +10 to −60) enhanced correct discrimination among the schema-related concepts.

By selecting schema-related concepts from the computer models and semantic definers relevant to meaning formation (e.g., emergence of common definers in SAM groups or concepts relevant to a schema), schema word pairs can be selected to perform a semantic word priming study.

Schema-related concepts following the course involve longer word-recognition times since a whole schema is activated (not simply a lexical association).

To illustrate this point, **Figure 7** shows interaction graphs describing a frequent result on schemata word-related time recognition. **Figure 7A** shows that at the beginning of the course, schemata related are not significantly differentiated from other semantically related word pairs. This is not the case at the end of the course where students required significant higher processing time to recognize schemata words (schemata priming).

**Figure 6.** User interface for modeling schema-based behavior, and a surface plot of its underlying weight association matrix (bottom center).

**Figure 7.** Students' word-recognition latency times corresponding to associative, schema-related, and nonrelated words (A). Comparison of schema priming effects obtained from this study and from similar studies involving other knowledge domains (B).

Is it possible to obtain the same results with an ITS-AHS? Well, notice that now the problem is not cognitive modeling students but instructors. As it will be described next, current aca-

Formative E-Assessment of Schema Acquisition in the Human Lexicon as a Tool in Adaptive…

http://dx.doi.org/10.5772/intechopen.81623

79

**Figure 8.** A fractured mental representation on computer usability (A), changing after a corrective course (B).

Up to this point, the discussion of applying e-assessment to navigation support and adaptation of content has focused only on AHS. Note, however, that the same arguments can be applied to alternative e-instruction systems or alternative e-instruction. For instance, adaptive navigation support for AHS or in e-instruction can be implemented using the same assump-

In a functional adaptive instruction system such as the one shown in **Figure 8**, the student model is a domain-specific well-trained classifier. Empirical research in several knowledge domains has shown that this type of classifier yields successful classification in 95–98% of

During the defining of a target concept (in natural semantic net mapping), after a student decides which is the highest-ranking concept definer (indicated by its M value), the nexthighest-ranked concept from the set of definers depends on the concept frequency (F) in the definition task and the time required to produce it, that is, its interresponse time (ITR; see

**6. Adaptive e-instruction through e-assessment in e-learning** 

**6.2. Expert model: determining concept organization of meaning formation**

demic efforts are being made on this direction.

tions by considering the model presented in **Figure 9**.

**environments: a proposal**

**6.1. The student model**

instances [38].

This schema priming is assumed to occur when schema information is stored in long-term memory, which likely explains why the neural net (after training) is useful for discriminating between successful and unsuccessful learners.

This is relevant because even when we cannot see the existence of a schema in the lexicon, we can track its footsteps as evidence that long-term learning has occurred. On the other hand, it is not necessary to specify a lexicon; it is enough to say that lexical information is obtained and organized as proposed by a no-lexicon view. **Figure 7B** shows that this effect might vary depending on the knowledge domain and the effect of instruction [34, 37, 39].

Several possibilities are introduced by considering a cognitive assessment of learning like the one just described. Let us consider a study [43] carried over 60 first-semester bachelor engineering students who took a course on computer usability. Here, 15 students failed to pass the course, but after a post-season corrective course, they succeeded and achieved the course credit. **Figure 8** shows the mental concept representations obtained by a constructivechronometric assessment of learning before and after the corrective course.

Note that at the beginning of the course, the EVCOG system shows that students have a mental representation with separated concept clusters (A). This leads to confusion in terms of meaning of a topic. After a corrective course, students presented a single unified schemata knowledge where DESIGN, INTERFACE, and USER showed meaningful centrality to knowledge representation (B). The teacher in charge of the course argued that after looking at the system cognitive report at the beginning of the course, she tried meaningful integration of topics by having the concept of DESIGN as the main reference for meaning formation. Chronometric assessment provided support to this learning process since schemata priming was not obtained at the beginning of the learning period but appeared at the end of classes supporting the idea that students not only successfully passed the course but also obtained long-term retention of schemata.

Formative E-Assessment of Schema Acquisition in the Human Lexicon as a Tool in Adaptive… http://dx.doi.org/10.5772/intechopen.81623 79

**Figure 8.** A fractured mental representation on computer usability (A), changing after a corrective course (B).

Is it possible to obtain the same results with an ITS-AHS? Well, notice that now the problem is not cognitive modeling students but instructors. As it will be described next, current academic efforts are being made on this direction.
