**6.1 Overview of study and study design**

*Metacognition in Learning*

metacomprehension accuracy.

comprehension accuracy.

**6. Drawing to improve metacomprehension accuracy**

Theoretically, drawing has promise as an intervention to improve metacomprehension because it has been shown to facilitate construction of the situation model. Although the results examining the effect of drawing on learning are mixed, with some studies showing drawing improves learning [34] and others showing no benefit to drawing [35]. The results fairly consistently show that drawing improves conceptual understanding but not factual learning [36]. Put differently, deep comprehension, which requires a complete mental model, benefits from drawing. The generative theory of drawing construction [36] helps explain the benefit of drawing on conceptual understanding and comprehension. According to this theory, readers construct a verbal representation of written words and a visual representation when drawing. Constructing a mental model of the content involves (a) selecting key elements from the verbal and visual representations, (b) organizing the key elements and connecting them to prior knowledge, and (c) integrating the verbal and visual representations into a coherent mental model. Thus, a drawing generated while reading represents a reader's integrated verbal and visual representations, which may provide a more coherent representation of a phenomenon that a representation based purely on verbal information (e.g., a summary of a text).

A high quality drawing connects key elements and illustrates how the system as a whole functions. If a person can create a high quality drawing, he or she should be able to perform well on a test of deeper comprehension because the drawing and the test both depend on a coherent mental model. If a person cannot generate a high quality drawing, he or she should not be able to perform well on a test of deeper comprehension. Therefore, the quality of a drawing should be predictive of performance on a test of comprehension—and using drawings as a cue for judging comprehension should promote high levels of metacomprehension accuracy. Thus, drawing while reading has potential as an encoding-based approach to improving

Drawings have also been shown to provide valuable feedback regarding level of understanding [37]. That is, drawings help students identify gaps in understand. Thus, drawing also has potential as a retrieval-based approach to improving meta-

Despite the theoretical appeal of using drawings to improve metacomprehension accuracy, only recently have researchers examined the effect of drawing on accuracy. In particular, drawing has been used as an encoding task [38, 39] and as a retrieval task [40]. The results of these studies are mixed; however, methodological

Drawing had no effect on metacomprehension accuracy in two studies [38, 40]. In these studies, rather than read a set of different texts and generate a drawing for each, participants read contiguous texts and generate a *single* drawing based on all the texts. Although generating a single drawing might help participants create a model for all the texts, generating a single drawing would not likely provide cues to help participants differentiate more-understood from less-understood texts. Without cues for individual texts to help differentiate texts, it is not surprising that

By contrast, Thiede et al. [39] had fifth grade students generate drawings for different science text while they read. Student then predicted their performance and completed a test for each text. This is the standard experimental procedure with the encoding-based approach to influence metacomprehension, as illustrated in **Figure 1**. A key finding of this study was that drawing dramatically improved metacomprehension accuracy when students received instruction on generating

differences make it difficult to compare the results across studies.

drawing did not improve metacomprehension accuracy.

**70**

According to the cue-utilization framework, monitoring accuracy is dependent on cue diagnosticity (how predictive a cue is of test performance) and cue utilization (which cues a person uses for the metacognitive judgment). van Loon et al. [33] developed a procedure to decompose judgment accuracy into these two components. In particular, they examined the diagnosticity of a cue by computing the correlation between the cue and test performance across texts. Similarly, they examined cue utilization by computing the correlation between the cue and the metacomprehension judgment across the texts. As in Thiede et al. [39], we used an experimental design to examine the effect of drawing instruction on cue diagnosticity and cue utilization.

We evaluated the effect of two kinds of drawing instruction on cue diagnosticity and cue utilization. Ninety-two fifth grade students were randomly assigned to two instructional groups. Students in each group read five texts on different science topics and generated drawings as they read. They then predicted their performance, and completed a test for each text. The Organizational-Drawing group (*n* = 47) received instruction on generating organizational drawing of scientific texts, which emphasized including relational information in their drawing. The Representational-Drawing group (*n* = 45) received instruction on generating representational drawing, which emphasized including many elements in their drawings. As the organizational instructions were designed to promote connecting ideas in the text to each other and to prior knowledge, we hypothesized that this group would generate more diagnostic cues than would the group receiving representational instructions.

#### **6.2 Potential cues for metacomprehension judgments**

As noted above, theories of comprehension, like the construction integration model [15], define deeper comprehension as a representation of a text that includes connections of ideas contained in a text to each other and prior knowledge (the situation model). The metacomprehension literature suggests that metacomprehension accuracy improves when people base their metacomprehension judgments on cues related to their situation model. Moreover, studies of self-reported cue use provide evidence that accuracy is greater for people who report using cues related to the situation model (i.e., their ability to link ideas contained in a text) than for people who reported using other cues [21]. Thus, cues that provide information related to connecting ideas and use of prior knowledge should be highly diagnostic.

To examine cue diagnosticity and cue utilization of drawings, we refined the graphic analysis protocol (GAP), which had been used to score graphics contained in science textbooks [41, 42], to score student drawings of scientific texts. The GAPdrawing provides a more fine-grained measure of drawing quality than the overall measure of quality typically used in drawing literature [43]. The GAP-drawing provides scores on two broad dimensions: drawing content and drawing relations.

*Drawing Content* describes the composition and substance of drawings. For each text, we created a master list of the actions, elements, and big ideas described in a text. We then scored each drawing for the number of these attributes. We

also scored drawings for the number of novel elements related to the topic but not explicitly described in the text and unrelated elements.

*Drawing Relations* describes the relations among the elements in the drawing. Based on the definition of systematicity for published graphics, the *systematicity* of drawings describes how well the drawing demonstrates that a reader has built a situation model of the system described in a text. *Systematicity* ranges from a score of 1 (low) indicates the drawing illustrated isolated units, not integrated into a larger system, 2 (medium) indicates the drawing has some aspects of the system, and 3 (high) indicates the drawing is a complete model of the system. *Semantic relations* describe how the text and drawing are related. Drawings earn a score of 0 when they are only vaguely related to the text context, 1 (representational) when drawings directly show what was described in the text, 2 (organizational) when drawings add coherence by putting the information within a greater scheme or system, and 3 (interpretational) when drawings that contain both representation and organizational elements, but extend this by showing how the elements are related. *Connections* describe whether drawings represent the information in the text and include information from the reader's background knowledge or prior learning. A drawing scored as 0 does not add information not present in the text; 1 provides additional examples of a topic described in the text; 2 indicates the drawing includes additional examples of a process or phenomena not explicitly described in the text; and 3 appropriately connects the information to a different field of scientific study. Captions and labels can identify the parts of a diagram, the steps in a process or both. We categorized the *captions* and/or labels on a scale of 0–4. A score of 0 indicates a lack of captions, a 1 indicates that captions only identify the target of the graphic, a 2 indicates the captions identify parts, a 3 indicates captions identify the steps in a system, and a 4 indicates that the captions identify both the parts and steps in a system. We hypothesized that drawing relations metrics would be more diagnostic than drawing content because these metrics capture features of a situation model.

For each text, students generated a drawing as they read. Students also made a metacomprehension judgment (i.e., they predicted their performance on a five-item test of comprehension) and completed an inference test of reading comprehension for each text. Drawings were scored using the GAP-drawing. Cue diagnosticity was operationalized as the intra-individual correlation between drawing metrics and test performance. Cue utilization was operationalized as the intra-individual correlation between drawing metrics and metacomprehension judgments. To illustrate these measures and how cue diagnosticity and cue utilization influence metacomprehension accuracy, consider the example shown in **Table 1**.

For the student below, the number of elements was fairly weakly correlated with test performance, which indicates this is not diagnostic of performance on the test of comprehension. The number of big ideas was more strongly correlated with test performance than was the number of elements, but the correlation is only moderate. By contrast, the connections are perfectly correlated with test performance test performance was higher for texts with higher connections scores and lower for texts with lower connections scores. Connections are a highly diagnostic cue of comprehension. Regarding cue utilization, the number of elements is weakly and negatively correlated with metacomprehension judgments, the number of big ideas was moderately correlated with judgments, and connections was highly correlated with judgments. These correlations suggest that this student used connections as bases of metacomprehension judgments and relied less on the number of big ideas and the number of elements to judged comprehension.

Cue diagnosticity and cue utilization help explain the relative high level of metacomprehension accuracy for this student (metacomprehension accuracy = 0.78).

**73**

*Drawings as Diagnostic Cues for Metacomprehension Judgment*

**Text Judgment Performance Number** 

For this student, connections were a highly diagnostic cue and the students used this cue for judging comprehension accuracy. Had this student relied heavily on the number of elements to judge comprehension, metacomprehension would have been reduced because the number of elements is not predictive of test performance.

Number of elements 0.20 −0.11 Number of big ideas 0.40 0.33 Connections 1.00 0.75

Text 1 5 4 12 5 3 Text 2 2 3 21 3 2 Text 3 4 3 18 1 1 Text 4 2 1 10 4 1 Text 5 1 0 16 2 0

**of elements**

Cue diagnosticity Cue utilization

**Number of big ideas**

**Connections**

This chapter focuses on cue diagnosticity and cue utilization; however, it is important to note that metacomprehension accuracy was significantly greater for the Organizational-Drawing group (mean metacomprehension accuracy = 0.51) than for the Representational-Drawing group (mean metacomprehension accuracy = −0.03). Cue diagnosticity and cue utilization help explain the difference in

As shown in **Table 2**, several drawing metrics were predictive of performance on tests of comprehension for the Organizational-Drawing group. In particular, for this group, systematicity, semantic relations, connections and the number of big ideas were are significantly correlated with test performance. By contrast, for the Representational-Drawing group, none of the drawing metrics were predictive of

These results suggest that instruction on how to generate drawings significantly

To better understand how these cues might affect metacomprehension accuracy, we need to examine cue utilization. As shown in **Table 3**, for the Organizational-Drawing group, a variety of drawing metrics were correlated with metacomprehension judgments, which suggests students in this group utilized a number of different drawing metrics in making their judgments. Most importantly, this group utilized four of the cues that were highly diagnostic of performance on comprehension test

affects cue diagnosticity. That is, with instruction on how to generate organizational drawings, drawing metrics related to connecting ideas to one another and to prior knowledge are predictive of performance on a test of comprehension (see the rightmost column of **Table 2**). It is important to note that the cues identified as diagnostic for this group are those hypothesized to be predictive of comprehension by theories of comprehension. Without instruction on generating organizational drawings, drawings do not provide diagnostic cues. Thus, for this group, drawing

does little to provide useful cues for judging comprehension.

*DOI: http://dx.doi.org/10.5772/intechopen.86959*

Metacomprehension accuracy = 0.78

*Sample data to illustrate cue diagnosticity and cue utilization.*

**6.3 Results of study**

**Table 1.**

accuracy across groups.

comprehension test performance.


*Drawings as Diagnostic Cues for Metacomprehension Judgment DOI: http://dx.doi.org/10.5772/intechopen.86959*

#### **Table 1.**

*Metacognition in Learning*

a situation model.

also scored drawings for the number of novel elements related to the topic but not

*Drawing Relations* describes the relations among the elements in the drawing. Based on the definition of systematicity for published graphics, the *systematicity* of drawings describes how well the drawing demonstrates that a reader has built a situation model of the system described in a text. *Systematicity* ranges from a score of 1 (low) indicates the drawing illustrated isolated units, not integrated into a larger system, 2 (medium) indicates the drawing has some aspects of the system, and 3 (high) indicates the drawing is a complete model of the system. *Semantic relations* describe how the text and drawing are related. Drawings earn a score of 0 when they are only vaguely related to the text context, 1 (representational) when drawings directly show what was described in the text, 2 (organizational) when drawings add coherence by putting the information within a greater scheme or system, and 3 (interpretational) when drawings that contain both representation and organizational elements, but extend this by showing how the elements are related. *Connections* describe whether drawings represent the information in the text and include information from the reader's background knowledge or prior learning. A drawing scored as 0 does not add information not present in the text; 1 provides additional examples of a topic described in the text; 2 indicates the drawing includes additional examples of a process or phenomena not explicitly described in the text; and 3 appropriately connects the information to a different field of scientific study. Captions and labels can identify the parts of a diagram, the steps in a process or both. We categorized the *captions* and/or labels on a scale of 0–4. A score of 0 indicates a lack of captions, a 1 indicates that captions only identify the target of the graphic, a 2 indicates the captions identify parts, a 3 indicates captions identify the steps in a system, and a 4 indicates that the captions identify both the parts and steps in a system. We hypothesized that drawing relations metrics would be more diagnostic than drawing content because these metrics capture features of

For each text, students generated a drawing as they read. Students also made a metacomprehension judgment (i.e., they predicted their performance on a five-item test of comprehension) and completed an inference test of reading comprehension for each text. Drawings were scored using the GAP-drawing. Cue diagnosticity was operationalized as the intra-individual correlation between drawing metrics and test performance. Cue utilization was operationalized as the intra-individual correlation between drawing metrics and metacomprehension judgments. To illustrate these measures and how cue diagnosticity and cue utilization influence metacom-

For the student below, the number of elements was fairly weakly correlated with test performance, which indicates this is not diagnostic of performance on the test of comprehension. The number of big ideas was more strongly correlated with test performance than was the number of elements, but the correlation is only moderate. By contrast, the connections are perfectly correlated with test performance test performance was higher for texts with higher connections scores and lower for texts with lower connections scores. Connections are a highly diagnostic cue of comprehension. Regarding cue utilization, the number of elements is weakly and negatively correlated with metacomprehension judgments, the number of big ideas was moderately correlated with judgments, and connections was highly correlated with judgments. These correlations suggest that this student used connections as bases of metacomprehension judgments and relied less on the number of big ideas

Cue diagnosticity and cue utilization help explain the relative high level of metacomprehension accuracy for this student (metacomprehension accuracy = 0.78).

prehension accuracy, consider the example shown in **Table 1**.

and the number of elements to judged comprehension.

explicitly described in the text and unrelated elements.

**72**

*Sample data to illustrate cue diagnosticity and cue utilization.*

For this student, connections were a highly diagnostic cue and the students used this cue for judging comprehension accuracy. Had this student relied heavily on the number of elements to judge comprehension, metacomprehension would have been reduced because the number of elements is not predictive of test performance.

#### **6.3 Results of study**

This chapter focuses on cue diagnosticity and cue utilization; however, it is important to note that metacomprehension accuracy was significantly greater for the Organizational-Drawing group (mean metacomprehension accuracy = 0.51) than for the Representational-Drawing group (mean metacomprehension accuracy = −0.03). Cue diagnosticity and cue utilization help explain the difference in accuracy across groups.

As shown in **Table 2**, several drawing metrics were predictive of performance on tests of comprehension for the Organizational-Drawing group. In particular, for this group, systematicity, semantic relations, connections and the number of big ideas were are significantly correlated with test performance. By contrast, for the Representational-Drawing group, none of the drawing metrics were predictive of comprehension test performance.

These results suggest that instruction on how to generate drawings significantly affects cue diagnosticity. That is, with instruction on how to generate organizational drawings, drawing metrics related to connecting ideas to one another and to prior knowledge are predictive of performance on a test of comprehension (see the rightmost column of **Table 2**). It is important to note that the cues identified as diagnostic for this group are those hypothesized to be predictive of comprehension by theories of comprehension. Without instruction on generating organizational drawings, drawings do not provide diagnostic cues. Thus, for this group, drawing does little to provide useful cues for judging comprehension.

To better understand how these cues might affect metacomprehension accuracy, we need to examine cue utilization. As shown in **Table 3**, for the Organizational-Drawing group, a variety of drawing metrics were correlated with metacomprehension judgments, which suggests students in this group utilized a number of different drawing metrics in making their judgments. Most importantly, this group utilized four of the cues that were highly diagnostic of performance on comprehension test


*Note: the number in parentheses is the standard error of the mean. \**

*Indicates a correlation is significantly different than zero (p < 0.05).*

#### **Table 2.**

*Cue diagnosticity for drawing metrics by group.*


*\* Indicates a correlation is significantly different than zero (p < 0.05).*

#### **Table 3.**

*Cue utilization for drawing metrics by group.*

(i.e., systematicity, semantic relations, connections and the number of big ideas). By contrast, for the Representational-Drawing group, only connections were correlated with metacomprehension judgments. However, for this group, connections were not correlated with test performance; therefore, utilizing this cue would not contribute to a high level of judgment accuracy.

These results provide additional empirical evidence that metacomprehension accuracy is influenced by cue diagnosticity and cue utilization. Metacomprehension accuracy was greater for the Organizational-Drawing group than the Representational-Drawing group. Drawings provided diagnostic cues for the Organizational-Drawing group but not for the Representational-Drawing group. Moreover, diagnostic cues were utilized for metacomprehension judgments for the Organizational-Drawing group but not for the Representational-Drawing group.

**75**

*Drawings as Diagnostic Cues for Metacomprehension Judgment*

Metacomprehension accuracy is important to reading comprehension because monitoring guides decisions about rereading [31, 44], which improves overall comprehension [32, 45]. Thus, it is important to find ways to improve metacompre-

The cue-utilization framework of metacognitive monitoring [14] suggests improving monitoring accuracy involves identifying cues that are highly diagnostic of test performance and then instructing people to use those cues when making judgments. Thus, as described above, researchers have employed a variety of techniques to help facilitate the construction of a situation model or access the situation model prior to judging comprehension because this arguably provides cues that are highly diagnostic of comprehension tests. Researchers have also employed other techniques to promote use of diagnostic cues when making metacomprehension

Recent research using drawings as an encoding task shows promise in improving metacomprehension accuracy. This research shows that drawings need to emphasize the underlying organization of the phenomenon described in the text to improve metacomprehension accuracy, which is consistent with research showing the effect of graphics on metacomprehension accuracy is determined by the nature of the graphics presented with texts [46–48]. Specifically, organizational graphics improved metacomprehension accuracy and other graphics have little or adverse

The GAP-drawing provides a scoring system to help identify specific attributes of drawings that could be diagnostic of comprehension and utilized as a basis for metacomprehension judgments. Our findings suggest that with instruction on generating organizational drawings while reading, metrics related to drawing relations are predictive of test performance (diagnostic). Moreover, the instruction promoted

Instructions focused on generating organizational drawings improved metacomprehension accuracy and comprehension. Thus, drawing can influence learning. More research is needed to identify the most effective instruction for drawing. With attention to cue diagnosticity and cue utilization, this research could reshape the

Keith Thiede\*, Katherine L. Wright, Sara Hagenah and Julianne Wenner

© 2019 The Author(s). Licensee IntechOpen. This chapter is distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/ by/3.0), which permits unrestricted use, distribution, and reproduction in any medium,

\*Address all correspondence to: keiththiede@boisestate.edu

*DOI: http://dx.doi.org/10.5772/intechopen.86959*

effects on metacomprehension accuracy [47].

field of metacomprehension.

Boise State University, Boise, Idaho, U.S.A.

provided the original work is properly cited.

**Author details**

utilization of these cues when judging comprehension.

**7. Conclusions**

hension accuracy.

judgments [18].
