**5. The meaning of language (I)**

76 Neuroimaging – Cognitive and Clinical Neuroscience

more syntactic in nature, conforming a somehow unitary system together with part of BA 44; the anterior part, in turn, would be more semantic, working together with BA 47. Overall, the above-mentioned functional gradient within the left IFG might actually be more gradual than the labels currently available to describe it (i.e., semantic, syntactic, and articulatory/phonological), which might also explain why the ventral portion of BA 6, most consistently described as an articulatory/phonological area (it actually belongs to the premotor cortex) has also been seen occasionally involved in detecting syntactic errors or analyzing syntactically ambiguous sentences (Christensen, 2010; Friederici et al., 2006). The picture can be yet more complicated when considering that even language processing at the

discourse level consistently recruits large portions of the IFG, as we will see below.

particularly the SMG (Binder et al, 2009).

Fig. 2. Approximate locations of the syntactic system

Additionally, a number of studies also support the involvement of other regions apart from the IFG in syntactic processing. One of the most consistent findings in this regard is the existence of a fronto-temporal network supporting syntactic processing. Whereas the frontal pole of this network implies the left IFG, especially in and around BA 44, the temporal portion is mainly comprising the left superior temporal gyrus (STG) and superior temporal sulcus (STS), most likely excluding primary auditory areas (i.e., BA 41 and 42), and roughly corresponding to BA 22 (Christensen, 2010; Rodd et al., 2010). Interestingly, these activations may plainly include Wernicke's area (Shetreet et al., 2009), which is mainly involved in the processing of language sounds (see our previous section in this regard). In several studies, large (anterior and posterior) portions of BA 22 appear relevant in syntactic processes (Friederici, 2002; Rodd et al., 2010). In other occasions, however, it is only a small portion of BA 22 what is involved, such as the posterior portion of the STS (Pallier et al., 2011). Upper portions of the middle temporal gyrus (MTG), comprising part of BA 21, have been also reported to participate in syntactic processing (Christensen, 2010; Friederici et al., 2006; Shetreet et al., 2009). Interestingly, although the main findings are located within the left hemisphere, occasional activations in corresponding areas of the right hemisphere are also reported. Several studies also report activations in the ventral portion of the supramarginal gyrus (SMG, part of BA 40), together with the *planum temporale*, a posterior portion of BA 22 (Raettig et al., 2010), although these regions appear more consistently as rather semantic,

Finally, still other brain regions have been seen also involved in syntactic processing, though less consistently. Among these, we can mention the precuneus (in the medial Linguistic messages normally tell something about the world and its components (objects, persons, places, and so forth). As such, a linguistic message includes words (word-forms with their individual semantic contents or meanings) that are combined, usually through syntax, rendering a concrete description of their relationships intended to mirror a real situation or an idea. Within the brain, indeed, extensive regions of the cerebral cortex appear devoted to semantic information processing. This seemingly provides a clue on the relationships between language and other cognitive processes. But it also poses some doubts on the boundaries between what can be labeled as "linguistic" and "non-linguistic". It is also the case that "semantic" might appear as a rather vague and imprecise term, covering a large number of otherwise different processes or operations. In fact, terms as "pragmatic", "conceptual", as well as several others, often appear next to "semantic" as equivalent or corresponding to a somehow unitary system.

Actually, the meaning of words, one of the main features that the term semantic can refer to, can be just about anything in the world. In other words, human languages have words –and, then, meanings- for absolutely all (or almost all) things known so far in the world, be they real, imagined, or with a large amount of ambiguity and abstraction (e.g., Pinker, 2007). In this regard, some authors even think that the so-called "syntactic words" (i.e., complements, determiners, suffixes, and so many words or particles with a specific syntactic function) have also a meaning to be considered as plainly pertaining to the same semantic system of the brain as any other type of content word, such as nouns or verbs. In this line, it could be the case that syntactic regions reviewed above might be part of the "semantic" system, but only the part preferentially dealing with abstract structural hierarchical relationships between a number of items, be they words or whatever. Indeed, it is a plausible scenario that syntax words emerged initially as any other, less functional words during the evolution of human language (Heine & Kuteva, 2007).

This said, it should not be surprising that semantic areas have been proposed to occupy most of the cerebral cortex. Providentially, Binder et al. (2009) have recently published an extensive review of functional neuroimaging studies of semantic processing, in which not only strict inclusion criteria were applied but also advanced statistical analyses for determining the probability of a given region as belonging to the semantic system. The studies included in that review used words as stimuli, so that we can be certain that the areas suggested as supporting the semantic system are indeed areas activated by language. This note is important because, as we will see, the areas constituting the semantic system are actually and for the most part classically considered as heteromodal association areas of the neocortex, located both in frontal and posterior regions. They are therefore common to a large amount of non-linguistic processes involved in either perception or action. Additionally, areas of the limbic system involved in emotional processing are also part of the semantic system.

According to the review by Binder et al. (2009), the semantic system in the brain can be subdivided into three main widespread locations. A first one includes large portions of the posterior multimodal and heteromodal cortex, namely the angular gyrus (AG, in BA 39), the

Functional and Structural Magnetic Resonance Imaging of Human Language: A Review 79

"smile"), they activate corresponding areas for these actions within the primary motor cortex (BA 4), also largely respecting its somatotopic organization. The same is the case for words referring to specific stimulus features, or in which these features are crucial in their definition. Words such as "ellipse" or "red", or words belonging to semantic categories in which visual features prevail (like "animals"), activate visual areas specifically related to the processing of those perceptual features. Binder and colleagues, in their 2009 review, mention this type of findings, but consider them as secondary, less conspicuous and consistent than the other regions substantiating their proposal. It is possible, nonetheless, that the participation of these primary regions in the semantic system of the brain is less systematic namely because they refer to very specific actions or perceptions, so that only the linguistic material referring to these very concrete body features would activate them. This would not be the case, however, of the heteromodal and multimodal association areas, the main areas according to Binder et al. (2009) review, which by definition would be activated by any

stimulus of any modality. This depiction is supported in Pulvermüller et al. (2009a).

imagery).

By considering the semantic system as composed by both heteromodal and multimodal association areas as well as by primary or secondary areas of the perceptual and motor systems extends the size of the semantic system and is a very plausible scenario. Under this perspective, the semantic system would be substantiated by the cortical circuits involved in all of our interactions with the world; the semantic system would be equivalent to our whole "world knowledge" system. Part of this knowledge is concrete, but also part of it refers to abstractions and relations performed in the heteromodal and multimodal association areas. The involvement of limbic regions in the semantic system also fits with this line of reasoning, since emotions are also an important part of our world knowledge. Indeed, this depiction harmonizes well with recent theories of "embodied language" (e.g., de Vega et al., 2008), according to which language directly and straightforwardly makes use of the brain areas involved in performing or processing what is described in an utterance. Embodied language theories contrast with traditional proposals for a more "abstract" code created by language (or from which language emerges, in case of production) that can in turn be converted into the mental representation of specific perceptions and actions. Both views could complement each other, however, if both abstract and "body" codes working simultaneously and in cooperation are accepted. The former would be related to heteromodal and multimodal association areas, the latter to more primary or secondary areas. Indeed, not all that can be uttered can be visualized or executed externally. We can also add that there is a noticeable overlap between the most abstract and heteromodal portions of the semantic system -i.e., the proposed by Binder et al. (2009)- and part of the so-called "human default system", a rather bilateral network of activations in the human brain appearing when the subject is involved in mental tasks other than those linked to externally present stimuli or tasks. The fact that the human default system is involved in such a variety of mental operations as autobiographical memory, envisioning the future, theory of mind, or moral decision making, among many others (for a good review, see Buckner et al., 2008), suggests that this system can eventually apply to circumstances that can be visualized or externally performed. If we apply the same reasoning as we did above for the semantic system, the parallelisms between both systems are more apparent, as situations supported by the human default system should also involve the occasional recruitment of more primary or secondary perceptual or motor cortical areas in order to imagine specific perceptions and actions. This is a very plausible scenario (see, e.g., Kosslyn & Thompson, 2003 for fMRI evidence of primary visual areas activated in visual

SMG, and the MTG, including part of the temporal pole (comprising small parts of BA 38 and 29). Also in the posterior parts of the brain other areas highly involved in semantics are basal temporal areas, particularly within the fusiform and parahippocampal gyri (comprising portions of BA 20 and 37; mainly high-order visual regions, as we saw for reading). A second location of the semantic system comprises portions of the heteromodal frontal cortex, namely the upper and –especially- medial portions of BA 8 and part of BA 9. Interestingly, BA 8 contains the supplementary motor area (SMA) and has been seen involved in a variety of tasks, including motor learning and imagery (Malouin et al., 2003; Matsumura et al., 2004), executive functions and planning (Kübler et al., 2006), and even speech motor programming (de Waele et al., 2001). On the other hand, BA 9 is also involved in executive functions (e.g. Kübler et al., 2006) and, as we will see below, in discourse processing. Another frontal heteromodal association area included in the second locus of the semantic system is BA 47 in the IFG, an area that was already mentioned in our previous section on syntactic processing, its most probable function being related to semantic/pragmatic unification. Finally, the third group of areas supporting the semantic system according to Binder et al. (2009) includes the posterior cingulate/precuneus region and the ventromedial prefrontal cortex. Whereas the former has been seen to be associated with emotional processing (Maddock, 1999), it has also been related to visuospatial memory and imagery (Burgess, 2008; Epstein et al., 2007), among several other functions, including – occasionally, as we saw- syntax. Indeed, the role of this area seems rather polyvalent; later on, we will see that it is implicated in the semantic analysis of whole sentences and longer language emissions (discourses). The ventromedial prefrontal cortex, roughly corresponding to BA 11 and other BAs (such as portions of the most ventral parts of BA 10, 24, 25, and 32), comprising the rostral part of the anterior-ventral cingulate, is linked to motivation, emotion, and decision making involving reward (e.g., Ernst et al., 2004), among several other functions such as olfaction (Royet et al., 1999).

It must be remarked that all the regions outlined so far in Binder et al. (2009) as constituting the semantic system are mainly and preferentially in the left hemisphere, in consonance with the fact that they were circuits activated by words. This in turn harmonizes with the left-lateralization of other linguistic functions, such as syntax and phonetics. Overall, it seems that the semantic system activated by words largely overlaps with the system used by our brain to understand and process, as well as to interact with, the external world. In fact, this is what language crucially conveys in the very end. We have seen that words can activate association areas involved in action planning, perception, and emotions, and certainly the meaning of words ultimately refers to any of these things, or to a combination of them. However, it is also possible that the view of the semantic system sketched by Binder et al. (2009) is to some extent a restrictive one. Indeed, interactions with the external (or internal) reality imply not only heteromodal association areas, but also more primary areas. Actually, the brain areas directly supporting body movements or first stages in the perceptual processing might also be part of our semantic system. This is the idea endorsed by Pulvermüler and colleages (e.g., Pulvermüler, 2010; Pulvermüller, & Fadiga, 2010). An overview of fMRI evidences in this regard by these authors (e.g., Boulenger et al., 2009), as well as by other groups (e.g., Martin & Chao, 2001; Tomasino et al., 2007) can be summarized as supporting that words referring to concepts in which movements or actions are crucial (e.g., tools, as well as many verbs), activate cortical areas specifically devoted to directly perform those movements or actions. When the words refer to arm movements (e.g., "catch"), leg movements (e.g., "run"), or face movements (as any facial expression, like

SMG, and the MTG, including part of the temporal pole (comprising small parts of BA 38 and 29). Also in the posterior parts of the brain other areas highly involved in semantics are basal temporal areas, particularly within the fusiform and parahippocampal gyri (comprising portions of BA 20 and 37; mainly high-order visual regions, as we saw for reading). A second location of the semantic system comprises portions of the heteromodal frontal cortex, namely the upper and –especially- medial portions of BA 8 and part of BA 9. Interestingly, BA 8 contains the supplementary motor area (SMA) and has been seen involved in a variety of tasks, including motor learning and imagery (Malouin et al., 2003; Matsumura et al., 2004), executive functions and planning (Kübler et al., 2006), and even speech motor programming (de Waele et al., 2001). On the other hand, BA 9 is also involved in executive functions (e.g. Kübler et al., 2006) and, as we will see below, in discourse processing. Another frontal heteromodal association area included in the second locus of the semantic system is BA 47 in the IFG, an area that was already mentioned in our previous section on syntactic processing, its most probable function being related to semantic/pragmatic unification. Finally, the third group of areas supporting the semantic system according to Binder et al. (2009) includes the posterior cingulate/precuneus region and the ventromedial prefrontal cortex. Whereas the former has been seen to be associated with emotional processing (Maddock, 1999), it has also been related to visuospatial memory and imagery (Burgess, 2008; Epstein et al., 2007), among several other functions, including – occasionally, as we saw- syntax. Indeed, the role of this area seems rather polyvalent; later on, we will see that it is implicated in the semantic analysis of whole sentences and longer language emissions (discourses). The ventromedial prefrontal cortex, roughly corresponding to BA 11 and other BAs (such as portions of the most ventral parts of BA 10, 24, 25, and 32), comprising the rostral part of the anterior-ventral cingulate, is linked to motivation, emotion, and decision making involving reward (e.g., Ernst et al., 2004), among

It must be remarked that all the regions outlined so far in Binder et al. (2009) as constituting the semantic system are mainly and preferentially in the left hemisphere, in consonance with the fact that they were circuits activated by words. This in turn harmonizes with the left-lateralization of other linguistic functions, such as syntax and phonetics. Overall, it seems that the semantic system activated by words largely overlaps with the system used by our brain to understand and process, as well as to interact with, the external world. In fact, this is what language crucially conveys in the very end. We have seen that words can activate association areas involved in action planning, perception, and emotions, and certainly the meaning of words ultimately refers to any of these things, or to a combination of them. However, it is also possible that the view of the semantic system sketched by Binder et al. (2009) is to some extent a restrictive one. Indeed, interactions with the external (or internal) reality imply not only heteromodal association areas, but also more primary areas. Actually, the brain areas directly supporting body movements or first stages in the perceptual processing might also be part of our semantic system. This is the idea endorsed by Pulvermüler and colleages (e.g., Pulvermüler, 2010; Pulvermüller, & Fadiga, 2010). An overview of fMRI evidences in this regard by these authors (e.g., Boulenger et al., 2009), as well as by other groups (e.g., Martin & Chao, 2001; Tomasino et al., 2007) can be summarized as supporting that words referring to concepts in which movements or actions are crucial (e.g., tools, as well as many verbs), activate cortical areas specifically devoted to directly perform those movements or actions. When the words refer to arm movements (e.g., "catch"), leg movements (e.g., "run"), or face movements (as any facial expression, like

several other functions such as olfaction (Royet et al., 1999).

"smile"), they activate corresponding areas for these actions within the primary motor cortex (BA 4), also largely respecting its somatotopic organization. The same is the case for words referring to specific stimulus features, or in which these features are crucial in their definition. Words such as "ellipse" or "red", or words belonging to semantic categories in which visual features prevail (like "animals"), activate visual areas specifically related to the processing of those perceptual features. Binder and colleagues, in their 2009 review, mention this type of findings, but consider them as secondary, less conspicuous and consistent than the other regions substantiating their proposal. It is possible, nonetheless, that the participation of these primary regions in the semantic system of the brain is less systematic namely because they refer to very specific actions or perceptions, so that only the linguistic material referring to these very concrete body features would activate them. This would not be the case, however, of the heteromodal and multimodal association areas, the main areas according to Binder et al. (2009) review, which by definition would be activated by any stimulus of any modality. This depiction is supported in Pulvermüller et al. (2009a).

By considering the semantic system as composed by both heteromodal and multimodal association areas as well as by primary or secondary areas of the perceptual and motor systems extends the size of the semantic system and is a very plausible scenario. Under this perspective, the semantic system would be substantiated by the cortical circuits involved in all of our interactions with the world; the semantic system would be equivalent to our whole "world knowledge" system. Part of this knowledge is concrete, but also part of it refers to abstractions and relations performed in the heteromodal and multimodal association areas. The involvement of limbic regions in the semantic system also fits with this line of reasoning, since emotions are also an important part of our world knowledge. Indeed, this depiction harmonizes well with recent theories of "embodied language" (e.g., de Vega et al., 2008), according to which language directly and straightforwardly makes use of the brain areas involved in performing or processing what is described in an utterance. Embodied language theories contrast with traditional proposals for a more "abstract" code created by language (or from which language emerges, in case of production) that can in turn be converted into the mental representation of specific perceptions and actions. Both views could complement each other, however, if both abstract and "body" codes working simultaneously and in cooperation are accepted. The former would be related to heteromodal and multimodal association areas, the latter to more primary or secondary areas. Indeed, not all that can be uttered can be visualized or executed externally. We can also add that there is a noticeable overlap between the most abstract and heteromodal portions of the semantic system -i.e., the proposed by Binder et al. (2009)- and part of the so-called "human default system", a rather bilateral network of activations in the human brain appearing when the subject is involved in mental tasks other than those linked to externally present stimuli or tasks. The fact that the human default system is involved in such a variety of mental operations as autobiographical memory, envisioning the future, theory of mind, or moral decision making, among many others (for a good review, see Buckner et al., 2008), suggests that this system can eventually apply to circumstances that can be visualized or externally performed. If we apply the same reasoning as we did above for the semantic system, the parallelisms between both systems are more apparent, as situations supported by the human default system should also involve the occasional recruitment of more primary or secondary perceptual or motor cortical areas in order to imagine specific perceptions and actions. This is a very plausible scenario (see, e.g., Kosslyn & Thompson, 2003 for fMRI evidence of primary visual areas activated in visual imagery).

Functional and Structural Magnetic Resonance Imaging of Human Language: A Review 81

Human language normally extends beyond sentences, most often consisting of longer messages usually known as discourses, narratives, stories, or –simply- texts. The global or unitary comprehension of a group of sentences would involve even more associative areas than simpler levels, namely because additional processes, known to demand higher levels of association, play a relevant role in discourse comprehension. Among these additional processes, the most outstanding appear to be the achievement of *inferences* and *pragmatic interpretations*, obtained by using world-knowledge and discourse context constantly in interaction. Finally, according to the so-called "situational models" (Zwaan, 2004), it is also assumed that reading or hearing long linguistic messages or discourses conveys, when feasible, the recreation of the situation depicted in the text, normally by simulating or recreating the events described in the story. Results from the fMRI studies reported so far in

Experiments studying text comprehension have been very varied in their designs and procedures. In principle, whenever we have two or more sentences, the same processes presumably involved in text comprehension should be already in play, as linking two sentences would be sufficient to activate inferential and interpretive processes. Accordingly, if two sentences, as "*The telephone was ringing*" and "*My brother wanted to tell me the news*", are uttered consecutively, one understands that the second is an idea related to the first sentence; in this case, that it is my brother who was calling, and that the reason for his call was that my brother wanted to tell me something of interest. Without these inferences, performed easily and automatically and based on our world-knowledge, the two sentences would be just two senseless isolated emissions. A good example of the study of the coherence we normally achieve during text comprehension is extracted from Ferstl and von Cramon (2001), where coherence was compared with cohesion, that is, the linkage of two sentences by means of a cohesive element as "*therefore*" in the following pair of sentences: "*Mary's exam was about to start. Therefore, her palms were sweaty*". These two sentences are cohesive, due to the presence of a linking element, as well as coherent, since our worldknowledge tells us that sweaty palms are a possible consequence of being nervous, the latter being a normal consequence of an examination situation. In fact, both sentences are coherent even without the cohesive element. Now read the following pair of sentences: "*Mary's exam was about to start. Therefore, the pizza arrived*". Even with the cohesive element, these two sentences are not coherent; our knowledge of the world cannot help us to infer a possible

In other occasions, discourse processing has been studied using loosely structured passages rendered coherent only by providing a title or an illustration. As a good example, consider a portion of a classical ambiguous paragraph from Bransford and Johnson (1972): "*A newspaper is better than a magazine. The seashore is a better place than the street. At first it is better to run than to walk but walking is fine after a while. You may have to try several times, it takes skill but it's easy to learn. Even young children can enjoy it. Once successful there are very few complications. Birds seldom get too close. […]*". This paragraph is noticeably better understood and remembered- when preceded by the title: "*Making and flying a kite*". Comparing the processing of this type of paragraphs preceded by the title and the same paragraphs without the title would yield, for the former, the activation of brain areas supplying global coherence

Two recent reviews by Evelyn C. Ferstl and colleagues (Ferstl, 2010; Ferstl et al., 2008) -the first one using similar statistical methods as in Binder et al. (2009) for the semantic system-

and, for the latter, the attempts to attain it (Martín-Loeches et al., 2008).

**6. The meaning of language (II)** 

this topic seem to support this overall depiction.

logical link between these two utterances.

The main purpose of language comprehension is nevertheless not the understanding of single words within a given utterance, but rather the specific relationships between those words. Helping to determine these relationships is the main role of syntax, which in turn contributes to elucidate the semantic structure of the sentence. The latter is a semantic frame representing the actual relationships between the different entities (objects, persons, places, and so on) mentioned in a sentence. Brain activations related to these combinatorial or propositional semantic processes are normally obtained in experiments in which grammatically correct but semantically incongruent sentences are compared with semantically congruent or plausible sentences. As an example, compare "*She spreads the warm bread with shoes*" and the same sentence ending with "*butter*" instead. In other occasions, the activations produced by normal sentences have been compared with the activations produced by sentences composed of pseudowords, or "jabberwocky" sentences, in which a syntactic structure can still be determined but -given that pseudowords have no semantic content as they are not real words-, no semantic structure can theoretically be extracted. As an example, here is a portion of the poem by Lewis Carroll that gave birth to the name of this type of paradigm: "*Twas brillig, and the slithy toves / Did gyre and gimble in the wabe; / All mimsy were the borogoves, /And the mome raths outgrabe*". Overall, the activations observed with these experimental paradigms tap on several places within the "more general" or heteromodal semantic system for words commented so far. Two of these places locate within the parietotemporal junction and the temporal pole (Mashal et al., 2009; Obleser & Kotz, 2010; Pallier et al., 2011), that is, two portions within the posterior heteromodal association areas of the system used for words. Another portion of the semantic brain system for words that seems also importantly involved in the combination of semantic information is the posterior cingulate/precuneus (Mashal et al., 2009; Whitney et al., 2009). Finally, an area within the IFG belonging to the frontal heteromodal association areas of the semantic system -BA 47, eventually spreading to BA 45-, has also been seen implied in combinatorial semantic operations (reviewed in Hagoort, 2007). As can be seen, most of these loci have been implied in the processing of the semantic content of words; a few of them (BA 45, posterior cingulate/precuneus) have been occasionally observed in syntactic operations as well (particularly BA 45; see above). Interestingly, several of these regions also play a relevant role in discourse processing, as we will see in the next section.

Fig. 3. Approximate locations of the semantic system

### **6. The meaning of language (II)**

80 Neuroimaging – Cognitive and Clinical Neuroscience

The main purpose of language comprehension is nevertheless not the understanding of single words within a given utterance, but rather the specific relationships between those words. Helping to determine these relationships is the main role of syntax, which in turn contributes to elucidate the semantic structure of the sentence. The latter is a semantic frame representing the actual relationships between the different entities (objects, persons, places, and so on) mentioned in a sentence. Brain activations related to these combinatorial or propositional semantic processes are normally obtained in experiments in which grammatically correct but semantically incongruent sentences are compared with semantically congruent or plausible sentences. As an example, compare "*She spreads the warm bread with shoes*" and the same sentence ending with "*butter*" instead. In other occasions, the activations produced by normal sentences have been compared with the activations produced by sentences composed of pseudowords, or "jabberwocky" sentences, in which a syntactic structure can still be determined but -given that pseudowords have no semantic content as they are not real words-, no semantic structure can theoretically be extracted. As an example, here is a portion of the poem by Lewis Carroll that gave birth to the name of this type of paradigm: "*Twas brillig, and the slithy toves / Did gyre and gimble in the wabe; / All mimsy were the borogoves, /And the mome raths outgrabe*". Overall, the activations observed with these experimental paradigms tap on several places within the "more general" or heteromodal semantic system for words commented so far. Two of these places locate within the parietotemporal junction and the temporal pole (Mashal et al., 2009; Obleser & Kotz, 2010; Pallier et al., 2011), that is, two portions within the posterior heteromodal association areas of the system used for words. Another portion of the semantic brain system for words that seems also importantly involved in the combination of semantic information is the posterior cingulate/precuneus (Mashal et al., 2009; Whitney et al., 2009). Finally, an area within the IFG belonging to the frontal heteromodal association areas of the semantic system -BA 47, eventually spreading to BA 45-, has also been seen implied in combinatorial semantic operations (reviewed in Hagoort, 2007). As can be seen, most of these loci have been implied in the processing of the semantic content of words; a few of them (BA 45, posterior cingulate/precuneus) have been occasionally observed in syntactic operations as well (particularly BA 45; see above). Interestingly, several of these regions also play a relevant role

in discourse processing, as we will see in the next section.

Fig. 3. Approximate locations of the semantic system

Human language normally extends beyond sentences, most often consisting of longer messages usually known as discourses, narratives, stories, or –simply- texts. The global or unitary comprehension of a group of sentences would involve even more associative areas than simpler levels, namely because additional processes, known to demand higher levels of association, play a relevant role in discourse comprehension. Among these additional processes, the most outstanding appear to be the achievement of *inferences* and *pragmatic interpretations*, obtained by using world-knowledge and discourse context constantly in interaction. Finally, according to the so-called "situational models" (Zwaan, 2004), it is also assumed that reading or hearing long linguistic messages or discourses conveys, when feasible, the recreation of the situation depicted in the text, normally by simulating or recreating the events described in the story. Results from the fMRI studies reported so far in this topic seem to support this overall depiction.

Experiments studying text comprehension have been very varied in their designs and procedures. In principle, whenever we have two or more sentences, the same processes presumably involved in text comprehension should be already in play, as linking two sentences would be sufficient to activate inferential and interpretive processes. Accordingly, if two sentences, as "*The telephone was ringing*" and "*My brother wanted to tell me the news*", are uttered consecutively, one understands that the second is an idea related to the first sentence; in this case, that it is my brother who was calling, and that the reason for his call was that my brother wanted to tell me something of interest. Without these inferences, performed easily and automatically and based on our world-knowledge, the two sentences would be just two senseless isolated emissions. A good example of the study of the coherence we normally achieve during text comprehension is extracted from Ferstl and von Cramon (2001), where coherence was compared with cohesion, that is, the linkage of two sentences by means of a cohesive element as "*therefore*" in the following pair of sentences: "*Mary's exam was about to start. Therefore, her palms were sweaty*". These two sentences are cohesive, due to the presence of a linking element, as well as coherent, since our worldknowledge tells us that sweaty palms are a possible consequence of being nervous, the latter being a normal consequence of an examination situation. In fact, both sentences are coherent even without the cohesive element. Now read the following pair of sentences: "*Mary's exam was about to start. Therefore, the pizza arrived*". Even with the cohesive element, these two sentences are not coherent; our knowledge of the world cannot help us to infer a possible logical link between these two utterances.

In other occasions, discourse processing has been studied using loosely structured passages rendered coherent only by providing a title or an illustration. As a good example, consider a portion of a classical ambiguous paragraph from Bransford and Johnson (1972): "*A newspaper is better than a magazine. The seashore is a better place than the street. At first it is better to run than to walk but walking is fine after a while. You may have to try several times, it takes skill but it's easy to learn. Even young children can enjoy it. Once successful there are very few complications. Birds seldom get too close. […]*". This paragraph is noticeably better understood and remembered- when preceded by the title: "*Making and flying a kite*". Comparing the processing of this type of paragraphs preceded by the title and the same paragraphs without the title would yield, for the former, the activation of brain areas supplying global coherence and, for the latter, the attempts to attain it (Martín-Loeches et al., 2008).

Two recent reviews by Evelyn C. Ferstl and colleagues (Ferstl, 2010; Ferstl et al., 2008) -the first one using similar statistical methods as in Binder et al. (2009) for the semantic system-

Functional and Structural Magnetic Resonance Imaging of Human Language: A Review 83

The fourth and last region involved in discourse processing comprises the fronto-medial prefrontal cortex (dorso-medial and ventromedial prefrontal cortex), including large portions of the medial side of BAs 8, 9, 10, and 11. To some degree, all of these areas have been mentioned before as mainly implied in the semantic analysis of words. Even though, this grouping here is somewhat different. The main divergence is that the system comprising these areas for discourse processing complements with substantial additions of neural tissue. One addition is the involvement of the entire BA 10, including not only the medial parts, but also the lateral parts -even spreading to BA 46-, which was not the case in word semantics. The other addition conveys the whole anterior cingulate gyrus (in semantics for words, only a very small ventral anterior portion of this gyrus appeared involved). The involvement of these additional portions in discourse processing may convey important consequences. First, BA 10 is the largest cytoarchitectonic area in the human brain, having increased its size substantially during human evolution, as is the case for its connections (Semendeferi et al., 2001). Second, most of these connections seem to affect the anterior cingulate particularly (Allman et al., 2002), which is another milestone in human brain evolution. In fact, the anterior cingulate is so peculiar in the human brain that it is the main structure containing a special type of neurons, the spindle or Von Economo cells. Only the great apes within the primate order posses this type of neurons presumably related with complex social behaviour, humans exhibiting a disproportionate larger number of them (Allman et al., 2011). Third, BA 10 has recently revealed as the single region showing a significant effect unique to *g*, the psychometric construct of *general intelligence* (Gläscher et al., 2010). Consequently, although this is a group of areas directly involved in language

The same appears to be the case of the posterior cingulate/precuneus region, the third group of findings involved in discourse processing. The concrete role of the posterior cingulate/precuneus has yet to be elucidated. This region participates in many linguistic processes, but also in a number of other non-linguistic operations. As mentioned earlier, it appears a certainly polyvalent region, involved in emotion, memory, and imagery; it also belongs to the human default system, and it is one of the very few regions connected reciprocally with most other cortical regions. Indeed, this part of the brain is one of the main hubs of the "human core system", the anatomical counterpart of the human default system (Chudek et al., 2008). On the other hand, the anterior medial regions also involved in discourse processing are again largely overlapping with corresponding portions of the human default system. This outstanding overlap between brain systems for word semantics and, particularly, discourse processing and the human default system has been already raised by Binder et al. (2009) and Speer et al. (2009). Ferstl et al. (2008) and Ferstl (2010) focused on the similarities of the discourse-processing system and the system supporting theory of mind. However, considering that the circuits for theory of mind and the human default system have been seen to be largely equivalent (e.g. Buckner et al., 2008), Ferstl and colleagues' suggestion could surely be reworded to imply the default system. Overall, the human default system appears to be such a general and abstract-coded system that it can apply to a considerable number of situations and circunstances, including word semantics and discourse processing in language. Eventually, the recruitment of more primary and secondary perceptual or motor areas would also be necessary in order to visualize or

imagine specific perceptions, actions, or any type of situations outlined in a text.

processing, its language-specificity does not appear evident.

provide an unbeatable account of the topic. Overall, and interestingly, the main results suggest that most of the areas supporting discourse comprehension overlap with the semantic system activated by words (see above), also comprising regions used as well for other more basic linguistic processes. There also exist brain regions specific for discourse processing, a remarkable finding that will be discussed later in detail.

The reviews by Ferstl et al. (2008) and Ferstl (2010) outline a number of results that could –in our view- be grouped into four principal regions. One of the most consistent findings appears to involve the anterior temporal lobes, bilaterally, particularly the temporal poles. As mentioned, this is part of the semantic system for words proposed by Binder et al., (2009), and we have seen it is also involved when sentences have to be interpreted semantically. Nonetheless, a remarkable particularity is that in discourse processing both anterior temporal poles, bilaterally, are involved whereas semantics for words was rather left-lateralized. Another distinctiveness is that the area of the anterior temporal poles involved in discourse processing is larger than the portion used for words, the former spreading ventrally and dorsally covering the whole temporal poles. Accordingly, the anterior temporal lobes, and especially the temporal poles, seem crucial for understanding the meaning of words, sentences, and paragraphs, seemingly constituting a main hub of the semantic system used in language processing. Fertl et al. (2008) propose that a main role for this area in *propositionalization*, the process required for combining words into semantically based content units. Together with the temporal poles, discourses seem also to consistently activate other region that is also crucial for word and sentence semantics: the parietotemporal junction. This is yet a portion of the posterior heteromodal association cortex already clustered with the temporal poles when we reviewed sentence semantics.

A second group of results would comprise the left IFG and STS/STG, spreading to part of the MTG. Ferstl et al. (2008) suggest that at least part of this "fronto-temporal network", substantiate language perception, integration, and interpretation. On the other hand, these activations are not present in a number of studies, being therefore less consistent than other areas contributing to discourse processing. As we have seen earlier, the left IFG seems to exhibit a functional gradient in language processing where syntax appears to be central but phonological and semantic processes are also importantly present; additionally, the role of the STG in syntax analysis is also a consistent finding, but again phonological processes are also observed in this region. Accordingly, even if activated during discourse processing, the role of these perisylvian areas might be not so specific of longer texts. However, this issue may still need further clarification. Recent studies of very slow brain blood flow fluctuations (around 0.1 Hz) have shown that regions in the posterior part of the left superior temporal sulcus/gyrus are consistently correlated at these frequencies with left IFG, particularly within BA 44 or the *pars opercularis* (Lohman et al., 2010). This type of fluctuations might thus reflect processes clearly beyond sentences, in the range of discourse or very large language emissions. The role of these very slow fluctuations for overall language comprehension is still unknown, but the fact that this fronto-temporal network participates, to a larger or a lesser degree, in apparently all the language processes studied so far (even if mainly in phonological and syntactic) emerges as a revealing cue to better understand human language and its possible evolutionary origins.

A third group of findings outlined in Ferstl et al. (2008) and Ferstl (2010) reviews relates to mid-parietal areas, namely the posterior cingulate/precuneus. Acccordingly, apart from participating in syntactic analysis (though very occasionally) and semantics of words and sentences, this region appears of relevance for discourse processing. Indeed, this is a very consistent finding.

provide an unbeatable account of the topic. Overall, and interestingly, the main results suggest that most of the areas supporting discourse comprehension overlap with the semantic system activated by words (see above), also comprising regions used as well for other more basic linguistic processes. There also exist brain regions specific for discourse

The reviews by Ferstl et al. (2008) and Ferstl (2010) outline a number of results that could –in our view- be grouped into four principal regions. One of the most consistent findings appears to involve the anterior temporal lobes, bilaterally, particularly the temporal poles. As mentioned, this is part of the semantic system for words proposed by Binder et al., (2009), and we have seen it is also involved when sentences have to be interpreted semantically. Nonetheless, a remarkable particularity is that in discourse processing both anterior temporal poles, bilaterally, are involved whereas semantics for words was rather left-lateralized. Another distinctiveness is that the area of the anterior temporal poles involved in discourse processing is larger than the portion used for words, the former spreading ventrally and dorsally covering the whole temporal poles. Accordingly, the anterior temporal lobes, and especially the temporal poles, seem crucial for understanding the meaning of words, sentences, and paragraphs, seemingly constituting a main hub of the semantic system used in language processing. Fertl et al. (2008) propose that a main role for this area in *propositionalization*, the process required for combining words into semantically based content units. Together with the temporal poles, discourses seem also to consistently activate other region that is also crucial for word and sentence semantics: the parietotemporal junction. This is yet a portion of the posterior heteromodal association cortex

already clustered with the temporal poles when we reviewed sentence semantics.

human language and its possible evolutionary origins.

consistent finding.

A second group of results would comprise the left IFG and STS/STG, spreading to part of the MTG. Ferstl et al. (2008) suggest that at least part of this "fronto-temporal network", substantiate language perception, integration, and interpretation. On the other hand, these activations are not present in a number of studies, being therefore less consistent than other areas contributing to discourse processing. As we have seen earlier, the left IFG seems to exhibit a functional gradient in language processing where syntax appears to be central but phonological and semantic processes are also importantly present; additionally, the role of the STG in syntax analysis is also a consistent finding, but again phonological processes are also observed in this region. Accordingly, even if activated during discourse processing, the role of these perisylvian areas might be not so specific of longer texts. However, this issue may still need further clarification. Recent studies of very slow brain blood flow fluctuations (around 0.1 Hz) have shown that regions in the posterior part of the left superior temporal sulcus/gyrus are consistently correlated at these frequencies with left IFG, particularly within BA 44 or the *pars opercularis* (Lohman et al., 2010). This type of fluctuations might thus reflect processes clearly beyond sentences, in the range of discourse or very large language emissions. The role of these very slow fluctuations for overall language comprehension is still unknown, but the fact that this fronto-temporal network participates, to a larger or a lesser degree, in apparently all the language processes studied so far (even if mainly in phonological and syntactic) emerges as a revealing cue to better understand

A third group of findings outlined in Ferstl et al. (2008) and Ferstl (2010) reviews relates to mid-parietal areas, namely the posterior cingulate/precuneus. Acccordingly, apart from participating in syntactic analysis (though very occasionally) and semantics of words and sentences, this region appears of relevance for discourse processing. Indeed, this is a very

processing, a remarkable finding that will be discussed later in detail.

The fourth and last region involved in discourse processing comprises the fronto-medial prefrontal cortex (dorso-medial and ventromedial prefrontal cortex), including large portions of the medial side of BAs 8, 9, 10, and 11. To some degree, all of these areas have been mentioned before as mainly implied in the semantic analysis of words. Even though, this grouping here is somewhat different. The main divergence is that the system comprising these areas for discourse processing complements with substantial additions of neural tissue. One addition is the involvement of the entire BA 10, including not only the medial parts, but also the lateral parts -even spreading to BA 46-, which was not the case in word semantics. The other addition conveys the whole anterior cingulate gyrus (in semantics for words, only a very small ventral anterior portion of this gyrus appeared involved). The involvement of these additional portions in discourse processing may convey important consequences. First, BA 10 is the largest cytoarchitectonic area in the human brain, having increased its size substantially during human evolution, as is the case for its connections (Semendeferi et al., 2001). Second, most of these connections seem to affect the anterior cingulate particularly (Allman et al., 2002), which is another milestone in human brain evolution. In fact, the anterior cingulate is so peculiar in the human brain that it is the

main structure containing a special type of neurons, the spindle or Von Economo cells. Only the great apes within the primate order posses this type of neurons presumably related with complex social behaviour, humans exhibiting a disproportionate larger number of them (Allman et al., 2011). Third, BA 10 has recently revealed as the single region showing a significant effect unique to *g*, the psychometric construct of *general intelligence* (Gläscher et al., 2010). Consequently, although this is a group of areas directly involved in language processing, its language-specificity does not appear evident.

The same appears to be the case of the posterior cingulate/precuneus region, the third group of findings involved in discourse processing. The concrete role of the posterior cingulate/precuneus has yet to be elucidated. This region participates in many linguistic processes, but also in a number of other non-linguistic operations. As mentioned earlier, it appears a certainly polyvalent region, involved in emotion, memory, and imagery; it also belongs to the human default system, and it is one of the very few regions connected reciprocally with most other cortical regions. Indeed, this part of the brain is one of the main hubs of the "human core system", the anatomical counterpart of the human default system (Chudek et al., 2008). On the other hand, the anterior medial regions also involved in discourse processing are again largely overlapping with corresponding portions of the human default system. This outstanding overlap between brain systems for word semantics and, particularly, discourse processing and the human default system has been already raised by Binder et al. (2009) and Speer et al. (2009). Ferstl et al. (2008) and Ferstl (2010) focused on the similarities of the discourse-processing system and the system supporting theory of mind. However, considering that the circuits for theory of mind and the human default system have been seen to be largely equivalent (e.g. Buckner et al., 2008), Ferstl and colleagues' suggestion could surely be reworded to imply the default system. Overall, the human default system appears to be such a general and abstract-coded system that it can apply to a considerable number of situations and circunstances, including word semantics and discourse processing in language. Eventually, the recruitment of more primary and secondary perceptual or motor areas would also be necessary in order to visualize or imagine specific perceptions, actions, or any type of situations outlined in a text.

Functional and Structural Magnetic Resonance Imaging of Human Language: A Review 85

The AF seems to have evolved substantially in the human brain from tiny tracts connecting the IFG with the posterior part of BA 22 and the inferior parietal regions, already present in the macaque brain. These connections appear more robust and abundant in the chimpanzee, thereafter reaching the plainest robustness of the human brain (Rilling et al., 2008). Actually, one of the main differences between the human and the chimpanzee brains in this regard is the notable expansion of the posterior ramifications of the AF, which spread not only to involve Wernicke's area and surrounding parietal regions, but also posterior portions of the MTG. The development of these connections, occurring particularly and noticeably within the left hemisphere, seem

Interestingly, two other tracts connecting anterior and posterior regions seem relevant in language processing. One is the superior longitudinal fasciculus (SLF), connecting Broca's area (particularly, BA 44) with the posterior temporal lobe, namely in the STG and the MTG and also involving portions of BA 40. As this tract runs parallel to the AF, several authors (e.g., Rilling et al., 2008) consider both as representing together a functional unit called the *dorsal stream*. The other connection is more primitive; part of it is actually the most developed fronto-temporal connection in the macaque brain and conveys the ventral portion of the extreme capsule and the uncinate fasciculus. Through these connections, the IFG is connected with the anterior and posterior STG (Frey et al., 2008; Rilling et al., 2008), and it is indeed possible that at least part of this *ventral stream* is preferentially used in simpler grammar such as finite-state, relatively accessible to other non-human primates

Finally, there are evidences for an additional number of connections importantly involved in

The moment arrives to summarize and interpret the major milestones that could be elucidated from the preceding exposition. In the following, we will also express a number of

A first and relatively robust conclusion that can be extracted so far is that the human brain contains at least two major "centers of gravity", or main hubs in the networks devoted to language processing. These foci are, on the one hand, the left inferior frontal gyrus (IFG) and, on the other, the left superior temporal gyrus (STG), the latter probably spreading to the superior temporal sulcus (STS) and posterior portions of the middle temporal gyrus (MTG) as well as to some parts of the inferior parietal cortex. Both foci are highly and densely interconnected by means of several tracts, the most outstanding one being the arquate fasciculus (AF). Most of the primary functions and processes involving these foci are seemingly phonological/articulatory and syntactic in nature. These two main hubs are located in perisylvian areas and appear critical for human language. Actually, the main loci of the cerebral lesions yielding core symptoms highly specific of language are the perisylvian areas; the most conspicuous aphasias are usually the consequence of lesions

If the depiction in the preceding paragraph can be taken as relatively robust, the same is not the case when we attempt to subdivide each hub (IFG and STG). An approximate

language, most of them located locally within the IFG and the STG (Friederici, 2009).

reflections on human language using brain function as a main perspective.

affecting either these regions or the AF (e.g., LaPointe, 2011).

to have played a critical role in the evolution of human language.

(Friederici et al., 2006).

**8. Conclusions** 

Fig. 4. Approximate locations of the system to process discourse
