**4. The structure of language**

72 Neuroimaging – Cognitive and Clinical Neuroscience

area-, the left basal ganglia, and even the (right) cerebellum, seem to play a crucial role in identifying the phonemes and sounds used during speech processing (Bozic et al., 2010). Although specific roles for these neural circuits have still to be elucidated, their involvement has been proposed as a mechanism to better process speech sounds regardless of large variability in the input, a way to internally produce those sounds as if the hearer herself were the emitter (Lieberman, 2000). Kotz and Schwartze (2010) stress that these regions, particularly the basal ganglia and the cerebellum, process timing variables crucial for speech. Overal, this is an example of the conjoint action of perceptual and motor brain systems in cognitive processing, as supported by direct evidences as the mirror neurons

If, overall, phonology has been scarcely studied by means of MRI, the case is still worse specifically for prosody, even if this type of auditory information may be as relevant as to determining the syntactic structure of a linguistic message (Snedeker, 2008). There is evidence of the involvement of right fronto-lateral cortical areas (fronto-opercular portions in the right inferior frontal gyrus ) and the right superior temporal regions in main analyses of prosody, as has been found when comparing normal speech and pseudo-speech (i.e., speech with normal prosodic intonations but devoid of known words) with degraded speech (e.g., Meyer et al., 2004). Even though, the role of the counterparts regions in the left hemisphere for the processing of prosodic information cannot be obliterated. A common circuit for language, music, and song perception comprising mid and superior temporal gyri as well as inferior and middle frontal gyri, all bilaterally, has been described (Schön et al., 2010). It is true, nonetheless, that the main implication of either hemisphere appears a function of the phonological vs. melodical nature of the input material (corresponding to left

As mentioned, language can also be visual (as well as gestural), even if this is not originally the "natural" modality for human language. The human brain exhibits a high degree of flexibility and adaptability, yielding high levels of efficacy in tasks to which it is most probably not genetically prepared; reading is an outstanding example in this regard. For a long time, the place in the brain for the "visual word-from area" has been the target of strong debates, even its existence has been put into doubt (Price et al., 2003). The angular

(Rizzolatti & Craighero, 2004).

vs. right side, respectively).

**3. The pictures of language** 

Fig. 1. Approximate locations of the phonological system

Common to any input modality there are processes involved in understanding linguistic messages that appear of the highest interest. Syntactic processes may be among the most outstanding of these factors. As outlined above, syntax permits to determine the hierarchical structure of a sentence composed by a sequence of words (word-forms and their meanings). Studies in this regard have usually approached brain areas involved in syntactic processing using either of two procedures. On the one hand, the comparison between syntactically incorrect and correct material would enhance the activity of brain areas specialized in detecting grammatical errors. As an example, the activation during a sentence like "*the cake was eat*" is compared with its corresponding correct version. On the other hand, comparing grammatically complex sentences with simpler sentences would imply activations in areas particularly handling the complexity of syntactic structures and, hence, areas presumably involved in the hierarchical organization of the sentences. Complexity is usually increased either by embedding material within (e.g.) a main clause, rendering what is called a "recursive" structure, or by changing canonical order (usually, SVO: subject-verb-object) to a non-canonical one, as in the case of passive sentences. Examples of these situations imply comparing "*the child that my mother saw was small*" or "*the cake is being eaten by the children*", respectively, with their corresponding simpler versions (i.e., "*my mother saw a child; the child was small*", and "*the children are eating a cake*"). The case of complexity poses a problem on whether it is actually syntax what is being measured or, instead, working or short-term memory activations necessary to hold information active until the corresponding structural assignments are completed. However, it is also possible to accept that the brain areas specifically involved in working memory for syntactic structures in fact pertain to syntax processing properly, as it can be assumed that working memory for syntax implies the transient activation of circuits actually devoted to syntactic processing (e.g., Fuster, 1999; MacDonald & Christiansen, 2002).

Overall, both types of approaches to the study of human syntax have been comparable, yielding largely similar results. As one of the most consistent findings, the left inferior frontal gyrus (IFG), emerges as a central place involved in syntactic errors detection, grammatical complexity processing, and verbal working memory (e.g., Bornkessel-

Functional and Structural Magnetic Resonance Imaging of Human Language: A Review 75

*pars opercularis*). However, if we approach working memory in the sense mentioned above – i.e., that it consists of the transient activation of circuits devoted to accomplish specific operations- then BA 44 might be better seen as containing core circuits for syntactic processing determining the hierarchical syntactic structure of a sentence. This would harmonize with the variety of different syntactic operations that have been seen to tap on this area, as outlined above. In sum, BA 44 seems a central place for syntax in the brain. The dorsal portion of the *pars opercularis* (overlapping with the most superior part of BA 44 and a portion of BA 9) appears also involved in the processing of syntactic complexity, even when working memory is factored out (Makuuchi et al., 2009). In this regard, however, it has also been claimed that this cortical region is involved in hierarchical ordering of sequences of events regardless of whether they are linguistic or not, as it has been seen to sequence (e.g.) colored shapes or nonlinguistic visual symbols (Bahlmann et al., 2009; Tettamenti et al., 2009). Its language-specificity, therefore, appears challenged. As we will

see below, this is also the case for most, if not all of the areas involved in language.

with fMRI gives rise to core questions on the very nature of human language.

In this regard, the role of BA 45, roughly coinciding with the *pars triangularis* of the IFG, might also appear ambiguous. As has been just-mentioned, it seems involved in analyzing the semantic structure of the sentence. Several studies comparing sentences containing semantic anomalies with their correct counterparts (Kuperberg et al., 2008), or sentences with and without semantic ambiguities (Davis et al., 2007), consistently report activations in BA 45. But this area also appears particularly involved in analyzing embedded structures (Shetreet et al., 2009), which can be considered as a more genuine syntactic process. In line with this, BA 45 has also been seen to support the syntactic constituent structure of the sentence, in a study in which syntactic and semantic structures were disentangled (Pallier et al., 2011). In this latter study, the activation of BA 45 in *pars triangularis* spread also to IFG *pars orbitalis*, therefore including BA 47. However, it is a consistent finding the role of BA 47 in semantic processing (e.g., Binder et al., 2009; see also our section below). In sum, and as an eclectic solution, it might be possible that the most posterior part of BA 45 is relatively

This is in fact the case of BA 44 or the ventral portion of the *pars opercularis* described earlier. Tactile imagery (Yoo et al., 2003), word and face encoding (Leube et al., 2001), object manipulation (Binkofki et al. 1999), smelling familiar odors (Ciumas et al., 2008), or music enjoyment (Koelsch et al., 2006), among several others, are tasks in which BA 44 has been seen importantly involved. Moreover, and within the frame of language, even the role of BA 44 as exclusive for syntax processing does not appear to be proved. In this regard, semantic and articulatory (phonological) processes have been seen to tap also on this area (see our previous section for phonology and the next one for semantics). Possibly, these data might be understood if we assume the proposal of a functional gradient along the whole left IFG, in which -using Brodmann's areas as reference, and from left to right- BA 47 and 45 would appear mainly involved in semantic unification, BA 45 and 44 in syntactic unification, and BA 44 and ventral BA 6 in phonological unification (Hagoort, 2007). Unification is, in the end, the main defining purpose of syntactic operations: unify or "put together", according to the hierarchical structure of the sentence, the different constituents of a sentence. As the posterior part of BA 44 has been seen involved in articulation/phonology and the anterior part in semantics, it might appear that BA 44 is relevant for both phonology and semantics; or, rather, for something in between, maybe what we properly call "syntax". It might also be the case –we are here certainly speculating- that what we call syntax is indeed an abstraction that actually relies on both phonology and semantics. As can be seen, studying language

Schlesewsky et al., 2009; Friederici et al., 2006; Friederici et al., 2009; Koelsch et al., 2009; Meltzer et al., 2010; Newman et al., 2009; Raettig et al., 2010; Rogalsky et al., 2008). Accordingly, the left IFG can be viewed as a main hub in the brain networks supporting syntax.

Nonetheless, IFG is a relatively extensive area, whereas syntactic rules and processes comprise a number of apparently different operations. In this regard, it seems that there are differential demands within specific portions of the left IFG as a function of the task in course. It is difficult, however, to condense the results from the different studies due to systematic inconsistencies in the criteria employed to describe their main results. In terms of Brodmann's cytoarchitectonic areas, IFG occupies, approximately -and starting from a more posterior position next to the precentral sulcus towards a more anterior one, in the left hemisphere- the most inferior portion of BA6, the whole of BA 44, the inferior half of BA 45, and BA 47 (Gray, 1918/2000; Brodmann, 1909/1994). At the same time, the IFG can be anatomically subdivided, following the same spatial sequential order as before, into the *pars opercularis*, the *pars triangularis* and the *pars orbitalis* (Gray, 1998/2000). Whereas both the anatomical and the cytoarchitectonic divisions do not match largely, some studies adopt one system but not the other, and vice versa. Several studies refine their findings by focusing on Broca's area, which certainly pertains to the left IFG. However, this is not solving the problem since there are also historical inconsistencies about what exactly are the boundaries of Broca's area. In this regard, Broca's area corresponds to BA 44 for a number of authors; for several others, BA 44, 45, and 47 should be included; for a number of other authors, the areas involved are just BA 44 and 45 (e.g., Uylings et al., 1999). Finally, in an attempt to refine anatomical exactitude when describing main results, several studies use Talairach or MNI 3D coordinates (Price, 2010). This highly precise system nonetheless obliterates the fact that fMRI is not as precise as to use these millimetrical coordinates, particularly considering the number of processing stages needed for normalization and statistical processing of the data. In addition, results in 3D coordinates usually refer to the centroid of an activated region regardless of its total size or whether its limits overlap with or surpass the anatomical or cytoarchitectonical subdivisions. In the following, we will try to minimize as far as possible these current limitations when describing the main results reported in the literature, carefully inspecting and contrasting the data reported by the different authors.

According to some reports, the most ventral part of the *pars opercularis*, roughly –but not solely- coinciding with BA 44, appears involved in verbal (syntactical) working memory (Friederici et al., 2006; Price, 2010; Rogalsky et al., 2008). In line with this might also be interpreted different results for this area as those by Bornkessel-Schlesewsky et al. (2009) for the processing of word-order variations in sentences, or Christensen (2010) and Rodd et al. (2010) for garden-path and ambiguous sentences -in which the structure must be reanalyzed and reconstructed, or several candidate structures must be kept active during sentence processing-. This ventral part of the *pars opercularis* has further been subdivided into two depending on whether the portions belong to BA 44 or to BA 6; the former would be involved in phrase structure grammar, the latter in finite state grammar (Friedrici et al., 2006). The first type of grammar refers to the use of embedded sentences, therefore demanding more working memory than the latter, simpler (linear) structures with no nesting. Detecting grammatical errors also tap on BA 44 (e.g., Heim et al., 2010), a result consistent with a role of this area in syntactic working memory to the extent that the detection of errors also increases processing demands. Overall, all these data are in line with a syntactic working memory interpretation as a main role of BA 44 (or the anterior ventral

Schlesewsky et al., 2009; Friederici et al., 2006; Friederici et al., 2009; Koelsch et al., 2009; Meltzer et al., 2010; Newman et al., 2009; Raettig et al., 2010; Rogalsky et al., 2008). Accordingly, the left IFG can be viewed as a main hub in the brain networks supporting

Nonetheless, IFG is a relatively extensive area, whereas syntactic rules and processes comprise a number of apparently different operations. In this regard, it seems that there are differential demands within specific portions of the left IFG as a function of the task in course. It is difficult, however, to condense the results from the different studies due to systematic inconsistencies in the criteria employed to describe their main results. In terms of Brodmann's cytoarchitectonic areas, IFG occupies, approximately -and starting from a more posterior position next to the precentral sulcus towards a more anterior one, in the left hemisphere- the most inferior portion of BA6, the whole of BA 44, the inferior half of BA 45, and BA 47 (Gray, 1918/2000; Brodmann, 1909/1994). At the same time, the IFG can be anatomically subdivided, following the same spatial sequential order as before, into the *pars opercularis*, the *pars triangularis* and the *pars orbitalis* (Gray, 1998/2000). Whereas both the anatomical and the cytoarchitectonic divisions do not match largely, some studies adopt one system but not the other, and vice versa. Several studies refine their findings by focusing on Broca's area, which certainly pertains to the left IFG. However, this is not solving the problem since there are also historical inconsistencies about what exactly are the boundaries of Broca's area. In this regard, Broca's area corresponds to BA 44 for a number of authors; for several others, BA 44, 45, and 47 should be included; for a number of other authors, the areas involved are just BA 44 and 45 (e.g., Uylings et al., 1999). Finally, in an attempt to refine anatomical exactitude when describing main results, several studies use Talairach or MNI 3D coordinates (Price, 2010). This highly precise system nonetheless obliterates the fact that fMRI is not as precise as to use these millimetrical coordinates, particularly considering the number of processing stages needed for normalization and statistical processing of the data. In addition, results in 3D coordinates usually refer to the centroid of an activated region regardless of its total size or whether its limits overlap with or surpass the anatomical or cytoarchitectonical subdivisions. In the following, we will try to minimize as far as possible these current limitations when describing the main results reported in the literature, carefully inspecting and contrasting the data reported by the different authors. According to some reports, the most ventral part of the *pars opercularis*, roughly –but not solely- coinciding with BA 44, appears involved in verbal (syntactical) working memory (Friederici et al., 2006; Price, 2010; Rogalsky et al., 2008). In line with this might also be interpreted different results for this area as those by Bornkessel-Schlesewsky et al. (2009) for the processing of word-order variations in sentences, or Christensen (2010) and Rodd et al. (2010) for garden-path and ambiguous sentences -in which the structure must be reanalyzed and reconstructed, or several candidate structures must be kept active during sentence processing-. This ventral part of the *pars opercularis* has further been subdivided into two depending on whether the portions belong to BA 44 or to BA 6; the former would be involved in phrase structure grammar, the latter in finite state grammar (Friedrici et al., 2006). The first type of grammar refers to the use of embedded sentences, therefore demanding more working memory than the latter, simpler (linear) structures with no nesting. Detecting grammatical errors also tap on BA 44 (e.g., Heim et al., 2010), a result consistent with a role of this area in syntactic working memory to the extent that the detection of errors also increases processing demands. Overall, all these data are in line with a syntactic working memory interpretation as a main role of BA 44 (or the anterior ventral

syntax.

*pars opercularis*). However, if we approach working memory in the sense mentioned above – i.e., that it consists of the transient activation of circuits devoted to accomplish specific operations- then BA 44 might be better seen as containing core circuits for syntactic processing determining the hierarchical syntactic structure of a sentence. This would harmonize with the variety of different syntactic operations that have been seen to tap on this area, as outlined above. In sum, BA 44 seems a central place for syntax in the brain.

The dorsal portion of the *pars opercularis* (overlapping with the most superior part of BA 44 and a portion of BA 9) appears also involved in the processing of syntactic complexity, even when working memory is factored out (Makuuchi et al., 2009). In this regard, however, it has also been claimed that this cortical region is involved in hierarchical ordering of sequences of events regardless of whether they are linguistic or not, as it has been seen to sequence (e.g.) colored shapes or nonlinguistic visual symbols (Bahlmann et al., 2009; Tettamenti et al., 2009). Its language-specificity, therefore, appears challenged. As we will see below, this is also the case for most, if not all of the areas involved in language.

This is in fact the case of BA 44 or the ventral portion of the *pars opercularis* described earlier. Tactile imagery (Yoo et al., 2003), word and face encoding (Leube et al., 2001), object manipulation (Binkofki et al. 1999), smelling familiar odors (Ciumas et al., 2008), or music enjoyment (Koelsch et al., 2006), among several others, are tasks in which BA 44 has been seen importantly involved. Moreover, and within the frame of language, even the role of BA 44 as exclusive for syntax processing does not appear to be proved. In this regard, semantic and articulatory (phonological) processes have been seen to tap also on this area (see our previous section for phonology and the next one for semantics). Possibly, these data might be understood if we assume the proposal of a functional gradient along the whole left IFG, in which -using Brodmann's areas as reference, and from left to right- BA 47 and 45 would appear mainly involved in semantic unification, BA 45 and 44 in syntactic unification, and BA 44 and ventral BA 6 in phonological unification (Hagoort, 2007). Unification is, in the end, the main defining purpose of syntactic operations: unify or "put together", according to the hierarchical structure of the sentence, the different constituents of a sentence. As the posterior part of BA 44 has been seen involved in articulation/phonology and the anterior part in semantics, it might appear that BA 44 is relevant for both phonology and semantics; or, rather, for something in between, maybe what we properly call "syntax". It might also be the case –we are here certainly speculating- that what we call syntax is indeed an abstraction that actually relies on both phonology and semantics. As can be seen, studying language with fMRI gives rise to core questions on the very nature of human language.

In this regard, the role of BA 45, roughly coinciding with the *pars triangularis* of the IFG, might also appear ambiguous. As has been just-mentioned, it seems involved in analyzing the semantic structure of the sentence. Several studies comparing sentences containing semantic anomalies with their correct counterparts (Kuperberg et al., 2008), or sentences with and without semantic ambiguities (Davis et al., 2007), consistently report activations in BA 45. But this area also appears particularly involved in analyzing embedded structures (Shetreet et al., 2009), which can be considered as a more genuine syntactic process. In line with this, BA 45 has also been seen to support the syntactic constituent structure of the sentence, in a study in which syntactic and semantic structures were disentangled (Pallier et al., 2011). In this latter study, the activation of BA 45 in *pars triangularis* spread also to IFG *pars orbitalis*, therefore including BA 47. However, it is a consistent finding the role of BA 47 in semantic processing (e.g., Binder et al., 2009; see also our section below). In sum, and as an eclectic solution, it might be possible that the most posterior part of BA 45 is relatively

Functional and Structural Magnetic Resonance Imaging of Human Language: A Review 77

parietal lobe), small portions of BA 37 (next to BA 22), superior parietal cortex (BA 7), as well as the lentiform and the caudate nuclei (e.g., Chistensen, 2010; Friederici et al., 2006;

Linguistic messages normally tell something about the world and its components (objects, persons, places, and so forth). As such, a linguistic message includes words (word-forms with their individual semantic contents or meanings) that are combined, usually through syntax, rendering a concrete description of their relationships intended to mirror a real situation or an idea. Within the brain, indeed, extensive regions of the cerebral cortex appear devoted to semantic information processing. This seemingly provides a clue on the relationships between language and other cognitive processes. But it also poses some doubts on the boundaries between what can be labeled as "linguistic" and "non-linguistic". It is also the case that "semantic" might appear as a rather vague and imprecise term, covering a large number of otherwise different processes or operations. In fact, terms as "pragmatic", "conceptual", as well as several others, often appear next to "semantic" as equivalent or

Actually, the meaning of words, one of the main features that the term semantic can refer to, can be just about anything in the world. In other words, human languages have words –and, then, meanings- for absolutely all (or almost all) things known so far in the world, be they real, imagined, or with a large amount of ambiguity and abstraction (e.g., Pinker, 2007). In this regard, some authors even think that the so-called "syntactic words" (i.e., complements, determiners, suffixes, and so many words or particles with a specific syntactic function) have also a meaning to be considered as plainly pertaining to the same semantic system of the brain as any other type of content word, such as nouns or verbs. In this line, it could be the case that syntactic regions reviewed above might be part of the "semantic" system, but only the part preferentially dealing with abstract structural hierarchical relationships between a number of items, be they words or whatever. Indeed, it is a plausible scenario that syntax words emerged initially as any other, less functional words during the evolution

This said, it should not be surprising that semantic areas have been proposed to occupy most of the cerebral cortex. Providentially, Binder et al. (2009) have recently published an extensive review of functional neuroimaging studies of semantic processing, in which not only strict inclusion criteria were applied but also advanced statistical analyses for determining the probability of a given region as belonging to the semantic system. The studies included in that review used words as stimuli, so that we can be certain that the areas suggested as supporting the semantic system are indeed areas activated by language. This note is important because, as we will see, the areas constituting the semantic system are actually and for the most part classically considered as heteromodal association areas of the neocortex, located both in frontal and posterior regions. They are therefore common to a large amount of non-linguistic processes involved in either perception or action. Additionally, areas of the limbic system involved in emotional processing are also part of

According to the review by Binder et al. (2009), the semantic system in the brain can be subdivided into three main widespread locations. A first one includes large portions of the posterior multimodal and heteromodal cortex, namely the angular gyrus (AG, in BA 39), the

Shetreet el al., 2009).

**5. The meaning of language (I)** 

corresponding to a somehow unitary system.

of human language (Heine & Kuteva, 2007).

the semantic system.

more syntactic in nature, conforming a somehow unitary system together with part of BA 44; the anterior part, in turn, would be more semantic, working together with BA 47. Overall, the above-mentioned functional gradient within the left IFG might actually be more gradual than the labels currently available to describe it (i.e., semantic, syntactic, and articulatory/phonological), which might also explain why the ventral portion of BA 6, most consistently described as an articulatory/phonological area (it actually belongs to the premotor cortex) has also been seen occasionally involved in detecting syntactic errors or analyzing syntactically ambiguous sentences (Christensen, 2010; Friederici et al., 2006). The picture can be yet more complicated when considering that even language processing at the discourse level consistently recruits large portions of the IFG, as we will see below.

Additionally, a number of studies also support the involvement of other regions apart from the IFG in syntactic processing. One of the most consistent findings in this regard is the existence of a fronto-temporal network supporting syntactic processing. Whereas the frontal pole of this network implies the left IFG, especially in and around BA 44, the temporal portion is mainly comprising the left superior temporal gyrus (STG) and superior temporal sulcus (STS), most likely excluding primary auditory areas (i.e., BA 41 and 42), and roughly corresponding to BA 22 (Christensen, 2010; Rodd et al., 2010). Interestingly, these activations may plainly include Wernicke's area (Shetreet et al., 2009), which is mainly involved in the processing of language sounds (see our previous section in this regard). In several studies, large (anterior and posterior) portions of BA 22 appear relevant in syntactic processes (Friederici, 2002; Rodd et al., 2010). In other occasions, however, it is only a small portion of BA 22 what is involved, such as the posterior portion of the STS (Pallier et al., 2011). Upper portions of the middle temporal gyrus (MTG), comprising part of BA 21, have been also reported to participate in syntactic processing (Christensen, 2010; Friederici et al., 2006; Shetreet et al., 2009). Interestingly, although the main findings are located within the left hemisphere, occasional activations in corresponding areas of the right hemisphere are also reported. Several studies also report activations in the ventral portion of the supramarginal gyrus (SMG, part of BA 40), together with the *planum temporale*, a posterior portion of BA 22 (Raettig et al., 2010), although these regions appear more consistently as rather semantic, particularly the SMG (Binder et al, 2009).

Fig. 2. Approximate locations of the syntactic system

Finally, still other brain regions have been seen also involved in syntactic processing, though less consistently. Among these, we can mention the precuneus (in the medial parietal lobe), small portions of BA 37 (next to BA 22), superior parietal cortex (BA 7), as well as the lentiform and the caudate nuclei (e.g., Chistensen, 2010; Friederici et al., 2006; Shetreet el al., 2009).
