**4.1 Understanding activations: defining the relationship between the generation of meaning(s) and activations**

Imaging techniques that involve the hemodynamic response (PET, fMRI) only measure correlates of neuronal activity and processes and not actual neuronal firings. Electrophysiological techniques (including EEG, MEG) can measure electrical activity in ensembles of neurons and changes in magnetic fields related to neuronal activity.

Invasive direct single-neuron mappings, as found in cortical stimulation mappings (CSM), are only conducted during surgery. George Ojemann, who developed the technique, conducted over 1100 surgeries that included CSM mappings to identify areas important to production and comprehension of language(s). This technique was developed in order to identify areas related to important functions of production (motor) and comprehension (sensory) in language(s) and has proven successful in preserving these functions in surgeries that require removal of tissue. Dr. Ojemann has published over 170 papers of CSM data with important conclusions about neural mappings of language and languages [29–41].

Understanding activations found in BOLD fMRI studies presents a significant challenge.6 When considering the results of fMRI studies of language(s), one finds a much broader range of activations across both hemispheres in areas not represented in the traditional Broca/Wernicke targets from the "classical model"; these results are in keeping with research findings in fMRI language studies and represent a consensus among the neuroscience community (cf. [4, 5, 45–51]). There are challenges in analyzing neural activations, and Raichle reminds us that it is:

*"…impossible to distinguish inhibitory from excitatory cellular activity on the basis of changes in either blood flow or metabolism. Thus, on this view a local increase in inhibitory activity would be as likely to increase blood flow and the fMRI BOLD signal as would a local increase in excitatory activity" (*[48]*, p. 12).*

Other challenges in using fMRI for cognitive studies include the timing delay in the hemodynamic response that lags behind neuronal activity. It is essential to remember that there can be no one-to-one relationship between neural activations acquired during imaging and the subject's knowledge or ability.

Interpreting the activations recorded from ROIs in fMRI language studies is strengthened when significance can be found with behavioral data (e.g., empirical proficiency testing data) using multivariate analysis of covariance (MANCOVA). One such example is a longitudinal study of second language acquisition that also includes Common European Framework proficiency testing data [5, 45]. The use of empirical **language proficiency** data is particularly important for bi- and

<sup>6</sup> Eklund et al. ([42], pp. 7900-7905) is one of a series of papers that discusses problems with fMRI analysis and parametric statistical methods, including "false positives" that resulted in a series of studies using the following fMRI analysis software: AFNI, FSL, and SPM. Eklund et al. note that results are more reliable for voxel-wise inference and invalid for cluster-based inference ([42], p. 7903). Additional sources that examine inter-method discrepancies in brain imaging include Katuwal et al. [43] and Bowring et al. [44].

multilingual fMRI studies, where it is not atypical that a subject may demonstrate higher levels of activation in a language that they do not know well (or at all) than in a language that is the L1 or highly proficient L2. Abutalebi et al. [52] also includes language proficiency and show that it contributes explanatory power in understanding language switching and cognitive control in bi-/multilinguals. Birdsong notes that brain imaging studies demonstrate that second language (L2) proficiency, not age of acquisition, is "the strongest predictor of degree of similarity between late learners & monolingual natives" ([53, 54] pp. 24-5).

The importance of developing ecologically valid protocols for imaging experiments is another factor that can strengthen validity in interpreting activations. Other considerations in protocol development include protocol design that lends itself to the subtractive method that is often applied in fMRI experiments and minimizing "confounding factors" ([55, 56], p. 290). Activation levels resulting from task-based fMRI, by themselves, may not be interpretable (activation levels do not necessarily correlate with knowledge). Thus, the importance of including other statistical models and empirical measurements (e.g., proficiency) becomes critical for strengthening conclusions.

Raichle [57–60] proposes the *default mode network* (DMN) as one approach to understanding the changes in activations where specific brain regions decrease their activity during a task condition. Gusnard and Raichle ([61], p. 689) also suggest a way to characterize "tonically active areas" by distinguishing between "functionally active" and "activated." For Raichle, the DMN is one of the most important of the hierarchical networks, and it plays a central role in coordinating among brain systems and their interactions across system boundaries.7 The DMN and other such networks point toward the heightened interest in the application and inclusion of *resting state fMRI* in protocols and connectivity models [62, 63]*.* Functional connectivity modeling and analysis in recent neuroimaging studies are an important move away from older approaches that focus on modularity and localization.

## **4.2 Embodied cognition and languages**

Sensory-motor interactive modeling of language and brain systems has become an important part of cognitive neuroscientific discourse over the past 15 years. The debates concerning *embodied cognition* are central to acquiring and developing new methods in order to understand the neurological interface of human languages, the relevance of multimodality and functional connectivity, and a rejection of modularity for modeling the processing of language(s) in the brain. Gallese and Lakoff [15] emphasize the importance of multimodal modeling of language and brain, because this perspective not only (1) takes into account its evolutionary trajectory but (2) characterizes these linguistic structures as part of the sensory-motor system at the neuronal level. Multimodal modeling, if acknowledged, requires a rejection of the modular view of language mappings in the brain.

The central arguments presented by Gallese and Lakoff stress the importance of multimodality, including mirror, premotor and parietal neurons, and the realization of multimodalities through functional clusters ([15, 64], p. 458). Approaches based on principles of embodied cognition will play an important role in developing robust theories of brain and language(s).

As I have noted earlier, "the kind of sensory-motor alignment that Gallese and Lakoff present is but one type of the significant multimodal aspects of human language and the brain, and the cautionary statements given in Mahon and Caramazza

**19**

*Semiotic Principles in Cognitive Neuroscience DOI: http://dx.doi.org/10.5772/intechopen.89791*

motor synthesis in three ways:

pp. 128-137)

one" ([45], p. 198):

*p. 111)*

*language" (ibid.)*

**4.3 Empiricism and interpretation of results**

[16] are important to keep in mind" (2013, p. 136). Mahon and Caramazza, while rejecting a disembodied cognition hypothesis, argue that it is necessary to understand "whether the motor system is activated due to 'leakage' of (or cascading) activation from an 'abstract' conceptual level', or occurs in parallel to (or independently of) activation of the 'abstract' conceptual level?" ([16], p. 60). While they suggest that some concepts might include sensory-motor information, they are not

*"For abstract concepts there is no sensory or motor information that could corre-*

While Gallese and Lakoff provide one important form of the alignment (e.g., English lexeme *grasp* ([15], p. 457)), one can identify a more pervasive sensory-

1.The realizations of specific embodied forms of grammatical and lexical meaning as produced/articulated and perceived (cf. Bolinger's/sl/and/gr/lexemes in English—*slippery*, *sleezy*, *slimy*, *slinky*, *etc.; grasp*, *grip*, *grab*, *grub*, *greed*, etc.)

2.The specific gestures that accompany language-specific lexical categories and sound-based alternative systems of auditory perception (lyric and music)

3.Visual meanings given in written language (e.g., *to/two/too*, *sea/see/C*] or ideograms) that are not given in the sound forms (see also Andrews, 2013,

As noted in Andrews [45], all forms of linguistic meanings are negotiated *in context* via speech acts, and these speech acts "are always multiples and are embedded in… *speech communities and communities of practice*,*"* which are, in the end, what one could call "the inalienable context" ([45], pp. 196, 198). And while sensory-motor systems are internally determined, there is never "language in the

*"Language is a consequence of humans interacting in cultural space…We are always multifaceted users of language; we play the roles of speakers, hearers and observers (sometimes simultaneously), and we as users are defined by the multiple and variegated…speech communities and communities of practice in which we* 

*"What is important in a model is not its accord with experiment, but, on the contrary, its 'ontological range,' in which it states the manner in which the phenomena take place and in which it describes their underlying mechanisms." Thom ([65],* 

Strong empirical methods that yield repeatable results are an important component of achieving reliable conclusions in the analysis of neuroimaging data. Standard software programs typically used in fMRI analysis (e.g., FSL, SPM) are an important component of the statistical methods used in analysis, and these systems continue to be expanded and improved upon. Collecting behavioral data that can be used to correlate with imaging data is another important way to strengthen confidence in the results of the analysis. Below are four major points that describe

willing to concede the case with abstract concepts ([16], p. 60):

*spond in any reliable or direct way to their 'meanings'."*

<sup>7</sup> For a discussion of the importance of slow cortical potentials (SCP), see Raichle ([59], pp, 182-185).

*Semiotic Principles in Cognitive Neuroscience DOI: http://dx.doi.org/10.5772/intechopen.89791*

*Cognitive and Intermedial Semiotics*

for strengthening conclusions.

multilingual fMRI studies, where it is not atypical that a subject may demonstrate higher levels of activation in a language that they do not know well (or at all) than in a language that is the L1 or highly proficient L2. Abutalebi et al. [52] also includes language proficiency and show that it contributes explanatory power in understanding language switching and cognitive control in bi-/multilinguals. Birdsong notes that brain imaging studies demonstrate that second language (L2) proficiency, not age of acquisition, is "the strongest predictor of degree of similarity

The importance of developing ecologically valid protocols for imaging experiments is another factor that can strengthen validity in interpreting activations. Other considerations in protocol development include protocol design that lends itself to the subtractive method that is often applied in fMRI experiments and minimizing "confounding factors" ([55, 56], p. 290). Activation levels resulting from task-based fMRI, by themselves, may not be interpretable (activation levels do not necessarily correlate with knowledge). Thus, the importance of including other statistical models and empirical measurements (e.g., proficiency) becomes critical

Raichle [57–60] proposes the *default mode network* (DMN) as one approach to understanding the changes in activations where specific brain regions decrease their activity during a task condition. Gusnard and Raichle ([61], p. 689) also suggest a way to characterize "tonically active areas" by distinguishing between "functionally active" and "activated." For Raichle, the DMN is one of the most important of the hierarchical networks, and it plays a central role in coordinating among brain

networks point toward the heightened interest in the application and inclusion of *resting state fMRI* in protocols and connectivity models [62, 63]*.* Functional connectivity modeling and analysis in recent neuroimaging studies are an important move

Sensory-motor interactive modeling of language and brain systems has become an important part of cognitive neuroscientific discourse over the past 15 years. The debates concerning *embodied cognition* are central to acquiring and developing new methods in order to understand the neurological interface of human languages, the relevance of multimodality and functional connectivity, and a rejection of modularity for modeling the processing of language(s) in the brain. Gallese and Lakoff [15] emphasize the importance of multimodal modeling of language and brain, because this perspective not only (1) takes into account its evolutionary trajectory but (2) characterizes these linguistic structures as part of the sensory-motor system at the neuronal level. Multimodal modeling, if acknowledged, requires a rejection

The central arguments presented by Gallese and Lakoff stress the importance of multimodality, including mirror, premotor and parietal neurons, and the realization of multimodalities through functional clusters ([15, 64], p. 458). Approaches based on principles of embodied cognition will play an important role in developing

As I have noted earlier, "the kind of sensory-motor alignment that Gallese and Lakoff present is but one type of the significant multimodal aspects of human language and the brain, and the cautionary statements given in Mahon and Caramazza

<sup>7</sup> For a discussion of the importance of slow cortical potentials (SCP), see Raichle ([59], pp, 182-185).

away from older approaches that focus on modularity and localization.

The DMN and other such

between late learners & monolingual natives" ([53, 54] pp. 24-5).

systems and their interactions across system boundaries.7

of the modular view of language mappings in the brain.

**4.2 Embodied cognition and languages**

robust theories of brain and language(s).

**18**

[16] are important to keep in mind" (2013, p. 136). Mahon and Caramazza, while rejecting a disembodied cognition hypothesis, argue that it is necessary to understand "whether the motor system is activated due to 'leakage' of (or cascading) activation from an 'abstract' conceptual level', or occurs in parallel to (or independently of) activation of the 'abstract' conceptual level?" ([16], p. 60). While they suggest that some concepts might include sensory-motor information, they are not willing to concede the case with abstract concepts ([16], p. 60):

*"For abstract concepts there is no sensory or motor information that could correspond in any reliable or direct way to their 'meanings'."*

While Gallese and Lakoff provide one important form of the alignment (e.g., English lexeme *grasp* ([15], p. 457)), one can identify a more pervasive sensorymotor synthesis in three ways:


As noted in Andrews [45], all forms of linguistic meanings are negotiated *in context* via speech acts, and these speech acts "are always multiples and are embedded in… *speech communities and communities of practice*,*"* which are, in the end, what one could call "the inalienable context" ([45], pp. 196, 198). And while sensory-motor systems are internally determined, there is never "language in the one" ([45], p. 198):

*"Language is a consequence of humans interacting in cultural space…We are always multifaceted users of language; we play the roles of speakers, hearers and observers (sometimes simultaneously), and we as users are defined by the multiple and variegated…speech communities and communities of practice in which we language" (ibid.)*

## **4.3 Empiricism and interpretation of results**

*"What is important in a model is not its accord with experiment, but, on the contrary, its 'ontological range,' in which it states the manner in which the phenomena take place and in which it describes their underlying mechanisms." Thom ([65], p. 111)*

Strong empirical methods that yield repeatable results are an important component of achieving reliable conclusions in the analysis of neuroimaging data. Standard software programs typically used in fMRI analysis (e.g., FSL, SPM) are an important component of the statistical methods used in analysis, and these systems continue to be expanded and improved upon. Collecting behavioral data that can be used to correlate with imaging data is another important way to strengthen confidence in the results of the analysis. Below are four major points that describe

the advantages of empirical data and analysis that includes behavioral "can do" data in opposition to more traditional experiments that put more emphasis on static, essentialist characteristics of the experimental subjects:

