**2. What is semiotics?**

It is often the case that in research that identifies itself as "semiotic," there is not always a clear explanation of what it is and what it means. In the present work, we will synthesize the epistemological approach to semiotics and semiosis found in Peirce, Sebeok, Lotman, Jakobson, and Eco, where semiotics as an epistemology is articulated originally by Locke as a "doctrine" and evolves into what Sebeok calls "a peculiarly human form of inquiry that is in its essence a theory of perception" ([6], p. 12). For all of these approaches, *signification* is primary and the basis for *communication* and *text generation*. 1

Umberto Eco points out that semiotics in the United States and France was deeply associated with *structuralism* in the context of research on cultural and linguistic phenomena in the 1960s ([9], pp. ix-x). It is also at this time that Lotman, working in Estonia, publishes a fundamental work on semiotic methods in the study of literature, in which he calls for the elimination of the opposition between exact science and humanities, it presents semiotic systems as **modeling systems** that both construct and explain the world in which humans live, all cultural spaces are based on multiple linguo-cultural codes, and it considers semiotics to be a *cognitive science*. 2 These modeling systems are not ahistorical and are embedded in complex systems of "non-hereditary collective memory" [11].

Peircean semiotics is a clear departure from the structuralist tradition and has strong ties to the philosophy of science and provides a framework for categorizing triadic sign types in a fundamentally non-binary modeling system. Peirce's most important contribution to semiotics is found in his theory of signs (minimally triadic, never binary) and types of inference (Peirce 2.246, 2.250-2, [12, 13]). For Peirce, the triadic sign always involves (1) its relation to itself (*sign-sign* relationship), (2) relation

**15**

pp. 66-71; [5], pp. 51-58; [18–20]).

*Semiotic Principles in Cognitive Neuroscience DOI: http://dx.doi.org/10.5772/intechopen.89791*

(*sign-final interpretant*) (ibid.).

changes (cf. [14]).

used [6, 10].

human cognition.

to its *object* (*sign-dynamic object* relationship), and (3) its relation to its *interpretant*

**3. The semiotic approach and semiosis: no more binaries**

*is never solely determined or determinable by reference alone*.

**3.1 Defining icon/index/symbol: redefining non-arbitrariness**

It was Jakobson who introduced Peirce's *icon/index/symbol* triad (part of the sign-object relationship) into modern linguistic theory. Through this triad, Peircean semiotic theory became widely known in linguistics and was applied in works that demonstrated the significance of the non-arbitrariness of the linguistic sign [1]. Non-arbitrariness must be understood as a relative phenomenon, not an absolute, and is particularly powerful at the phonological, morphological, and morphophonemic levels in languages. Much later, sociolinguistics and identity studies borrowed the terminology as well but with

The semiotic perspective requires a re-evaluation of research techniques and applications. Such a re-evaluation brings into question the very object of study, as well as the instrumentation used to evaluate the object. Once it is recognized that the focal point of Peircean semiotic theory is not the sign, but *semiosis (the sign in action*, *the prerequisite process of the exchange of information [signification])*, the next step is to take a closer look at the interface between functioning signs as they are

Semiotic theory that focuses on *signs in action* provides an epistemological framework where any linguistic speech act includes the *users* of the language, and *codes and messages are embodied in the users* themselves. This idea can be extended to neuroscience and can provide the basis for proposing that the sensory-motor neural image of a given linguistic phenomenon, be it grammatical or lexical, is multisensory and involves cross-modal effects—a notion that is central to all of the current neuroscience research on *embodied cognition* (including [15, 16])*.* Another outcome of a theory of dynamic signs requires treating *referential* meaning as a *type* of meaning—not the *only kind* of meaning and not even the most important kind in all instances. In other words, the outcome of this framework is that *linguistic meaning* 

a profound impact on the modeling of *multimodality* as an organizing principle of

Following Peirce (Peirce 8.368, [13] pp. 4–23;[21]), the *icon* is defined as a qualitative likeness between a sign and its object. The quality upon which the similarity is based belongs to the sign, whether or not its object exists. The *index* is a real connection between the sign and its object, and the relationship is given via the dynamic object of the sign. The *symbol* is a general rule between the sign and object, and the object is related to the sign via the *interpretant.* [Jakobson's rendering of these terms is reanalyzed back into the dyadic terms *signifier/signans* and *signified/signatum, similarity/*contiguity (i.e., paradigmatic/syntagmatic axes), and the notions of *factual* (effectual) and *habitual* [22, 23]] Each of these 3 sign-object relationships are embedded in each other such that none of these categories may

<sup>3</sup> For a full discussion of the Jakobsonian speech act model, which is applicable to explicating the generation of linguistic meanings, as well as language usage both in healthy subjects and in pathology, see ([17],

3

This outcome has

<sup>1</sup> Signification is the "initial and primary ability that underlies human language and all of human cognition" and is essential to the creation of non-hereditary collective memory/cultural systems. For an important discussion of the importance of signification and invention of collective symbols, see Donald [7]. Tomasello's characterization of linguistic reference as a "social action" is an important corollary to the phenomenon of signification ([8], p. 97). At the point children begin participating in the signification process as learners of linguistic symbols, they not only can tap into the richness of "preexisting" knowledge but also participate in the uniqueness of linguistic symbols and their inherent polysemic nature. This provides the opportunity to cognitively embrace an event or object at multiple levels simultaneously (cf. "a rose, a flower, and a gift") ([8], p. 107).

<sup>2</sup> As early as the 1970s, one sees a tension between the place of semiotics and the original *cognitive sciences* (founded on four primary disciplines—computer science, philosophy, psychology, and linguistics). Sebeok at that time, under the influence of Hofstadter's work in artificial intelligence, prefers the term "perception" over "cognition" in order to separate semiotic theory from the cognitive sciences and to emphasize the central role that the generation of meaning plays in understanding sign systems in dynamic interaction ([10], p. 53). By the 1980s semiotics became known in the context of not only human anthropological and linguistic systems but also biological phenomena.

*Semiotic Principles in Cognitive Neuroscience DOI: http://dx.doi.org/10.5772/intechopen.89791*

*Cognitive and Intermedial Semiotics*

**2. What is semiotics?**

*nication* and *text generation*.

*science*. 2

neuroscience is discussed in the conclusion.

1

systems of "non-hereditary collective memory" [11].

simultaneously (cf. "a rose, a flower, and a gift") ([8], p. 107).

human anthropological and linguistic systems but also biological phenomena.

with human subjects (healthy and lesion-deficit, behavioral and neuroimaging, laboratory and in situ) or textual studies. The questions discussed here focus on identifying the key areas where semiotic theory has impacted the cognitive neurosciences, which include approaches to understanding sensory-motor mappings in the human brain, *multimodality* as opposed to modularity, and *embodied cognition*. The contribution of semiotic principles to reliable scientific research in cognitive

It is often the case that in research that identifies itself as "semiotic," there is not always a clear explanation of what it is and what it means. In the present work, we will synthesize the epistemological approach to semiotics and semiosis found in Peirce, Sebeok, Lotman, Jakobson, and Eco, where semiotics as an epistemology is articulated originally by Locke as a "doctrine" and evolves into what Sebeok calls "a peculiarly human form of inquiry that is in its essence a theory of perception" ([6], p. 12). For all of these approaches, *signification* is primary and the basis for *commu-*

Umberto Eco points out that semiotics in the United States and France was deeply associated with *structuralism* in the context of research on cultural and linguistic phenomena in the 1960s ([9], pp. ix-x). It is also at this time that Lotman, working in Estonia, publishes a fundamental work on semiotic methods in the study of literature, in which he calls for the elimination of the opposition between exact science and humanities, it presents semiotic systems as **modeling systems** that both construct and explain the world in which humans live, all cultural spaces are based on multiple linguo-cultural codes, and it considers semiotics to be a *cognitive* 

These modeling systems are not ahistorical and are embedded in complex

Peircean semiotics is a clear departure from the structuralist tradition and has strong ties to the philosophy of science and provides a framework for categorizing triadic sign types in a fundamentally non-binary modeling system. Peirce's most important contribution to semiotics is found in his theory of signs (minimally triadic, never binary) and types of inference (Peirce 2.246, 2.250-2, [12, 13]). For Peirce, the triadic sign always involves (1) its relation to itself (*sign-sign* relationship), (2) relation

<sup>1</sup> Signification is the "initial and primary ability that underlies human language and all of human cognition" and is essential to the creation of non-hereditary collective memory/cultural systems. For an important discussion of the importance of signification and invention of collective symbols, see Donald [7]. Tomasello's characterization of linguistic reference as a "social action" is an important corollary to the phenomenon of signification ([8], p. 97). At the point children begin participating in the signification process as learners of linguistic symbols, they not only can tap into the richness of "preexisting" knowledge but also participate in the uniqueness of linguistic symbols and their inherent polysemic nature. This provides the opportunity to cognitively embrace an event or object at multiple levels

<sup>2</sup> As early as the 1970s, one sees a tension between the place of semiotics and the original *cognitive sciences* (founded on four primary disciplines—computer science, philosophy, psychology, and linguistics). Sebeok at that time, under the influence of Hofstadter's work in artificial intelligence, prefers the term "perception" over "cognition" in order to separate semiotic theory from the cognitive sciences and to emphasize the central role that the generation of meaning plays in understanding sign systems in dynamic interaction ([10], p. 53). By the 1980s semiotics became known in the context of not only

**14**
