**Abstract**

This chapter revises evolving theories on cognition in relation to semiotics, the transdisciplinary study and doctrine of sign systems, and meaning-making. Cognition entails very complex networks of biological processes and actions that encompass perception, attention, manipulation of objects, memory mechanisms, and the formation of knowledge by means of direct experience as well as by learning from others, for which forms of communication and comprehension are also necessary. In view of this complexity, many different disciplines are involved in the study of cognition. These include neuroscience, anthropology, psychology, sociology, philosophy, semiotics, linguistics, and more recently, computational intelligence, information processing, and neural networks used in machine learning, to name but a few. The chapter opens with an introduction to the field of cognitive semiotics and continues with a brief presentation of the interdisciplinary evolution of the 4Es. It also includes an in-depth discussion of Peircean semiotics in relation to the approaches known as wide cognition.

**Keywords:** anthropocene, AI, 4Es, intermediality, mind-life continuum, semiotics, wide cognition

### **1. Introduction**

Jordan Zlatev defined cognitive semiotics 'CogSem' as a transdisciplinary field that integrates "methods and theories developed in the disciplines of cognitive science with methods and theories developed in semiotics and the humanities, with the ultimate aim of providing new insights into the realm of human signification and its manifestation in cultural practices" [1]. Zlatev explains that as a transdisciplinary pursuit, the study of semiotics is concerned with "the overarching unity of knowledge" [2].

CogSem has been mainly associated with the Center for Semiotics (CfS) established in 1995 in Aarhus, Denmark by Per Åage Brandt, with researchers such as Frederik Stjernfelt, Peer Bundgaard, Svend Østergaard or Riccardo Fusaroli, among others. The Centre for Cognitive Semiotics (CCS) at Lund University (Sweden) was created in 2009, with members such as Göran Sonesson and Zlatev himself. An international journal, Journal of Cognitive Semiotics (JCS), is running since 2007. The International Association for Cognitive Semiotics (IACS) was established in 2011 in Lund. Research by the so-called Grupo μ at Liège, Belgium, is also worth noting.

Semiotics occupies a transdisciplinary area that can contribute to build bridges between various disciplines; in this case cognition and its material forms of instantiation. Defined as 'the action of signs' [3], semiotics has recognized the inter-actions and intra-actions of anything acting as a sign; present also within the framework of the 4Es (embodied, embedded, enacted, extended). At a systemic scale, semiotic interactions include specific media channels, with the term 'medium' understood in its broadest sense, including bio-entities along with artefacts and technologies that can be material, and thus physically perceived by humans and animals, but also digital, which can lie beyond the scope of perception. The lines that follow attempt to advance towards a clearer formulation in this regard and point out some of the potentials of semiotic studies, focusing in particular on Charles S. Peirce's contribution.

### **2. Embodied cognition**

Research on 'Embodied Cognition' was popularized in the 1980s [4, 5]. This approach is mostly human-centred and contemplates biological factors and bodily experience, notably body symmetry, perception, and motor interaction involving the physical manipulation of objects, as affecting the formation of abstract mental structures known as 'image schemas' [6]. There is still no consensus about the specific nature of image schemas (the term 'model' is preferred by cognitive anthropologists such as David Kronenfeld). For instance, from the perspective of cognitive linguistics, a noun instantiates a schema related to 'thing' whereas a verb instantiates the schema related to 'process' [7]. Propositions are then combined into larger-scale mental and discourse structures by way of metaphorical and metonymic conceptual mappings, explained by shifts from one conceptual domain (the target domain) to another conceptual domain (the source).

Schemas could be simplified pre-conceptual experiences turned into abstract mental structures. They define how humans make sense of the world within specific perceptual and cognitive domains. Since they have functional biological bases, some schemas are basic in the sense of being irreducible to anything more fundamental. For instance, the temporal duration and spatial perception are realms of potential experience within which conceptions emerge through analogic structures that are in relation to perceptual and motor-experience (dynamic inter-actions are subject to physical constraints such as the pull of gravity) and human bodily orientation.

Thus, embodied cognition considers that abstract and high-level cognition is explained in terms of physical experiences (body as a container, based on symmetry, balance, and centre-periphery experiences; action explained as source-path-goal schema, and so on). Schemas are imported from these pre-conceptual structures by way of metaphorical and metonymic conceptual mappings. Even the language of emotions (i.e. 'you broke my heart') largely reflects culturally mediated conceptualisations of feelings in terms of body parts, transferred across domains through conceptual metaphors and metonymies.

Ultimately, schemas might be based on the human ability to detect and recognize recurrent patterns and establish mappings or conceptual correspondences from the source, generally more abstract, to the target, more grounded on the physical world [8]. Neuroscientist Antonio Damasio has identified that fundamental mappings taking place in our brain are those that register the internal state and condition of the body, as well as those that map in relation to the external environment [9].

However, sociocultural aspects are also important because humans share an inborn basis for social interaction. This is manifested for instance in their capacity to follow someone's gaze, to read intentions, face recognition, and so on. Collectively, these factors provide a universal neuro-physical basis for cognitive development, also grounded on sociocultural interaction. Applying the framework

#### *Cognitive Semiotics: An Overview DOI: http://dx.doi.org/10.5772/intechopen.101848*

proposed by Merlin Donald, Jordan Zlatev explored the re-enactment or representation via imitation as a fundamental prerequisite of shared communication. Donald defined 'mimesis' as "the ability to produce conscious, self-initiated, representational acts that are intentional but not linguistic" [10] and found it to be rooted in movements, involving a range of aspects from facial expressions, eye movement, gestural signs, and tones of voice, including the communication of emotional states. He recognized the dependence of mimesis on joint attentional frames, that is, deictic markers such as child and mother gaze interactions. Donald observed that this ability operates based on the metaphorical principle of perceptual resemblance, enhanced by repetition [11]. While he emphasized metaphor as a cognitive tool, recent research also notes the importance of metonymy, because an enactment evokes the imagery of a background scene and the elements within it (in Peirce's terms, 'indexical proximity').

Paul Friedrich aligns image schemas to Peirce's notion of 'Firstness', explaining that they depend on qualities (Peirce's 'qualisign' refers to a quality, a feeling or a possibility of functioning as a sign) that are primary or irreducible, leaning towards physicality, and dependent on emotional content. Peirce wrote that a quality "cannot act as a sign until it is embodied; but the embodiment has nothing to do with its character as a sign" [12, 13]. Even habits, routines, and general laws (that Peirce integrates in 'Thirdness') are embodied in actual things or events (themselves manifesting a range of possible qualities). Thus, ultimately, the fact that something is interpreted as a sign depends on material phenomenological aspects. Peirce's wellknown classification of signs as icons, indexes, and symbols is based on how signs are related to their objects. An icon signifies its object by virtue of shared qualities, an index by virtue of a causal relation, and a symbol by virtue of an action ruled by a norm or habit [14] (see the section below for more details).

'Secondness' initiates relations between domains and is essentially dyadic. According to Friedrich, the modal forms that accompany some verbal processes include expressions of mood that run from emphatic assertion to passivity and emotional content such as outrage, fury, joy and sadness, sarcasm, threat, and irony. These can interact and combine with each other introducing a 'sinsign', a sign usually consisting in a reaction/resistance, or an actual singular thing, occurrence, or fact. Peirce also held that an index can be a general thing (not only singular; the etymology of 'seme' points in this direction). For instance, a symptom of a disease, a label, a diagram (which can be both iconic and indexical), a proper name, a pronoun, etc. Indexes or pointers make connections through spatiotemporal proximity or contiguity, crucially bound up with the situation or context. If interpreted as linguistic signs in triadic symbolic relations, pointers become personal pronouns (I, you, we, they, etc.), deictic adverbs (here/there, now/then), demonstratives (this/that), and grammatical categories of tense and aspect, all of which that are situationally contingent.

Thus, starting at the lowest level, embodiment creates the potential for schemas to emerge, as conceptual blending theory has shown [8]. Schemas are associated with experientially based forms of behaviour, specific to certain situations. The cognitive operations originating in bodily experience pass through processes of metaphorical and metonymic projections based on recognition of patterns presented in experience. Once corresponding qualities between material and mental spaces have been mapped, their integrative projections yield symbolic signs. As structures become more symbolic, their connection with bodily experience turns more indirect, and cultural particularities emerge. Particularities are filled-in with salient 'situated' cultural content within specific population groups. Conceptual integration includes out-of-awareness forms of cognition, such as tacit knowledge of what is possible, permissible, and acceptable within a community [15].
