Contents

#### **Preface XI**


Vladimir L. Averbukh



## Preface

Chapter 7 **Developing Building Information Modelling for Facility Services with Organisational Semiotics 135**

Chapter 8 **Grounding Functional Requirements Classification in**

Chapter 9 **An Operational Approach to Conceptual Understanding Using**

Chapter 10 **Using Signs for Learning and Teaching Physics: From Semiotic Tools to Situations of Misunderstanding 185**

Bohan Tian and Haomin Jiang

**Organizational Semiotics 151**

Alaric Kohler and Bernard Chabloz

**Section 5 Semiotics and Science Education 171**

**Semiotic Theory 173** Daniel L. McGee

Richa Sharma

**VI** Contents

Semiotics has a long tradition as the science of signs, signification and meaning-making. Four traditions have contributed to Western semiotics: semantics (including the philosophy of language), logic, rhetoric and hermeneutics. However, as this volume will show, there are many other fields contributing to make semiotics an interdisciplinary arena and an evergrowing field of interest.

The volume opens with a paper entitled 'Semiotics of Conscience' by **Dr. Rufus Duits**, which offers a fundamental introduction to the semiotic analysis of conscience. The paper starts from Martin Heidegger's phenomenological analysis and moves towards a Peircean perspective where the author discusses the concept of conscience as a fundamental semiotic operation beyond deconstructive scepticism. Positing conscience as a psychical component of semiosis, based on Heidegger's insights on the notion of 'care' and on how human hori‐ zons are mapped out by concern, Dr. Duits stresses that it is not only that humans seek to create meanings, but that humans' perceived needs and desires give rise to such meanings. Objects around us become part of our perceptual experience only when they acquire mean‐ ing for our task-oriented actions. Organised around goals to be accomplished, they only take significance from the projects we envision to perform with them. Thus, in Heidegger's view, conscience is a sort of inner moral 'voice', the result of the phenomenological structure of 'care' and the notion of responsibility attached to it, defined ontologically around three so‐ cial moments: the call (i.e. the message), the caller and the one summoned by the call. The author sees the three Peircean semiotic moments—*representamen* (the sign vehicle, or form that the sign takes), *object* (the referent that the sign stands for) and *interpretant* (the sense conveyed by the sign)—as part of a liminal conscience sign, 'at the boundary of semiosis'; a 'sign of signs' where 'the sign vehicle, the sense and the referent of the sign are all the same thing', 'the point at which semiosis begins or ends', writes the author. In the last part of the paper, the analysis moves towards ethical implications of the theory of signs based on the notion of 'concern' for the self. To the author, this is an essentially normative dimension that compels a way of thinking about semiotics that is inherently critical.

The paper by **Dr. Miguel López-Astorga** 'Mental Models Are Compatible with Logical Forms' addresses the continuing debate as to whether human reasoning is based on mental models versus formal rules of inference, domain-specific rules of inference or probabilities. Thus, the paper traces the relationships between MMT (which focuses on the semantic as‐ pects of content, drawing on inference reasoning rather than on syntax or formal structure) and standard logic. López-Astorga's research includes correspondences between the combi‐ nations of possibilities that MMT assigns to some of the traditional connectives in standard logic (e.g. the conditional conjunction and the inclusive disjunction), the truth value and the definitions that standard logic attributes to the same connectives. The paper also shows that

the possibilities or models that, following MMT, correspond to those connectives and their negations are evidently compatible with the truth tables in standard logic for those very connectives and their negations. The author proposes that the models that MMT assigns to such connectives are consistent with their definitions by means of other connectives valid in standard logic.

**Dr. Fionn Bennett** deals with 'The Art of "Scoring" Cosmopoiesis in Archaic Melic Verse'. This chapter sheds light on the semiotic correspondence between the arithmetic of music for the Hellenes and the blends of cosmic energies behind sacred melodies and subject matter. Delving into Plato's works, the author explores the experience of the divine and how this experience was enacted in musical melic verse and conveyed effectively to the audiences at the time, thus contributing through the values, norms and aspirations encod‐ ed to create a sense of community. Dr. Bennett also points out how melic verse had the power to 'epi-*phon*-ise', that is, to re-enact the 'astrocentric paradigm of cosmopoiesis' or the 'harmony of the spheres', exerting a sort of cosmic agency upon being-in-the-world. The paper explores the semiotic relational behind these sacred songs, which voiced the divine because the melodies and rhythms that accompanied the words mimed cosmic powers and encoded immortal intelligences.

The paper by **Dr. Mark Reybrouck**, 'Music and Semiotics: An Experiential Approach to Mu‐ sical Sense-Making', explores recent cognitive bio-semiotic theory, ecological psychology and the transition from a disembodied to an embodied approach as applied to music. Rey‐ brouck brings to the fore three dimensions of musical sense-making: the syntactic, the se‐ mantic and the pragmatic levels. Within the syntactic, the author notes the conceptual tools of deixis and indexical devices, which anchor referential exchanges, systematised in terms of personal, spatial and temporal axes, defined with reference to the sound event. However, Reybrouck argues that the deictic approach, as applied to music, favours an experientialist as against a merely conceptual-symbolic, stressing first-hand information in perception rath‐ er than relying on second-order stimuli. Such approach might disregard the centripetal ten‐ dency in music, where attention is directed from external references to the semantic selfreflection of elements that trigger internal processes of sense-making (bodily resonance) along the lines of the cognitive embodiment hypothesis, which understands perception as perceptually guided enactment. The listener conceives the sounds of music at a symbolic level or representation, beyond their experiential qualities. Cognitive neuroscience has also noted this inductive power of music on the human brain and the reactions within the inter‐ nal environment of the body. With regard to the pragmatic level, the concept of 'musical affordances' is of particular importance and deals with the extensions of the possibilities of sound production at several levels (i.e. musical instruments, the shaping of sound through playing and modulatory techniques and motor induction, including the possibility of mov‐ ing in reaction to music). These affordances make possible the conception of music in terms of 'activity signatures', writes Reybrouck, including the mental simulation of movement in terms of bodily based schemata induced by music.

**Dr. Vladimir L. Averbukh**'s contribution to the present volume, entitled 'Semiotic Analysis of Computer Visualisation', discusses the foundation of design, development and evaluation of visualisation systems from a semiotic perspective. Computer visualisation involves three interrelated areas: computer graphics (hardware and software including mathematical and algorithm components), software engineering and human factors. This chapter connects the author's previous research on the human factor subdomain on the transformation of the

symbolic into figurative geometric visualisation, involving three aspects: perception, cogni‐ tion and interpretation. Averbukh describes human-computer interaction as a semiotic proc‐ ess (a unity of lexicon, syntax, semantics and pragmatics), where visualisation is also a sign process and language is understood as a base sign system, which involves the systematic description of entities under consideration, methods of their representation, modes of changes of visual display, as well as techniques of manipulations and interaction with them. Displays are explored as elements of visualisation lexicons, whereas syntax is the set of rules describing (a) relationships of visual objects, (b) possible dynamics of visual objects and (c) techniques and results of interaction with visual objects. Semantics specifies the goals and tasks of computer modelling. Finally, pragmatics determines meaning drawn by users. The paper includes concrete examples of 2D and 3D graphs and their semiotic interpretation.

the possibilities or models that, following MMT, correspond to those connectives and their negations are evidently compatible with the truth tables in standard logic for those very connectives and their negations. The author proposes that the models that MMT assigns to such connectives are consistent with their definitions by means of other connectives valid in

**Dr. Fionn Bennett** deals with 'The Art of "Scoring" Cosmopoiesis in Archaic Melic Verse'. This chapter sheds light on the semiotic correspondence between the arithmetic of music for the Hellenes and the blends of cosmic energies behind sacred melodies and subject matter. Delving into Plato's works, the author explores the experience of the divine and how this experience was enacted in musical melic verse and conveyed effectively to the audiences at the time, thus contributing through the values, norms and aspirations encod‐ ed to create a sense of community. Dr. Bennett also points out how melic verse had the power to 'epi-*phon*-ise', that is, to re-enact the 'astrocentric paradigm of cosmopoiesis' or the 'harmony of the spheres', exerting a sort of cosmic agency upon being-in-the-world. The paper explores the semiotic relational behind these sacred songs, which voiced the divine because the melodies and rhythms that accompanied the words mimed cosmic

The paper by **Dr. Mark Reybrouck**, 'Music and Semiotics: An Experiential Approach to Mu‐ sical Sense-Making', explores recent cognitive bio-semiotic theory, ecological psychology and the transition from a disembodied to an embodied approach as applied to music. Rey‐ brouck brings to the fore three dimensions of musical sense-making: the syntactic, the se‐ mantic and the pragmatic levels. Within the syntactic, the author notes the conceptual tools of deixis and indexical devices, which anchor referential exchanges, systematised in terms of personal, spatial and temporal axes, defined with reference to the sound event. However, Reybrouck argues that the deictic approach, as applied to music, favours an experientialist as against a merely conceptual-symbolic, stressing first-hand information in perception rath‐ er than relying on second-order stimuli. Such approach might disregard the centripetal ten‐ dency in music, where attention is directed from external references to the semantic selfreflection of elements that trigger internal processes of sense-making (bodily resonance) along the lines of the cognitive embodiment hypothesis, which understands perception as perceptually guided enactment. The listener conceives the sounds of music at a symbolic level or representation, beyond their experiential qualities. Cognitive neuroscience has also noted this inductive power of music on the human brain and the reactions within the inter‐ nal environment of the body. With regard to the pragmatic level, the concept of 'musical affordances' is of particular importance and deals with the extensions of the possibilities of sound production at several levels (i.e. musical instruments, the shaping of sound through playing and modulatory techniques and motor induction, including the possibility of mov‐ ing in reaction to music). These affordances make possible the conception of music in terms of 'activity signatures', writes Reybrouck, including the mental simulation of movement in

**Dr. Vladimir L. Averbukh**'s contribution to the present volume, entitled 'Semiotic Analysis of Computer Visualisation', discusses the foundation of design, development and evaluation of visualisation systems from a semiotic perspective. Computer visualisation involves three interrelated areas: computer graphics (hardware and software including mathematical and algorithm components), software engineering and human factors. This chapter connects the author's previous research on the human factor subdomain on the transformation of the

standard logic.

VIII Preface

powers and encoded immortal intelligences.

terms of bodily based schemata induced by music.

The chapter 'Developing Building Information Modelling for Facility Services with Organi‐ sational Semiotics' co-authored by **Dr. Bohan Tian** and **Dr. Haomin Jiang** explores space and habitat facilities where services are provided. The research uses organisational semiotics (OS) to explore Building Information Modelling (BIM) and employs an object-oriented mod‐ elling technology that integrates information from building projects in order to enhance BIM and target facility service activities. BIM has advantages in facilitating design and construc‐ tion because it can provide specific object description by means of 3D, 4D (which includes time scheduling) and 5D (with integrated cost estimating). The integration of FM (facility management) and BIM is an emerging area. Facility services are generally operated and de‐ livered according to personal preference and organisational policies. Their integration into technical engineering knowledge and the semantic and knowledge-based building informa‐ tion, which includes social and organisational aspects, is the aim of the FM-BIM model in order to connect it to D&B (Design and Build) and O&M (Operation and Maintenance), which enables FM managers to better understand how a building is operated and optimised. In this sense, organisational semiotics (OS) provides a framework that integrates six semiotic levels: social, pragmatic, semantic, syntactic, empirical and physical. Thus, the paper consid‐ ers buildings as sociotechnical environments whose virtual representations as BIM can be contemplated as complex sign systems that allow stakeholders' interaction. The paper also explores how OS can bridge Building Information Modelling (BIM) by focusing on building fabrics and on facility management activities that concern service management.

Similarly, **Dr. Richa Sharma's** paper 'Grounding Functional Requirements Classification in Organisational Semiotics' uses organisational semiotics (OS) in order to analyse functional requirements in software engineering regarding three defining parameters to ensure consis‐ tent, correct, complete and unambiguous requirements: repeatability, quantifiability and systematic thought processes. The paper tries to define these parameters more clearly and put them in relation to activities in requirements engineering (RE) regarding the type of soft‐ ware system concerned, its applicability to other systems and, more specifically, the validity of the proposed solutions with regard to RE activities. The chapter is concerned with infor‐ mation systems that are database driven and have applications in retail, financing, ERP sys‐ tems, etc. These information systems need to embed organisation structure, hierarchy, policies, processes and behaviour in the form of software requirements. The paper argues that OS presents a feasible solution towards understanding the requirements of such infor‐ mation systems. The experimental study presents two separate approaches: one semiauto‐ mated using lexical heuristics and word-tagging and the second of ML classification. Observations drawn from both approaches are similar in the goal of searching for an auto‐ matised process of identifying and extracting functional requirements from existing docu‐ mentation using organisational semiotics. Results are encouraging, revealing that one solution towards a classification scheme for wide-spectrum software systems is functional requirements categorisation by grounding the classification scheme in an established theory of organisational semiotics.

**Dr. Daniel L. McGee, Jr.** discusses 'An Operational Approach to Conceptual Understand‐ ing Using Semiotic Theory' in mathematical representations that employ algebraic, geo‐ metric, numerical and verbal registers when concepts are synergically presented and discussed. The paper starts from previous theoretical conceptions of 'simultaneous aware‐ ness' of the various registers of representations associated with mathematical concepts, for instance, the semiotic chain of two associated conversions each of which is represented by an arrow in the diagram: (1) the geometric register to the numerical register and (2) the numerical register to the symbolic register. It gradually moves to more complex conver‐ sions employing six variables and to recent theories that present operational frameworks to map this fluidity of registers. The research summarises data obtained from these stud‐ ies and provides insight into their implications, applications and methodology in assess‐ ing student understanding. In a classroom context, the simultaneous mobilisation of representations can be very difficult, and the vast majority of learning modules and text‐ books use an ordered sequence of semiotic registers when presenting a mathematical con‐ cept. The paper argues that students are accustomed to starting with a symbolic representation (a formula), performing a conversion to obtain the associated numerical representation (a table) and finally obtaining a graphical representation through ordered pairs on the Cartesian plane, leading to a unique semiotic chain: symbolic register --> numeric register --> geometric register. Being unable to perform conversions not included in this semiotic chain, such as geometric register --> numeric register (graph to table) or numerical register --> symbolic register (table to formula), suggests that awareness of reg‐ isters occurs sequentially as they appear in the semiotic chain. The author considers this inconsistent with a simultaneous awareness or a synergy of registers, which is achieved only if students are able to pass seamlessly among table, graph and formula.

The last paper in the volume continues to develop the topic of science education, conceptual change and teaching methods and approaches. 'Using Signs for Learning and Teaching Physics: From Semiotic Tools to Situations of Misunderstanding' by **Dr. Alaric Kohler** and **Dr. Bernard Chabloz** investigates a few semiotic objects mediating the communication in physics classroom, in particular the usage of arrows and graphics. The authors explore stu‐ dents' understanding in problem-solving tasks by taking examples drawn from research da‐ ta at the high school and college level. In the case of the 'arrow' as a semiotic object, the lack of clues or conventions in its use might lead to misunderstandings emerging in the class‐ room. Challenges addressed include (a) the coordination between various semiotic registers and objects, extending previous research by Duval in order to analyse specific cognitive tasks of interpretation of signs of various kinds within their specific semiotic context; (b) the lack of clues or conventions in the use of semiotic objects that can play different semiotic roles and (c) the communicative counterpart of the use of semiotic tools for mediating knowledge. In order to explain the coordination of semiotic objects and registers, the au‐ thors start from Piaget's theoretical framework where the coordination is a higher-order process relating operations on (semiotic) objects (i.e. signs), but in order to avoid the theoret‐ ical reductionism inherent to formal logic, they rely on Grize's logic-discursive operations

rather than on Piaget's logic. While interpreting physics tasks, the challenge of semiotic co‐ ordination involving several resisters comes from the establishment of synergic patterns of logic-discursive operations.

matised process of identifying and extracting functional requirements from existing docu‐ mentation using organisational semiotics. Results are encouraging, revealing that one solution towards a classification scheme for wide-spectrum software systems is functional requirements categorisation by grounding the classification scheme in an established theory

**Dr. Daniel L. McGee, Jr.** discusses 'An Operational Approach to Conceptual Understand‐ ing Using Semiotic Theory' in mathematical representations that employ algebraic, geo‐ metric, numerical and verbal registers when concepts are synergically presented and discussed. The paper starts from previous theoretical conceptions of 'simultaneous aware‐ ness' of the various registers of representations associated with mathematical concepts, for instance, the semiotic chain of two associated conversions each of which is represented by an arrow in the diagram: (1) the geometric register to the numerical register and (2) the numerical register to the symbolic register. It gradually moves to more complex conver‐ sions employing six variables and to recent theories that present operational frameworks to map this fluidity of registers. The research summarises data obtained from these stud‐ ies and provides insight into their implications, applications and methodology in assess‐ ing student understanding. In a classroom context, the simultaneous mobilisation of representations can be very difficult, and the vast majority of learning modules and text‐ books use an ordered sequence of semiotic registers when presenting a mathematical con‐ cept. The paper argues that students are accustomed to starting with a symbolic representation (a formula), performing a conversion to obtain the associated numerical representation (a table) and finally obtaining a graphical representation through ordered pairs on the Cartesian plane, leading to a unique semiotic chain: symbolic register --> numeric register --> geometric register. Being unable to perform conversions not included in this semiotic chain, such as geometric register --> numeric register (graph to table) or numerical register --> symbolic register (table to formula), suggests that awareness of reg‐ isters occurs sequentially as they appear in the semiotic chain. The author considers this inconsistent with a simultaneous awareness or a synergy of registers, which is achieved

only if students are able to pass seamlessly among table, graph and formula.

The last paper in the volume continues to develop the topic of science education, conceptual change and teaching methods and approaches. 'Using Signs for Learning and Teaching Physics: From Semiotic Tools to Situations of Misunderstanding' by **Dr. Alaric Kohler** and **Dr. Bernard Chabloz** investigates a few semiotic objects mediating the communication in physics classroom, in particular the usage of arrows and graphics. The authors explore stu‐ dents' understanding in problem-solving tasks by taking examples drawn from research da‐ ta at the high school and college level. In the case of the 'arrow' as a semiotic object, the lack of clues or conventions in its use might lead to misunderstandings emerging in the class‐ room. Challenges addressed include (a) the coordination between various semiotic registers and objects, extending previous research by Duval in order to analyse specific cognitive tasks of interpretation of signs of various kinds within their specific semiotic context; (b) the lack of clues or conventions in the use of semiotic objects that can play different semiotic roles and (c) the communicative counterpart of the use of semiotic tools for mediating knowledge. In order to explain the coordination of semiotic objects and registers, the au‐ thors start from Piaget's theoretical framework where the coordination is a higher-order process relating operations on (semiotic) objects (i.e. signs), but in order to avoid the theoret‐ ical reductionism inherent to formal logic, they rely on Grize's logic-discursive operations

of organisational semiotics.

X Preface

To conclude, we can affirm that papers in the collection prove that semiotics continues to provide a framework for varied emerging knowledge traditions.

> **Asunción López-Varela Azcárate** Universidad Complutense de Madrid, Spain

**Section 1**
