*Mind and Matter - Challenges and Opportunities in Cognitive Semiotics and Aesthetics*

whereby some resources can act as or be oriented towards as participants in the interaction" [23].

This dual-focused analysis, particularly in the analysis of learners' oral turns from the audio-recordings, also revealed another type of human-computer 'relationship' in the form of screen-based resources that acted as "shaping agents" [18]. The resources that acted as "shaping agents" were invoked or made relevant in the context of learners' oral turns, thereby they became part of the oral messages in the human, user-user, oral interaction. Specifically, the screen-based resources became: 1) embedded or modified in oral turns (as in **Figure 6**); 2) resources to initiate and support oral turns (also in **Figure 6**) or 3) diverse topics of talk (as in **Figure 7**).

Regarding shaping agents, The '?'sign, with suggested answers next to it (**Figure 6**), was used by learners as a resource to initiate and support oral turns. Also in **Figure 6**, a textual lexical item on the screen, 'Where do you live?' becomes embedded in learners' oral turns.

#### **Figure 6.**

*Transcript of student-student oral interaction whereby screen-based resources became embedded or modified in oral turns or resources to initiate and support oral turns.*


#### **Figure 7.**

*The screen-based resources become topics of talk.*

*Rethinking 'Affordance', 'Agency' and 'User' from a Semiotic Technologies Perspective… DOI: http://dx.doi.org/10.5772/intechopen.99699*

In **Figure 7**, the visual textual signs are used as topics of talk, namely about whether the students are going to 'close' the pop-up or not, a word that also appears on the pop-up.

Signs were also considered turn-takers in the interaction, as shown in **Figure 7** where the pop-ups initiates an action and the students refer to the pop-up as "the computer" or "it" (**Figure 7**).

These examples and results from the exploratory study on learner agency in an online language scenario contributed theoretically as a first step towards a typology of signs-as-agents: Signs can shape human (oral) turns [23] and can be understood as 'shaping agents' [18]. In addition, signs that can establish an exchange structure with humans (either 'system-user' [19] or 'user-system' [18]) can be understood as either

**Figure 8.** *Emerging typology: Version A.*

'active' or 'passive' 'direct discourse agents' [18]. This emerging typology is illustrated in **Figure 8**. It reflects the classification of agents from HCI studies but also expands on how the concept of agents relates to the notion of signs from the field of Social Semiotics. The notion of "shaping agents" is not in **Figure 3** of the classification of pedagogical agents in HCI studies because social practices, or use of the digital tool or signs was not explicitly considered in such HCI studies. However, I incorporate it into the typology of signs-as agents now in **Figure 8**. **Figure 8** also includes the notion related to "shaping agents", that signs are agentalised when the screen-based signs shape the human (oral) turn in some way.

The following sections, Sections 4 and 5, illustrate how this emerging typology can be applied to two other educational scenarios, involving digital tools, and can be developed even further.
