*Mind and Matter - Challenges and Opportunities in Cognitive Semiotics and Aesthetics*

need to check whether the activities will develop the particular competences that they are trying to develop in their students. Additional screenshots (**Figures 13**–**16**) of the webpages were also taken by the researcher, while navigating to and within the web pages. The two websites in the analysis pertain to 'Cool Kids Facts on Photosynthesis' and 'Kids Britannica' focused on the topic of 'culture'.

As in keeping with a Semiotic Technology approach, the signs on the screen, captured from a pre-service teacher's video of her/him "checking out" different webpages that form part of other pre-service teachers' WebQuests, were analysed separately from their use. During the researcher's follow up navigation, the researcher used the opportunity to check if the signs, were 'passive' or 'active' by clicking various signs, as the pre-service teacher did not do this because he/she had his/her own navigational path and intentions. The screenshots were chosen, to illustrate how the developing typology, outlined in Section 3, might

#### **Figure 13.**

*Animated advertisement for furniture company 'Rochebobois' (right) and hyperlink for a 'quiz' (bottom).*

#### **Figure 14.**

*Animated advertisement for an technology company 'IBM' (right) and animated advertisement for furniture compant 'Rochebobois' (bottom).*

*Rethinking 'Affordance', 'Agency' and 'User' from a Semiotic Technologies Perspective… DOI: http://dx.doi.org/10.5772/intechopen.99699*

#### **Figure 15.**

*Pop-up with 'subscribe today' (bottom) and an invite to know about cookies (top).*

#### **Figure 16.**

*Tabs with different options – 'Fundamentals', 'kids', 'students', 'scholars' that can be clicked (top) and 6 square images representing 'articles and videos'.*

be applied to different educational, digital resources and scenarios apart from the Tandem App.

**Table 1** illustrates how the typology was used to analyse web pages in order to identify signs-as agents. While the analysis is not systematic<sup>1</sup> in the sense that not every sign was analysed in relation to the different semiotic modes (e.g., layout, texture, colour, sound, etc.), which is a common feature of studies from a Social Semiotics perspective, there is enough visual, textual and navigational information from the two web pages (Cool Kids Facts Photosynthesis and Kids Britannica) to tentatively apply the typology for the purposes of this chapter.

<sup>1</sup> To be systematic with these examples, the analytical tool created by Knight, Dooly Barberà [18] could be applied. It involves a detailed identification of Initiation and Response turns, agents, their role and purpose.


#### **Table 1.**

*Analysis of two webpages in order to identify signs-as-agents.*

#### **4.1 Pop-ups as active signs-as-agents**

The pop-ups that appeared when navigating to both webpages, 'Cool Kids Facts', and the 'Kids Britannica', can be conceptualised as signs as potential agents because they "invite" the user to click on them. The pop-up consent for using Data and Cookies in **Figure 9** requires the user to "Aceptar" (Accept) or "Cambiar Las Preferencias" (Change the preferences). When the user clicks in response, the exchange structure can be considered to be established and the sign is 'agentalised'.

Similarly, when navigating to the Kids Britannica webpage, a pop-up in the top right corner occurs before the webpage fully appears (**Figure 11**). The text on the pop-up says "Traducir siempre de Ingles" (Always tanslate from English) which can be considered to be an invite to change the language preferences when navigating to a page in another language. The pop-up requires the user to click "Espanyol" (Spanish) to translate to or close the pop-up indicated by an 'x'. In this case, the pop-up was closed. This pop-up was not generated by the webpage Kids Britannica but rather "Google Translate" (**Figure 11**). Importantly, this highlights a need to recognise distributed agents and agency across the "layers" [36] "of message, interface, communication flow, organisation of information, the platform" [36] in order to unveil power relationships more fully. Designers (or signmakers) can be involved not only in webpage design but also navigation design between pages, namely browser design, in this case from Google.

*Rethinking 'Affordance', 'Agency' and 'User' from a Semiotic Technologies Perspective… DOI: http://dx.doi.org/10.5772/intechopen.99699*

#### **4.2 Hypertext and visual icons as passive signs-as-agents**

Both websites had what can be considered to be 'passive agents' on the pages. These were in the form of hypertext, indicated by a different colour than the body copy (**Figures 12** and **13**) and visual icons that could be made up of colour, text and shape such as the 'Free 7 day Trial' (**Figure 12**) and 'Subscribe Today' icon (**Figure 15**). These could be clicked, or "initiated", at any moment by the user, depending on user intentions. The "response" of the system would be to move the user on to another page. Therefore, they are considered to be passive signs as potential agents that can be "agentalised" when clicked. Similarly, visual/textual icons including 'tabs' (4 Tabs in **Figure 16**) and visual icons including images could also be clicked (6 square images in **Figure 16**).

During the analysis, another type of agent was present that did not emerge in the results of the exploratory study with the Tandem tool, namely 'Animated agents'. Animated agents simulate human behaviour, such as facial expression, body movement and gesture. In the case of webpages for example, certain signs flickered on and off and people in embedded videos moved, all of which humans could click through navigation. Such animated movement can be understood as mirroring human behaviour.

#### **4.3 Animated signs-as-agents**

Finally, with respect to animated signs, only the Cool Kids Facts webpage housed these. These were located on the right hand side of the page (**Figures 13** and **14**), taking up a third of the page space. In addition, an animated sign banner was also present at the bottom of the screen saying 'Nueva Tienda' (New shop) for the furniture company Roche Borbois (**Figure 14**). This banner changed adverstisements from different companies including the companies 'Just Eat' and 'Western Union' while the researcher was observing. Clicking on this banner led the user to the various company web pages.

The key point to highlight before applying the typology to another digital tool, is to note the presence of animated signs that were highlighted in the HCI classification of agents (**Figure 3**. Classification of pedagogical agents in HCI studies), as well as passive and active signs/agents. These are signs/agents that can be agentalised through the completion of either a computer generated or human initiation and response. These turns are multimodal in nature and through analysing the webpage, as well as its use, agents can be identified.

The analysis of the webpages for WebQuests shows how the typology outlined in Section 3, can be applied to another digital tool and practice within Education. Unlike the Tandem tool, where the social interests were pedagogical in nature, the webpages house social interests that are not purely educational in nature, but rather, have legal and commercial interests. Legal interests are present in the Cookies 'agreement' as the pop-up seeks acceptance to use users' data. Commercial interests can be seen in the '7 Day Free Trial' for access to Britannia Kids Encyclopedia, which requires later payment. Although this commercial aspect does relate to the area of education, commercial interests that are unrelated to education are also present. Notably, these interests can take up approximately half of the screen space, underscoring how the co-existence of different social interests can be embued in the diferent design intentions of signs on one 'educational' webpage. While these signs/ agents can be considered as 'direct discourse agents' because users can click on them as a response or initiation, they can also be understood as attempts to shape users' tool use. The adverts (**Figures 13** and **14**) have the potential to shape a user's attention, and therefore the user's interests, away from the educational content on

the webpage on 'photosynthesis', towards commercial interests. Similarly, the navigation choices on the Cookies pop-up are limited to 'accept' and continue, to change preferences or to click away from the site to another. The use of a solid block of colour used to highlight 'accept' can be understood as an attempt to shape the user's navigation pattern towards 'accepting' a legal interest before the continuation to the educational content for the user is made navigationally possible.
