**Designing Dialogs between Users and Products through a Sensory Language through a Sensory Language**

**Designing Dialogs between Users and Products** 

DOI: 10.5772/intechopen.71127

Sara Bergamaschi, Lucia Rampino and Jelle van Dijk

Additional information is available at the end of the chapter Additional information is available at the end of the chapter

Sara Bergamaschi, Lucia Rampino and

http://dx.doi.org/10.5772/intechopen.71127

#### **Abstract**

Jelle van Dijk

This paper presents a research-through-design exploring interaction as a dialog between the user and the product that are in contact for sharing something, typically quantitative and/or qualitative information (e.g., data, points of view, feelings, and so on). This exchange is made possible by the implementation of a given kind of common language. Traditionally, human-computer interaction relies on an explicit, codified language, as for example when designers use icons, text, or pictures to convey a message. In contrast, we define *empirical sensory language* as those sensory stimuli coming from an artifact, processed most often unconsciously, which play a constructive role in generating a meaningful interactive experience, yet do not require any explicit exchange of information messages. Our investigation aimed at exploring potentialities and limits of applying a sensory language to arouse meaningful interactions leading to a desired change in routine behaviors. We thus designed two product prototypes intended to lead users to decrease water consumption. Our approach opens up a new space for design that is currently not covered by explicit, codified forms of interaction. We discuss implications for a product designer to design for a sensory language and the results of an exploratory user evaluation.

**Keywords:** sensory language, material interaction, user-product dialog, water saving

### **1. Introduction**

In recent years, microchips have become so small that they can be embedded in traditional materials such as wood, glass, polymers, fabrics and even more, making such materials "smart" or, as Vallgårda and Redström define them [1], "computational." Thanks to computational materials, products can now change their sensory features (i.e., shape, color, texture, etc.)

© 2016 The Author(s). Licensee InTech. This chapter is distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/3.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited. © 2017 The Author(s). Licensee InTech. This chapter is distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/3.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

proactively and in a reversible manner, according to a specific situation. Such changes can be designed with an informative intent, i.e., a device that changes shape to communicate the temperature inside the home. This means product designers have currently new material opportunities to work on; and the industrial design field can be enriched with new forms of material interaction and novel ways to convey meanings: indeed, products are able to establish dialogs with users through their sensory features, defining a novel "sensory language." Previous studies demonstrated that the use of a sensory language instead of the traditional alphanumerical one, conveyed through digital screens, could be a fruitful strategy to engage users during the interaction with products [2]. The positive engagement and the sensorial richness offered by the materiality of artifacts might support the behavioral changes during the interaction with products.

In order to explore both limits and potentialities of applying a sensory language to the userproduct interaction, we focused on the issue of water savings in the household environment. Indeed, this appeared to be a promising field of application: previous studies [3–6] stressed the importance to establish a dialog with the final user (mainly through digital screen) in order to change his/her behavior toward more sustainable consumptions.

### **2. Interaction as a dialog**

Technological evolution makes products dynamic and interactive. Products can be designed to be responsive and able to adapt to the surroundings (environment, users, and context in general). Thus, "a domain which was once considered pure industrial design is faced with many interaction design challenges" [7].

"An interaction is a transaction between two entities, typically an exchange of information" [8]. Indeed, in the Cambridge dictionary, an interaction is defined as "an occasion when two or more people or things communicate with or react to each other"1 . Thus, the main aspect of an interaction is that both the involved subjects have to be reactive and responsive to each other. For this reason, we assume that users and products should be related to each other in a circle of influences: the object with its (changing) material features (e.g., shape, weight, color, etc.) affects the user's behaviors and thoughts, and vice versa.

The idea of a transaction between two entities in an interactive relationship and the idea of an exchange of information over time is also at the core of the definition of "dialog": "A conversation between two or more people"2 . To converse means to "talk between two or more people in which thoughts, feelings, and ideas are expressed, questions are asked and answered, or news and information is exchanged"3 .

In these definitions, it is possible to find several correspondences between an "interaction" and a "dialogue." Both require the involvement of two actors who are in contact for sharing

<sup>1</sup> http://dictionary.cambridge.org/dictionary/english/interaction.

<sup>2</sup> http://www.oxforddictionaries.com/definition/english/dialogue.

<sup>3</sup> http://dictionary.cambridge.org/dictionary/english/conversation.

something. Typically, they share information but it is also possible to enrich this relationship with emotional factors (like feelings) and personal point of view (like thoughts and ideas). This sharing creates a cycle of correspondences between actions and responses that is inherent to the human use of artifacts [9].

#### **2.1. Language as a design matter**

proactively and in a reversible manner, according to a specific situation. Such changes can be designed with an informative intent, i.e., a device that changes shape to communicate the temperature inside the home. This means product designers have currently new material opportunities to work on; and the industrial design field can be enriched with new forms of material interaction and novel ways to convey meanings: indeed, products are able to establish dialogs with users through their sensory features, defining a novel "sensory language." Previous studies demonstrated that the use of a sensory language instead of the traditional alphanumerical one, conveyed through digital screens, could be a fruitful strategy to engage users during the interaction with products [2]. The positive engagement and the sensorial richness offered by the materiality of artifacts might support the behavioral

Proceedings of the Conference on Design and Semantics of Form and Movement - Sense and Sensitivity, DeSForM

In order to explore both limits and potentialities of applying a sensory language to the userproduct interaction, we focused on the issue of water savings in the household environment. Indeed, this appeared to be a promising field of application: previous studies [3–6] stressed the importance to establish a dialog with the final user (mainly through digital screen) in

Technological evolution makes products dynamic and interactive. Products can be designed to be responsive and able to adapt to the surroundings (environment, users, and context in general). Thus, "a domain which was once considered pure industrial design is faced with

"An interaction is a transaction between two entities, typically an exchange of information" [8]. Indeed, in the Cambridge dictionary, an interaction is defined as "an occasion when two

of an interaction is that both the involved subjects have to be reactive and responsive to each other. For this reason, we assume that users and products should be related to each other in a circle of influences: the object with its (changing) material features (e.g., shape, weight, color,

The idea of a transaction between two entities in an interactive relationship and the idea of an exchange of information over time is also at the core of the definition of "dialog": "A conversa-

in which thoughts, feelings, and ideas are expressed, questions are asked and answered, or

In these definitions, it is possible to find several correspondences between an "interaction" and a "dialogue." Both require the involvement of two actors who are in contact for sharing

. To converse means to "talk between two or more people

. Thus, the main aspect

order to change his/her behavior toward more sustainable consumptions.

or more people or things communicate with or react to each other"1

.

etc.) affects the user's behaviors and thoughts, and vice versa.

changes during the interaction with products.

**2. Interaction as a dialog**

2017

150

many interaction design challenges" [7].

tion between two or more people"2

news and information is exchanged"3

http://dictionary.cambridge.org/dictionary/english/interaction.

http://www.oxforddictionaries.com/definition/english/dialogue.

http://dictionary.cambridge.org/dictionary/english/conversation.

1

2

3

"Nowadays, design becomes a matter of using the right language to generate a dialogue about the functionality, intended use of the object, and to generate thoughts and meanings in the user's mind" [10].

Dialog, as a way for sharing ideas and information, requires a common basis, the language. In the interaction design field, Moggridge [11] categorized four languages according to their "dimensions" (**Figure 1**):


The limit of this categorization is that languages are considered as separated. However, they can be used together to enrich the interaction between a system (products or digital interface) and a user. We indeed believe that any of these languages may be enriched transferring some features of one to another. Particularly, this research is based on a number of philosophical studies of the use of words (i.e., 1D language), which suggest two different ways of using them: as a *codified language* and as an *empirical language* [12–14].

Starting from the phenomenological analyses of Merleau-Ponty [15], we define *codified language* as a form of expression based on conventions/symbols/codes shared by a group of people. This language is useful to be applied when the contents of the information have to be clear and not misunderstood. For example, the traffic light informs users about their

**Figure 1.** Representation of the four languages.

possibility to cross the street applying three colors with conventional meanings: green, red, and yellow. One important aspect of codified language is that there is an arbitrary link between the code (e.g., the shape of a letter) and what it signifies (an element in a linguistic utterance). Furthermore, when a designer uses these codes to convey a message, the meaning of each of the codes is predefined: it does not depend on the situational details of any particular interaction in which the codes figure. So, for example, it does not matter on the emotional state or the previous series of events a person experienced for determining the meaning of the "black coffee" symbol on the coffee machine: it means black coffee. It could be that the person is lead to misunderstand the correct meaning, for example when the symbol looks a bit like cappuccino and the person is tired and not paying attention. But this kind of variability in meaning would be considered an error and certainly not an intended aspect of the design.

On the other side, we defined *empirical language* a form of expression in which the contents of the dialog are not completely established a priori, but meanings are to some extent subjectively created by the user during the interaction itself. In other words, the contribution of both the designed artifact and the person determine what meaning is experienced, rather than the artifact or the person "transferring" certain predefined meanings over a codified communication channel. Empirical language is often used to elicit or catalyze meaningful experiences, which the person will have courtesy of being engaged in. Paradigmatic examples can be found in the artistic and poetic fields.

Historically, industrial designers are familiar with the physical forms of artifacts. For this reason, this research focuses on 3D languages, interpreting them not only as pure physical shapes but, more in general, as all that can be perceived by the human sensory apparatus. In other words, this research intends "sensory language" as a language based on any kind of stimuli that can be perceived by one of the five senses (view, smell, touch, taste, and hearing). The concept "Scent of Time" (**Figure 2**) is a good example: it is a clock that releases a specific smell in the environment to denote a specific time of the day.

**Figure 2.** Scent of time by Hyun Choi.

### **3. Research-through-design question**

This research aimed at exploring the design qualities of a sensory language in establishing meaningful dialogs between users and interactive products, used in mundane, practical settings (i.e., not as an art exhibition or critical provocation). To this end, we delve into the differences of applying a *codified* versus an *empirical* language, also attempting to create a "cross-fertilization" between features that traditionally apply to well-separated kind of languages: the 1D language (based on words) and the 3D language (based on materiality as defined above).

As already said, nowadays, in applying a sensory language to products, any designer can exploit the new opportunities coming from the development of smart materials and new technologies.

### **4. Methodology**

We applied a research-through-design approach: two design activities were carried out in order to develop and test design proposals based on insights coming from the theoretical investigation.

The first design activity explored the sensory language as a *codified language*, and the second activity explored the sensory language as an *empirical language*. As a result, two functioning prototypes were developed: "glass of water" and "feelings and experiences for an embodied learning" (F.E.E.L.).

The two prototypes were tested with users during three focus groups, organized with the aim of exploring the interests, feelings, and engagement of the users as well as their understanding of the information conveyed.

### **5. Design activities**

#### **5.1. Glass of water**

**Figure 2.** Scent of time by Hyun Choi.

be found in the artistic and poetic fields.

2017

152

smell in the environment to denote a specific time of the day.

possibility to cross the street applying three colors with conventional meanings: green, red, and yellow. One important aspect of codified language is that there is an arbitrary link between the code (e.g., the shape of a letter) and what it signifies (an element in a linguistic utterance). Furthermore, when a designer uses these codes to convey a message, the meaning of each of the codes is predefined: it does not depend on the situational details of any particular interaction in which the codes figure. So, for example, it does not matter on the emotional state or the previous series of events a person experienced for determining the meaning of the "black coffee" symbol on the coffee machine: it means black coffee. It could be that the person is lead to misunderstand the correct meaning, for example when the symbol looks a bit like cappuccino and the person is tired and not paying attention. But this kind of variability in meaning would be considered an error and certainly not an intended aspect of the design.

Proceedings of the Conference on Design and Semantics of Form and Movement - Sense and Sensitivity, DeSForM

On the other side, we defined *empirical language* a form of expression in which the contents of the dialog are not completely established a priori, but meanings are to some extent subjectively created by the user during the interaction itself. In other words, the contribution of both the designed artifact and the person determine what meaning is experienced, rather than the artifact or the person "transferring" certain predefined meanings over a codified communication channel. Empirical language is often used to elicit or catalyze meaningful experiences, which the person will have courtesy of being engaged in. Paradigmatic examples can

Historically, industrial designers are familiar with the physical forms of artifacts. For this reason, this research focuses on 3D languages, interpreting them not only as pure physical shapes but, more in general, as all that can be perceived by the human sensory apparatus. In other words, this research intends "sensory language" as a language based on any kind of stimuli that can be perceived by one of the five senses (view, smell, touch, taste, and hearing). The concept "Scent of Time" (**Figure 2**) is a good example: it is a clock that releases a specific

> The first design activity focused on the exploration of a *codified language* based on the correspondence between a signifier and significant. In the field of semiotics, this correspondence is called "code" [16]. Specifically, Lachman et al. [17] defined a code as: *"a set of specific rules or transformations whereby messages, signals, or states of the world are converted from one representation to another, one medium of energy to another, one physical state to another"* (p. 68). In brief, codes specify how information is to be converted from one form to another [18].

#### *5.1.1. How to transform alphanumerical information into sensorial stimuli*

A literature review was performed in order to define a set of specific communication parameters that can be useful for designers to translate information related to water

consumption into a sensory language. The objective of this investigation was to find new ways to inspire designers in the concept generation phase. The result is a list of opportunities/limits for the application of the sensory language to convey certain kinds of information that are usually intended to be quantitative (e.g., average water/energy consumption per day/month/year).

Even if the general focus was on the exploration of tangible aspects of products, in this phase, we enlarged the research also to the field of digital interfaces, persuasive technologies, and interaction design. Indeed, in these fields, several studies have already investigated the importance of giving feedbacks on energy consumption, and some of these studies tried to give suggestions to designers facing this matter. Only studies reporting results on users' investigation were considered, since our intent was to explore the user understanding of the contents of the dialog. As a result, 13 studies [5, 6, 17, 19–28] were selected and analyzed in deep, and three main communication parameters emerged [29]:


Any of this parameter has to be communicated through a sensorial language (i.e., through the products material features), to make the product itself a media for conveying information about the user's consumption.

#### *5.1.2. Concept description*

Glass of water is a tentative of applying the three parameters coming from the literature review (i.e., the first and the third one). It consists of a set of little spheres suitable for all faucets of the house. The spheres aim at reminding the user that the quantity of water at their disposal is not unlimited. To this end, they are conceived as small meters able to give information about the water consumption of the connected faucet. By means of a blue light, they tell users the correct amount of water they should use for their domestic activities (according to Gleick [30], around 50 L per person per day).

Any time the user turns the faucet on, the connected sphere gradually lose its brightness to show that the amount of water suitable for the domestic activities is decreasing until ending, when all the lights of all the spheres turn off. These changes are intended to recall the idea of a glass of water getting empty (**Figure 3**).

Designing Dialogs between Users and Products through a Sensory Language http://dx.doi.org/10.5772/intechopen.71127 155

**Figure 3.** Concept of glass of water.

consumption into a sensory language. The objective of this investigation was to find new ways to inspire designers in the concept generation phase. The result is a list of opportunities/limits for the application of the sensory language to convey certain kinds of information that are usually intended to be quantitative (e.g., average water/energy

Proceedings of the Conference on Design and Semantics of Form and Movement - Sense and Sensitivity, DeSForM

Even if the general focus was on the exploration of tangible aspects of products, in this phase, we enlarged the research also to the field of digital interfaces, persuasive technologies, and interaction design. Indeed, in these fields, several studies have already investigated the importance of giving feedbacks on energy consumption, and some of these studies tried to give suggestions to designers facing this matter. Only studies reporting results on users' investigation were considered, since our intent was to explore the user understanding of the contents of the dialog. As a result, 13 studies [5, 6, 17, 19–28] were selected and analyzed in deep, and three

• Metrics (related to the unit of measure). It has been observed that it is better to transmit qualitative information instead of quantitative ones (e.g., "Today your consumptions are

• Frequency, related to when and how often it is necessary to give information to users. It has been observed that users need continuous information instead of having information

• Representation, related to the "form" of the data. Some studies underlined the importance of visualizing information through metaphors or analogy (e.g., recalling the effects on the barrier reef of positive/negative consumption behaviors). Moreover, from these studies emerged the importance of giving positive information in order to support the motivation

Any of this parameter has to be communicated through a sensorial language (i.e., through the products material features), to make the product itself a media for conveying information

Glass of water is a tentative of applying the three parameters coming from the literature review (i.e., the first and the third one). It consists of a set of little spheres suitable for all faucets of the house. The spheres aim at reminding the user that the quantity of water at their disposal is not unlimited. To this end, they are conceived as small meters able to give information about the water consumption of the connected faucet. By means of a blue light, they tell users the correct amount of water they should use for their domestic activities (according to

Any time the user turns the faucet on, the connected sphere gradually lose its brightness to show that the amount of water suitable for the domestic activities is decreasing until ending, when all the lights of all the spheres turn off. These changes are intended to recall the idea of

good," instead of "today, you have consumed 120 KW of electricity.")

consumption per day/month/year).

2017

154

only when an event occurs.

about the user's consumption.

Gleick [30], around 50 L per person per day).

a glass of water getting empty (**Figure 3**).

*5.1.2. Concept description*

of users.

main communication parameters emerged [29]:

A prototype was developed, based on an Arduino board that controls the brightness of the sphere's light (**Figure 4**).

When the faucet is not used, the light is off (**Figure 5**); as soon as the water is open, the light gives (qualitative) information about the amount of water that can be still consumed. When the consumption of water exceeds 50 L per day, the light does not react anymore until the following day.

This raw prototype aims to be representative of the language. Thus, its functionality is simplified.

#### **5.2. Feelings and experiences for an embodied learning (F.E.E.L.)**

The second design activity was focused on the *empirical language,* based on the idea that meanings are created by the user during the interaction with the product.

During the development of the concept, we retained useful to apply the principles of embodied interaction. According to embodied interaction theories, there is a circle of influences among the physical properties of the products, the user's actions on/with the product, and the

**Figure 4.** Glass of water prototype.

**Figure 5.** Glass of water storyboard.

creation of meanings in the user's mind [9]. In phenomenological terms, material properties of the world are taken up as elements sustaining stabilities in action-perception loops, which govern a persons' skilled, routine-like dealing with the world [14, 31, 32]. The sense-making process is not just based on decoding information conveyed in the products features, but it is made possible by the creation of a personal dialog between the user and the product itself. Such dialog encourages reflection-on-action and the creation of new meanings.

Due to the complexity of the process, it was decided to focus on the water consumed into the shower, and it was decided to concentrate on the tactile stimuli provided by the shower tray. The aim of this second design activity, thus, was to create meaningful tactile experiences provided by an interactive shower tray able to affect the users' behaviors, influencing them "in the situation."

#### *5.2.1. Preliminary research*

With the aim of exploring the users' behaviors and feelings when taking a shower, users' observations were performed. Particularly, three short tests with users were organized in order to gain some insights into: (i) users' behaviors, (ii) tactile feelings, and (iii) users' emotional experiences. Video recording was the tool for exploring the users' feet behavior in the shower and the tactile interaction with the shower tray.

Moreover, testers were asked to fill a 1-week diary in order to keep track of their experiences and feelings during the shower. Both videos and diaries gave us insights into the context of use through pictures and words [33]. At the end of this phase, we collected four videos and four diaries.

#### *5.2.2. Concept description*

Thanks to the preliminary research on users' habits, useful insights were gained referring to: (i) the shape of the shower tray, (ii) the materials' selection, and (iii) the design of new behaviors and scenarios. These outcomes were useful for the development of the concept feelings and experiences for an embodied learning (F.E.E.L.). F.E.E.L. is a squared shower tray able to change its shape in order to create a new routine into the shower. It is composed by an external case and a number of soft "pins." These pins pop up randomly according to different rhythms, creating each time a novel tactile experience as a sort of feet massage (**Figure 6**).

The data collected by the 1-week diary show that users take a shower with two different aims:

• as a (short) refresh activity

creation of meanings in the user's mind [9]. In phenomenological terms, material properties of the world are taken up as elements sustaining stabilities in action-perception loops, which govern a persons' skilled, routine-like dealing with the world [14, 31, 32]. The sense-making process is not just based on decoding information conveyed in the products features, but it is made possible by the creation of a personal dialog between the user and the product itself.

Proceedings of the Conference on Design and Semantics of Form and Movement - Sense and Sensitivity, DeSForM

Due to the complexity of the process, it was decided to focus on the water consumed into the shower, and it was decided to concentrate on the tactile stimuli provided by the shower tray. The aim of this second design activity, thus, was to create meaningful tactile experiences provided by an interactive shower tray able to affect the users' behaviors, influencing them

With the aim of exploring the users' behaviors and feelings when taking a shower, users' observations were performed. Particularly, three short tests with users were organized in order to gain some insights into: (i) users' behaviors, (ii) tactile feelings, and (iii) users' emotional experiences. Video recording was the tool for exploring the users' feet behavior in the

Moreover, testers were asked to fill a 1-week diary in order to keep track of their experiences and feelings during the shower. Both videos and diaries gave us insights into the context of use through pictures and words [33]. At the end of this phase, we collected four videos and

Such dialog encourages reflection-on-action and the creation of new meanings.

"in the situation."

four diaries.

*5.2.1. Preliminary research*

**Figure 5.** Glass of water storyboard.

2017

156

shower and the tactile interaction with the shower tray.

• to relax and pamper themselves

According to these two scenarios, F.E.E.L. is designed to change its shape in a fast and more marked way in the initial minutes of the shower, for the average amount of time that users usually spend for a short shower. Then, it decreases the speed of its movements over time until stopping at the achievement of the maximum average time usually spend under the shower, that is, the "natural" moment at which people experience that they are now finished and it is time to get out. Over time, F.E.E.L. creates a dynamic coupling between the user's action and the temporal choreography played out in the responsive shower floor. Once this coupling is in place, in order to lead users to decrease the time spent under the shower, the system will over time *decrease* the durations of the temporal pattern. The size of the decrease would be small and not consciously noticeable. Indeed, it is important that the user feels as he/she has always the same routine during the shower. As a result, the user will after several weeks reach the "natural" feeling of "being finished" minutes earlier than before, thereby saving water, without feeling forced or without feeling as if one needs to make a conscious, ethical decision to "do the right thing."

We created a raw prototype in wood, steel, and soft materials. It aims to be representative of the language, and its functionality is simplified (**Figure 7**).

**Figure 6.** F.E.E.L. layout.

**Figure 7.** F.E.E.L. prototype.

### **6. Focus groups**

Three focus groups with users were organized in order to collect qualitative feedbacks about the two concepts. It is important to stress that they were not intended as traditional userevaluation focus groups, asking users whether the products would actually reduce water consumption. Rather, they were intended to extract some insights from the user's own sense making of our designs in order to answer our theoretical questions concerning the potentialities of applying a sensory language in designed interactions and the difference between codified and empirical languages. To this end, the first focus group presented to users "glass of water," that is, an example of codified language. The second focus group presented F.E.E.L that applies an empirical language. In a third focus group, both the concepts were presented in order to let users compare the two kinds of languages.

#### **6.1. Organizations**

Each focus group involved four people (two men and two women, aged 25–36). Our choice was to involve people with different backgrounds. None of the tester was a designer, since the aim of this activity was to gain feedbacks from nonexpert people. Moreover, we decided to involve people who lives alone and who already manage his/her consumptions.

All of them had the same organization: they started with a brief presentation aimed to introduce the concept/s to be discussed. During the session, videos and pictures were shown in order to describe the concepts in the real context. At the end of the presentation, participants were invited to discuss together. The discussion was guided by five open questions aimed to investigate the following aspects: (i) the engagement that they perceived with the concept, (ii) potentialities and limits of the concept, and (iii) how the concept could fit in their daily life.

#### **6.2. Results**

As a result, testers focused on two main aspects of the proposed concept: (i) the novelty factor and (ii) the clarity of the communicative intent. Both the projects were evaluated as novel, original, and unusual (compared to the objects that testers have in their home). However, participants evaluate the project F.E.E.L. as more innovative: "It proposes an unusual experience"; "This project is more cool!"; and "I have never seen something like that!"

Glass of water was perceived as less surprising and more similar to digital interfaces. During the focus groups, it was observed that the novelty factor influences the enthusiasm of the users to have a trial with the prototype in the real context.

Comparing the two projects, it emerged that the communicative intent of glass of water is more evident and immediate to users. Meanwhile, the communicative intent of F.E.E.L. required extra explanations by the facilitator. However, the concept F.E.E.L. was perceived as more able to support and motivate the changing of the users' behaviors, since the learning process was judged as subtle and more linked to personal awareness.

On the contrary, the learning process provided by glass of water was judged as cognitive and intentional: "Glass of water is telling me how much sustainable I am. In a way, it is pushing me to turn off the faucet to save water. Meanwhile, F.E.E.L. helps me to relax and enjoy my shower. Water consumptions are up to me."

### **7. Discussion**

**6. Focus groups**

**Figure 7.** F.E.E.L. prototype.

2017

158

**6.1. Organizations**

**6.2. Results**

Three focus groups with users were organized in order to collect qualitative feedbacks about the two concepts. It is important to stress that they were not intended as traditional userevaluation focus groups, asking users whether the products would actually reduce water consumption. Rather, they were intended to extract some insights from the user's own sense making of our designs in order to answer our theoretical questions concerning the potentialities of applying a sensory language in designed interactions and the difference between codified and empirical languages. To this end, the first focus group presented to users "glass of water," that is, an example of codified language. The second focus group presented F.E.E.L that applies an empirical language. In a third focus group, both the concepts were presented

Proceedings of the Conference on Design and Semantics of Form and Movement - Sense and Sensitivity, DeSForM

Each focus group involved four people (two men and two women, aged 25–36). Our choice was to involve people with different backgrounds. None of the tester was a designer, since the aim of this activity was to gain feedbacks from nonexpert people. Moreover, we decided to

All of them had the same organization: they started with a brief presentation aimed to introduce the concept/s to be discussed. During the session, videos and pictures were shown in order to describe the concepts in the real context. At the end of the presentation, participants were invited to discuss together. The discussion was guided by five open questions aimed to investigate the following aspects: (i) the engagement that they perceived with the concept, (ii) potentialities and limits of the concept, and (iii) how the concept could fit in their daily life.

As a result, testers focused on two main aspects of the proposed concept: (i) the novelty factor and (ii) the clarity of the communicative intent. Both the projects were evaluated as novel,

involve people who lives alone and who already manage his/her consumptions.

in order to let users compare the two kinds of languages.

Several studies on smart meters providing users with alphanumerical feedbacks on their consumptions have shown that they have a limited influence on users' behavior [4, 24, 34]. More detailed investigations show that the situation is complex: numerical feedbacks may fail either in taking into account the realities of users' life [20, 21, 26] or their understanding of units and quantities [35]. Moreover, they do not help users in having a wider understanding of their behaviors [36]. Another limit of information provided through a display is that they require users to look at them regularly, checking the consumption trends. To overcome such limits, studies carried out in different domains (interaction design, esthetics of interaction, ambient display, and visualization of data) have explored more sensorial ways (change in color, in light, in form) to give information about the amount of consumed resources.

Our results show that designing for a codified (sensory) language or for an empirical (sensory) language have different implications both for the design process and for the users' experience.

From the users' point of view, it was observed that establishing a material dialog through a codified code gives prominence to the message and to the communicative intent of the product (as underlined by the focus group discussion). Thus, the designer can apply a codified language when he/she wants the user to be conscious of the informative content, i.e., when the aim of the product is to give users' information on what would be a wise thing to do, to give them useful methods for achieving a certain goal, or to give them explicit reward for good behavior.

On the other side, applying an empirical sensory language means to focus on the embodied experience of interaction with the product [37]. This experience typically does not lead to a conscious reflection in the mind of the user about what to do. We strongly believe that the

empirical sensory language opens up the possibilities to design for behavioral change other than through passing on information that the user must process and make a conscious decision about. As the example of F.E.E.L. shows, we may be able to gradually change the experience of the user in such a way that the user finds himself wanting to get out of the shower. This tactic has been applied in similar fashion by Bruns et al. [35], who demonstrated how fiddling with a pen that gives haptic feedback would lead to a reduced level of stress *even when the user was not conscious of this.*

It is now important to briefly discuss what makes up for an empirical sensory language, and what would just be a natural consequence of interacting with the physical world. As a first idea, one might say that an empirical sensory language comprises anything sensed by a person that does not include explicit coding, but does help to generate a meaningful experience. This would, for example, include how the physical properties of a tennis racket in interaction with a student would help the student master the game of tennis through experiential learning. We do feel that our design examples are of a different kind. While the racket is indeed a designed artifact, it is not designed to generate a learning experience: it is designed to play tennis. The potential for experiential learning is in some sense a by-product that you get for free. More importantly, you would not be able to change such physical aspects of the product without changing the entire product.

On the other side, we consider what happens when you sit in a bath. The water slowly gets cold. This change in the baths' "behavior" leads to a temporal dynamic in sensory signals that may very well give rise to a quite similar experience as the one intended by the designed choreography of the shower tray. Processed largely unconsciously, the falling temperature will at some point make one think: I should get out now. But this is not the result of a designed experience. In contrast, in the shower floor, we were able to design the system in an adaptive way so as to first align the choreography with the natural routine of user and then gradually shorten the pattern in order to elicit an earlier onset of the experience of "being finished."

To conclude, in a empirical sensory language, rather than considering just the sensory signals coming from the physical properties of any part of the artifact more generally, there always remains to some degree an arbitrariness, or space for variation, for creating a coupling between the designed sensory signal that a person receives, and the meaning that will arise from it. This space for variation is precisely what gives designers the freedom they need to design the interaction. We say this space of variation exists "to some degree," because a complete freedom on part of the designer to determine the couplings between each sensory signal and the meaning it should generate in the user experience would of course turn the system into a codified system once more.

### **8. Conclusion**

With our research, we aimed to contribute to achieve a "crossfertilization" between features that traditionally apply to the alphanumeric language and the sensory language. Indeed, the sensory language can nowadays exploit new opportunities coming from the development of smart materials and new technologies.

In our research-through-design case study, we applied two different kinds of sensory language (a codified and an empirical one) to the design of two prototypes intended to lead users decrease the water consumption. In general, the results show it is possible to design for behavior change using a sensory language.

Entering into the differences between a codified and an empirical language, we have noted that establishing a material dialog through a codified code gives prominence to the message and to the communicative intent of the product. On the other side, an empirical sensory language addresses our embodied experience more directly, which opens up possibilities to design for behavior change in ways other than through explicit message passing or pure physical enforcing.

One limitation of this study is that this research does not explore the users' perception of sensory codified and empirical languages overtime. In this perspective, future investigations should be set in order to test the two languages capacity to affect the user behaviors over a prolonged period of time.

### **Author details**

empirical sensory language opens up the possibilities to design for behavioral change other than through passing on information that the user must process and make a conscious decision about. As the example of F.E.E.L. shows, we may be able to gradually change the experience of the user in such a way that the user finds himself wanting to get out of the shower. This tactic has been applied in similar fashion by Bruns et al. [35], who demonstrated how fiddling with a pen that gives haptic feedback would lead to a reduced level of stress *even* 

Proceedings of the Conference on Design and Semantics of Form and Movement - Sense and Sensitivity, DeSForM

It is now important to briefly discuss what makes up for an empirical sensory language, and what would just be a natural consequence of interacting with the physical world. As a first idea, one might say that an empirical sensory language comprises anything sensed by a person that does not include explicit coding, but does help to generate a meaningful experience. This would, for example, include how the physical properties of a tennis racket in interaction with a student would help the student master the game of tennis through experiential learning. We do feel that our design examples are of a different kind. While the racket is indeed a designed artifact, it is not designed to generate a learning experience: it is designed to play tennis. The potential for experiential learning is in some sense a by-product that you get for free. More importantly, you would not be able to change such physical aspects of the product

On the other side, we consider what happens when you sit in a bath. The water slowly gets cold. This change in the baths' "behavior" leads to a temporal dynamic in sensory signals that may very well give rise to a quite similar experience as the one intended by the designed choreography of the shower tray. Processed largely unconsciously, the falling temperature will at some point make one think: I should get out now. But this is not the result of a designed experience. In contrast, in the shower floor, we were able to design the system in an adaptive way so as to first align the choreography with the natural routine of user and then gradually shorten the pattern in order to elicit an earlier onset of the experience of "being finished."

To conclude, in a empirical sensory language, rather than considering just the sensory signals coming from the physical properties of any part of the artifact more generally, there always remains to some degree an arbitrariness, or space for variation, for creating a coupling between the designed sensory signal that a person receives, and the meaning that will arise from it. This space for variation is precisely what gives designers the freedom they need to design the interaction. We say this space of variation exists "to some degree," because a complete freedom on part of the designer to determine the couplings between each sensory signal and the meaning it should generate in the user experience would of course turn the system

With our research, we aimed to contribute to achieve a "crossfertilization" between features that traditionally apply to the alphanumeric language and the sensory language. Indeed, the sensory language can nowadays exploit new opportunities coming from the development of

*when the user was not conscious of this.*

2017

160

without changing the entire product.

into a codified system once more.

smart materials and new technologies.

**8. Conclusion**

Sara Bergamaschi<sup>1</sup> \*, Lucia Rampino1 and Jelle van Dijk2

\*Address all correspondence to: sara.bergamaschi@polimi.it

1 Politecnico di Milano, Italy

2 University of Twente, The Netherlands

### **References**

	- [7] Djajadiningrat T, Wensveen S, Frens J, Overbeeke K. Tangible products: Redressing the balance between appearance and action. Personal and Ubiquitous Computing. 2004;**8**(5):294-309
	- [8] Saffer D. Designing for Interaction: Creating Innovative Applications and Devices (Voices that Matter); 2009
	- [9] Author. Conference paper; 2015
	- [10] Redström J. Towards user design? On the shift from object to user as the subject of design. Design studies. 2006;**27**(2):123-139
	- [11] Moggridge B, Atkinson B. Designing Interactions. Vol. 14. Cambridge, MA: MIT press; 2007
	- [12] Krippendorff K. The Semantic Turn: A New Foundation for Design. Boca Raton: Taylor & Francis; 2005
	- [13] Philip LE. Merleau-ponty and the Phenomenology of Language. In: Yale French Studies. Vol. 36/37. Yale University Press. p. 19-40
	- [14] Merleau-Ponty M. Eye and mind. In: James E. editor. The Primacy of Perception, Northwestern UP. 1964 Carleton Dallery, Trans. editor, Evanston. pp. 159-190
	- [15] Merleau-Ponty M. Phénoménologie de la perception. (English translation. New York: Phenomenology of perception; 1962
	- [16] Socco C. Semiotica e progetto del paesaggio. Seminario: Organizzato; 1996
	- [17] Lachman R, Lachman J, Butterfield E. Cognitive Psychology and Information Processing. Hillsdale BJ. Lawrence Erlbau; 1979
	- [18] Durgee JF. How consumer sub-cultures code reality: A look at some code types. NA-Advances in Consumer Research Volume. 1986;13
	- [19] Bowden F, Lockton D, Brass C, Gheerawo R. Drawing Energy: Exploring the Esthetics of the Invisible. In: IAEA Congress 2014: Congress of the International Association of Empirical Esthetics; 2014
	- [20] Daae J, Boks CA. Classification of user research methods for design for sustainable behavior. Journal of Cleaner Production. 2015;**106**:680-689
	- [21] Fogg BJ. A behavior model for persuasive design. In: Proceedings of the 4th International Conference on Persuasive Technology. ACM; 2009, April p. 40
	- [22] Ham J, Midden C. Ambient persuasive technology needs little cognitive effort: The differential effects of cognitive load on lighting feedback versus factual feedback. In: International Conference on Persuasive Technology. Berlin Heidelberg: Springer; 2010, June. p. 132-142
	- [23] Jacucci G, Spagnolli A, Gamberini L, Chalambalakis A, Björkskog C, Bertoncini M, et al. Designing effective feedback of electricity consumption for mobile user interfaces. PsychNology Journal. 2009;**7**(3):265-289
	- [24] Kim T, Hong H, Magerko B. Coralog: Use-aware visualization connecting human microactivities to environmental change. In: CHI '09 Extended Abstracts on Human Factors in Computing Systems. ACM; 2009, April pp. 4303-4308

[25] Kuznetsov S, Paulos E. UpStream: Motivating water conservation with low-cost water flow sensing and persuasive displays. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. ACM; 2010, April pp. 1851-1860

[7] Djajadiningrat T, Wensveen S, Frens J, Overbeeke K. Tangible products: Redressing the balance between appearance and action. Personal and Ubiquitous Computing.

Proceedings of the Conference on Design and Semantics of Form and Movement - Sense and Sensitivity, DeSForM

[8] Saffer D. Designing for Interaction: Creating Innovative Applications and Devices (Voices

[10] Redström J. Towards user design? On the shift from object to user as the subject of

[11] Moggridge B, Atkinson B. Designing Interactions. Vol. 14. Cambridge, MA: MIT press;

[12] Krippendorff K. The Semantic Turn: A New Foundation for Design. Boca Raton: Taylor

[13] Philip LE. Merleau-ponty and the Phenomenology of Language. In: Yale French Studies.

[14] Merleau-Ponty M. Eye and mind. In: James E. editor. The Primacy of Perception, Northwestern UP. 1964 Carleton Dallery, Trans. editor, Evanston. pp. 159-190

[15] Merleau-Ponty M. Phénoménologie de la perception. (English translation. New York:

[17] Lachman R, Lachman J, Butterfield E. Cognitive Psychology and Information Processing.

[18] Durgee JF. How consumer sub-cultures code reality: A look at some code types.

[19] Bowden F, Lockton D, Brass C, Gheerawo R. Drawing Energy: Exploring the Esthetics of the Invisible. In: IAEA Congress 2014: Congress of the International Association of

[20] Daae J, Boks CA. Classification of user research methods for design for sustainable

[21] Fogg BJ. A behavior model for persuasive design. In: Proceedings of the 4th International

[22] Ham J, Midden C. Ambient persuasive technology needs little cognitive effort: The differential effects of cognitive load on lighting feedback versus factual feedback. In: International Conference on Persuasive Technology. Berlin Heidelberg: Springer; 2010,

[23] Jacucci G, Spagnolli A, Gamberini L, Chalambalakis A, Björkskog C, Bertoncini M, et al. Designing effective feedback of electricity consumption for mobile user interfaces.

[24] Kim T, Hong H, Magerko B. Coralog: Use-aware visualization connecting human microactivities to environmental change. In: CHI '09 Extended Abstracts on Human Factors in

[16] Socco C. Semiotica e progetto del paesaggio. Seminario: Organizzato; 1996

NA-Advances in Consumer Research Volume. 1986;13

behavior. Journal of Cleaner Production. 2015;**106**:680-689

Conference on Persuasive Technology. ACM; 2009, April p. 40

2004;**8**(5):294-309

that Matter); 2009

& Francis; 2005

2007

2017

162

[9] Author. Conference paper; 2015

design. Design studies. 2006;**27**(2):123-139

Vol. 36/37. Yale University Press. p. 19-40

Phenomenology of perception; 1962

Hillsdale BJ. Lawrence Erlbau; 1979

PsychNology Journal. 2009;**7**(3):265-289

Computing Systems. ACM; 2009, April pp. 4303-4308

Empirical Esthetics; 2014

June. p. 132-142


Provisional chapter

### **Designing the Expressiveness of Point Lights for Bridging Human-IoT System Communications** Designing the Expressiveness of Point Lights for Bridging Human-IoT System Communications

DOI: 10.5772/intechopen.71131

Yoga Liu, Rung-Huei Liang, Ya-Han Lee, Yaliang Chuang and Lin-Lin Chen Yoga Liu, Rung-Huei Liang, Ya-Han Lee,

Additional information is available at the end of the chapter Yaliang Chuang and Lin-Lin Chen

http://dx.doi.org/10.5772/intechopen.71131 Additional information is available at the end of the chapter

#### Abstract

In the last decade, the digital devices and intelligent systems are becoming popular in people's everyday lives. Many machines could communicate and collaborate as a system to provide various services. However, they seldom provide sufficient feed forwards or feedbacks to help users understand current states and aware what it is about to act. In this study, we explored the possibilities of expressiveness with the point lights embedded on ubiquitous devices. By applying the findings from related works and animation principles, we created nine basic individual patterns and composed 12 designs of group behaviors. We then conducted a survey with 69 participants to rate their expressiveness regarding nine vocabularies of the human-system communication. The results show that single light behavior and the performative group light behavior could help to convey specific state information for intuitive communication. At the individual light level, for instance, the fade in light behavior can indicate changes in status; rapid blinking can indicate important information. At the group level, the two designs developed in this study, leading and simultaneous type also can initiate human interactions and represent the machine-to-machine conversations taken place in the system, respectively.

Keywords: feedback, communication, understanding, vocabulary, LED lights, Internet of things (IoT)

#### 1. Introduction

In the recent decade, the connected products and services are emerging and becoming more and more popular. It is estimated that by 2020, everyone would have more than 20 devices worn on the body or used in the environments [1]. When the systems run well, they can provide users smooth experiences. However, when something unexpected happens, users usually feel frustrated because they do not know how the problem occurred [2, 3]. Even when

© 2017 The Author(s). Licensee InTech. This chapter is distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/3.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

© The Author(s). Licensee InTech. This chapter is distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/3.0), which permits unrestricted use, distribution, and eproduction in any medium, provided the original work is properly cited.

everything runs smoothly, feedbacks are also essential to reassure that this is the case. To increase users' awareness, the small light emitting diodes (LEDs) were embedded in most devices for providing information and feedbacks. However, the lighting behaviors are either ambiguous or unintuitive for understanding [4]. As a result, users are unable to obtain useful feedbacks to create the appropriate conceptual model and do not know what actions they could take.

Recently, several researchers have investigated the design of good feedbacks with lights or sounds. However, most of the studies were focusing on the interactions with a single device, for example in [4, 5], how to assist the communications between a user and system is seldom investigated. Our research aims to explore the expressiveness of light behaviors that could enhance users' awareness and understanding on the systems of smart things in the smart environments. In the study presented in this manuscript, we reported the preliminary results that focused on the expressiveness of the individual and group behaviors through the point light embedded on simple devices. Through a user survey with 69 participants, it was found that the combination of light behaviors could sufficiently convey eight of the nine vocabularies investigated in this study. The contribution of this study is a set of light pattern that could be used to provide informative clues of the Internet of things (IoT) system and facilitate the usersystem interactions.

### 2. Related works

In an interactive system, visible lights are commonly treated as versatile mediums to grab user's attentions or provide feedbacks on their interactions. For instance, [6] installed ring lights to a flying drone and designed the light behaviors to let users understand its intentions. Although the usage of LED is a common paradigm in designing electronic devices, Harrison et al. [4] found that the expressiveness of light behaviors was either limited or ambiguous. By using the smartphone as an example, they proposed eight light designs that might provide strong and iconic indications of a particular informational state. In addition, Pintus [5] created six light behaviors to express the status that two devices were connected for transmitting the data to each other. Extending from those studies, we aimed to investigate the design of light behaviors to create sufficient feedbacks of a system with multiple simple devices.

Because the point lights are both sequential and nonpersistent [7], it needs thoughtful designs in composing multiple lights to deliver coherent messages. It is similar to the directing of actors' movements in a movie or animation. Among the well-known 12 principles of animation [8], we thought the staging and timing could be two proper strategies in designing the system feedbacks. It could direct the user's attentions facilitate him/her to perceive the smart things' states, functions, and reactions.

Finally, Deckers et al. [9] applied the concept of perceptual crossing to show the acknowledgment of a smart thing to a user's approach. With perceptual crossing, he/she not only can recognize the possibility to initiate interactions with the system but also is invited to engage in a more continuous way with something akin to an artificial living creature. In this study, the analogy of "I see you seeing me" in their study was chosen as the semantic term for evaluating the alternative light behaviors developed.

### 3. Methodology

everything runs smoothly, feedbacks are also essential to reassure that this is the case. To increase users' awareness, the small light emitting diodes (LEDs) were embedded in most devices for providing information and feedbacks. However, the lighting behaviors are either ambiguous or unintuitive for understanding [4]. As a result, users are unable to obtain useful feedbacks to create the appropriate conceptual model and do not know what actions they

Proceedings of the Conference on Design and Semantics of Form and Movement - Sense and Sensitivity, DeSForM

Recently, several researchers have investigated the design of good feedbacks with lights or sounds. However, most of the studies were focusing on the interactions with a single device, for example in [4, 5], how to assist the communications between a user and system is seldom investigated. Our research aims to explore the expressiveness of light behaviors that could enhance users' awareness and understanding on the systems of smart things in the smart environments. In the study presented in this manuscript, we reported the preliminary results that focused on the expressiveness of the individual and group behaviors through the point light embedded on simple devices. Through a user survey with 69 participants, it was found that the combination of light behaviors could sufficiently convey eight of the nine vocabularies investigated in this study. The contribution of this study is a set of light pattern that could be used to provide informative clues of the Internet of things (IoT) system and facilitate the user-

In an interactive system, visible lights are commonly treated as versatile mediums to grab user's attentions or provide feedbacks on their interactions. For instance, [6] installed ring lights to a flying drone and designed the light behaviors to let users understand its intentions. Although the usage of LED is a common paradigm in designing electronic devices, Harrison et al. [4] found that the expressiveness of light behaviors was either limited or ambiguous. By using the smartphone as an example, they proposed eight light designs that might provide strong and iconic indications of a particular informational state. In addition, Pintus [5] created six light behaviors to express the status that two devices were connected for transmitting the data to each other. Extending from those studies, we aimed to investigate the design of light behaviors to create sufficient feedbacks of a system with multiple

Because the point lights are both sequential and nonpersistent [7], it needs thoughtful designs in composing multiple lights to deliver coherent messages. It is similar to the directing of actors' movements in a movie or animation. Among the well-known 12 principles of animation [8], we thought the staging and timing could be two proper strategies in designing the system feedbacks. It could direct the user's attentions facilitate him/her to perceive the smart things'

Finally, Deckers et al. [9] applied the concept of perceptual crossing to show the acknowledgment of a smart thing to a user's approach. With perceptual crossing, he/she not only can recognize the possibility to initiate interactions with the system but also is invited to engage in

could take.

2017

166

system interactions.

2. Related works

simple devices.

states, functions, and reactions.

To understand possible communications that will be needed between human and intelligent systems, in a previous study [10], we chose and analyze nine concept videos produced by leading companies, research institutes, and independent designers. Through the semantic analysis of the interactions between the user and a system of smart things demonstrated in the films, we extracted 43 design vocabularies and clustered them into 11 categories. Among those vocabularies, some of them were extracted from complex interactions, such as the emotional or social reactions with robots or anthropomorphic devices. For exploring the possible expressions of the simple devices with point lights, we reviewed the video clips and selected nine vocabularies that were essential for creating users' awareness of the system of smart things. This set (shown in Table 1) covers the basic communications between a user and a system of simple smart devices.


Table 1. The vocabularies selected for investigation in the study presented in this manuscript.

### 4. Designing the individual and group light behaviors

We first explored the different behaviors of single point light based on [4] and created nine basic patterns (shown in Figure 1) with variations in the changes of intensity and velocity. Those designs include the two lights that gradually fade in or out, two static designs (on/off) that maintain the intensity as 100% or zero, one breathing light, two designs (blink twice/ thrice) that embedded blinking two or three times, one random brightness, and one (dark flash) that quickly blinks once when it stays as the brightest state.

For the group behaviors, we utilized the animation principles of staging and timing [8] to create six types of arrangements for a system with five devices. In the first three types, all the devices had the same individual behaviors but would light up with different time settings,

Figure 1. The nine behaviors of point light.

including simultaneous, sequential, or random. In the other two types, there was a device dressed with different light behavior to grab users' attentions on it, such as leading and emphasizing type. Finally, there is a counting off type that simulates the machine-to-machine conversations by aligning the pairing responses in the same time or with a short delay in time. Through compositing the individual light behaviors with those group patterns, we generated 12 designs with the video editing software. The illustration of the light behaviors was shown in Figure 2.

Figure 2. The 12 lighting patterns for group communications. The settings in brackets refer to the individual point light behavior on the device.

### 5. Experiment

including simultaneous, sequential, or random. In the other two types, there was a device dressed with different light behavior to grab users' attentions on it, such as leading and emphasizing type. Finally, there is a counting off type that simulates the machine-to-machine conversations by aligning the pairing responses in the same time or with a short delay in time. Through compositing the individual light behaviors with those group patterns, we generated 12 designs with the video editing software. The illustration of the light behaviors

Proceedings of the Conference on Design and Semantics of Form and Movement - Sense and Sensitivity, DeSForM

Figure 2. The 12 lighting patterns for group communications. The settings in brackets refer to the individual point light

was shown in Figure 2.

behavior on the device.

Figure 1. The nine behaviors of point light.

2017

168

#### 5.1. Setting and procedure

An interactive survey was developed for evaluating the expressiveness of light behaviors in random order. In each round, a participant would see one of the 12 video clips (that would automatically replay) and rate the degree of their perceptions regarding the nine vocabularies shown in Table 1. A five-point Likert scale was provided next to each of the scale (ranging


Note: more \* indicates the mean is larger; the settings in brackets refer to the individual point light behavior on the device; #, in the repeated measures ANOVA analysis, the data of "Detecting Problems" vocabulary is the only one that the light behaviors were not significant (Mauchly's Test of Sphericity, df = 65, p = 0.175); and the data of the other eight vocabularies are all significant (p < 0.05).

Table 2. The top three light behaviors the participants thought are relevant to the specific vocabulary.

from 1, strongly disagree, up to 5, strongly agree). There were 69 participants (52% male and 48% female, 88% aged between 21 and 30) recruited and completed the survey. A total of 70% participants have expertise related to interaction design.

#### 5.2. Results

The statistical analysis shows that for eight of the nine vocabularies (excepting "Detecting problems"), there were several designs of the group light behaviors perceived with higher correlations to particular terms than the others (repeated measures ANOVA, p < 0.05). Table 2 provides an overview of the matching. Together with the factor analysis, it was found that the nine vocabularies could be clustered into four groups (with 93.91% of variance explained). Regarding the first active factor, the sequential (fade in), simultaneous (fade in), and leading (blink twice, fade in) light behaviors are the three ideal designs that can express the system's working state. For the second interaction factor, the sequential (blink twice) and leading (dark flash) are perceived to convey the machine-to-machine communications and human-system interactions, respectively. In addition, because the breathing light pattern was widely used in many electronic devices, the participants could easily perceive the system in sleeping or standby status with the pattern. Finally, for the alerting vocabulary, the most correlated designs are leading (blink thrice) and emphasizing (blink thrice). Similar to the first active factor, the individual light pattern articulates to alter or notify the users an important message. Besides, the group behavior provides additional information to help him/her aware of the different levels of the communication. Based on those findings, currently, we are implementing those light behaviors with tangible prototypes. We plan to conduct contextual experience experiments to investigate how will people interpret the light behaviors in the physical world. This will help us to understand how to design intuitive feedbacks to bridge the human-IoT system communications and create natural and seamless interactions.

### 6. Limitations

In this study, we mainly focused on the design of light behaviors and their semantic expressions. Therefore, we chose the video simulations to explore alternative design ideas. This helped us to investigate the subtle changes of single point light and the compositions of multiple lights. In our survey, the text descriptions were presented to help users imagine the context and rate the degree of correlations between the designs and vocabularies. The preliminary results could provide designers an initial set of individual and group light behaviors for conveying specific feedbacks.

### 7. Conclusion

As the IoT technology is maturing, many designers are focusing on creating smart products to improve customers' living experiences. There have been many devices installed and used in many people's daily lives. However, due to the lack of communications, users often encountered frustrations to the system's unexpected behaviors. This study is aimed to explore how to design the expressiveness of point light to improve users' situated awareness to the smart things in the surrounding space. Based on related studies of the point light design and the animation principles [8] of staging and timing, we composed 24 group light behaviors for a local system that consists of five virtual devices. After evaluating the esthetic quality and the consistency of message conveyed, we chose 12 designs and conducted a survey with 69 participants. The results show that the point light behaviors and the composition of grouping performance could create implicit and sufficient communications. As a result, we recommend a collection of design strategies. With this new knowledge, designers could successfully provide useful clues for users to have the natural and seamless experience while interacting with a system of smart things in a smart environment.

### Acknowledgements

from 1, strongly disagree, up to 5, strongly agree). There were 69 participants (52% male and 48% female, 88% aged between 21 and 30) recruited and completed the survey. A total of 70%

Proceedings of the Conference on Design and Semantics of Form and Movement - Sense and Sensitivity, DeSForM

The statistical analysis shows that for eight of the nine vocabularies (excepting "Detecting problems"), there were several designs of the group light behaviors perceived with higher correlations to particular terms than the others (repeated measures ANOVA, p < 0.05). Table 2 provides an overview of the matching. Together with the factor analysis, it was found that the nine vocabularies could be clustered into four groups (with 93.91% of variance explained). Regarding the first active factor, the sequential (fade in), simultaneous (fade in), and leading (blink twice, fade in) light behaviors are the three ideal designs that can express the system's working state. For the second interaction factor, the sequential (blink twice) and leading (dark flash) are perceived to convey the machine-to-machine communications and human-system interactions, respectively. In addition, because the breathing light pattern was widely used in many electronic devices, the participants could easily perceive the system in sleeping or standby status with the pattern. Finally, for the alerting vocabulary, the most correlated designs are leading (blink thrice) and emphasizing (blink thrice). Similar to the first active factor, the individual light pattern articulates to alter or notify the users an important message. Besides, the group behavior provides additional information to help him/her aware of the different levels of the communication. Based on those findings, currently, we are implementing those light behaviors with tangible prototypes. We plan to conduct contextual experience experiments to investigate how will people interpret the light behaviors in the physical world. This will help us to understand how to design intuitive feedbacks to bridge the human-IoT system communica-

In this study, we mainly focused on the design of light behaviors and their semantic expressions. Therefore, we chose the video simulations to explore alternative design ideas. This helped us to investigate the subtle changes of single point light and the compositions of multiple lights. In our survey, the text descriptions were presented to help users imagine the context and rate the degree of correlations between the designs and vocabularies. The preliminary results could provide designers an initial set of individual and group light behaviors for

As the IoT technology is maturing, many designers are focusing on creating smart products to improve customers' living experiences. There have been many devices installed and used in many people's daily lives. However, due to the lack of communications, users often

participants have expertise related to interaction design.

tions and create natural and seamless interactions.

5.2. Results

2017

170

6. Limitations

7. Conclusion

conveying specific feedbacks.

This research was supported in part by the Ministry of Science and Technology of Taiwan (MOST 106-2633-E-002-001), National Taiwan University (NTU-106R104045), and Intel Corporation.
