**Sensory Augmentation through Tissue Conduction**

DOI: 10.5772/intechopen.71126

Sensory Augmentation through Tissue Conduction

Peter Lennox and Ian McKenzie

Additional information is available at the end of the chapter Peter Lennox and Ian McKenzie

http://dx.doi.org/10.5772/intechopen.71126 Additional information is available at the end of the chapter

#### Abstract

One hundred volunteers have undergone short (5 min) listening tests in a novel multitransducer bone-and-tissue conduction apparatus for spatial audio. The subjects subsequently described their experiences in an unstructured qualitative elicitation exercise. Their responses were aggregated to identify key themes and differences. Emergent themes are: enjoyable, informative, spatial and strange. Tactile supplementation of spatial audio display was noted in a positive light. We note that some spatial attributes are more perceptible than others. The implications for perceptual augmentation are discussed, particularly in relation to conductive hearing deficits. We conclude that the technique has potential for development and discusses future research directions.

Keywords: bone conduction, tissue conduction, multimodal perception, spatial audio, augmented perception, vibrotactile

### 1. Introduction

Hearing impairment is a sensory deprivation that constrains the information bandwidth available to the individual. Consequences include poor speech discernment (especially in environments with high background noise), poor auditory spatial performance and lack of pleasurable access to music. Hearing impairment can be due to sensorineural or conductive inadequacies, or both. Amelioration strategies include assistive technologies to augment individuals' residual sensory capacities (for example: hearing aids) or to substitute alternative information pathways where one stage of auditory processing is defunct (for example: cochlea implants). Performance is generally better for communication problems than for spatial and pleasurablelistening problems. There is some evidence to indicate that age-related hearing deficits may play some causal role in the onset and progression of dementia, in part due to social disengagement because of increasing difficulty in disambiguating complex auditory scenes, and in a feedback effect, because neural pathways that receive little stimulus become less efficient [1].

© 2017 The Author(s). Licensee InTech. This chapter is distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/3.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

© The Author(s). Licensee InTech. This chapter is distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/3.0), which permits unrestricted use, distribution, and eproduction in any medium, provided the original work is properly cited.

Given that prevalence of hearing loss doubles with each age-decade [2], hearing rehabilitation techniques may be expected to become increasingly important as the average life-expectancy increases. There is a quality-of-life (QoL) issue here; as the auditory information-channel gradually falls into disuse, access to entertainment and intellectual stimulus, in the forms of conversation, music listening and television, becomes scarce.

There is a general trend towards heightened spatial competence in artificial audio, offering increased involvement and informativeness. We are investigating whether bone and tissue conduction techniques can be developed to provide increased enjoyment and informativeness through extended spatial impressions. We have developed a prototype 5-transducer vibrotactile tissue conduction system to display multichannel spatial sound recordings; our objective is to identify, and subsequently parameterise available qualia in this context. One hundred short listening demonstrations were conducted and responses were aggregated and examined for frequency of occurrence of adjectives and synonyms; these formed the basis of an initial set of themes. The cohort was then re-analysed to identify co-occurring themes.

### 2. Tissue conduction of sound

Techniques for utilising vibration to produce auditory percepts have been known of for centuries; 16th century Girolamo Capivaccio struck an iron rod held against the teeth to assess ear pathology; Ludwig van Beethoven used a wooden rod, with one end held between his teeth and the other resting on the piano he was able to continue his work even though considered profoundly deaf. Late 18th century saw the development of early bone conduction devices, the Fonifero in 1876 by Giovanni Paledino, and the Audiphone in 1879 by Richard Rhodes used mechanical transduction of sound to assist hearing.

These are putatively termed 'bone conduction' techniques, though this is a slight misnomer, as skin, fluid and the soft-tissue contents of the cranium also contribute transmission pathways to various extents. We prefer the term 'tissue conduction' (TC) as a more comprehensive description. There is general agreement that multiple transmission pathways are in use. Several sources record the pathways to be frequency dependant; inertial forces acting on the ossicular structure and cochlear fluids at low frequencies relative to skull vibration; high frequencies causing distortion of the temporal bone and cochlear shell. Sound generated in the occluded ear canal via Osseo-tympanic transmission provides increased sensitivity at low frequencies; contents of the skull and fluid pathways induce sensitivity at high frequencies [3–6]. The prominence of each pathways contribution remain in question; however, the resultant wave motion in the basilar membrane as a summed contribution of all pathways appears to be the same as for air-conducted sound; cancellation experiments between AC, BC and TC show this to be the case [3, 7, 8].

Cochlear stimulation through TC is commonly elicited using a vibrotactile transducer in contact with the skull; various body locations have also been shown effective [6]. Monaural and binaural presentation in contact with the mastoid, condyle or forehead have provided common TC stimulation sites; many additional locations on the skull have featured in research, all using singular or dual transducer presentation [3, 5, 9]. Until recently, TC conveyed audio signal but not spatial information, and so the experience was not equivalent (in this respect) to real-world hearing. Latterly, researchers have shown that a considerable of degree of lateralisation, in some case approaching that of normal binaural hearing, is feasible [10–12]. Nevertheless, the results lack equivalence in terms of overall spatial performance, significantly lacking spatial attributes such as externalisation, spaciousness, range perception and elevation perception.

### 3. Multi-transducer listening tests

#### 3.1. Equipment

Given that prevalence of hearing loss doubles with each age-decade [2], hearing rehabilitation techniques may be expected to become increasingly important as the average life-expectancy increases. There is a quality-of-life (QoL) issue here; as the auditory information-channel gradually falls into disuse, access to entertainment and intellectual stimulus, in the forms of

Proceedings of the Conference on Design and Semantics of Form and Movement - Sense and Sensitivity, DeSForM

There is a general trend towards heightened spatial competence in artificial audio, offering increased involvement and informativeness. We are investigating whether bone and tissue conduction techniques can be developed to provide increased enjoyment and informativeness through extended spatial impressions. We have developed a prototype 5-transducer vibrotactile tissue conduction system to display multichannel spatial sound recordings; our objective is to identify, and subsequently parameterise available qualia in this context. One hundred short listening demonstrations were conducted and responses were aggregated and examined for frequency of occurrence of adjectives and synonyms; these formed the basis of an

initial set of themes. The cohort was then re-analysed to identify co-occurring themes.

Techniques for utilising vibration to produce auditory percepts have been known of for centuries; 16th century Girolamo Capivaccio struck an iron rod held against the teeth to assess ear pathology; Ludwig van Beethoven used a wooden rod, with one end held between his teeth and the other resting on the piano he was able to continue his work even though considered profoundly deaf. Late 18th century saw the development of early bone conduction devices, the Fonifero in 1876 by Giovanni Paledino, and the Audiphone in 1879 by Richard

These are putatively termed 'bone conduction' techniques, though this is a slight misnomer, as skin, fluid and the soft-tissue contents of the cranium also contribute transmission pathways to various extents. We prefer the term 'tissue conduction' (TC) as a more comprehensive description. There is general agreement that multiple transmission pathways are in use. Several sources record the pathways to be frequency dependant; inertial forces acting on the ossicular structure and cochlear fluids at low frequencies relative to skull vibration; high frequencies causing distortion of the temporal bone and cochlear shell. Sound generated in the occluded ear canal via Osseo-tympanic transmission provides increased sensitivity at low frequencies; contents of the skull and fluid pathways induce sensitivity at high frequencies [3–6]. The prominence of each pathways contribution remain in question; however, the resultant wave motion in the basilar membrane as a summed contribution of all pathways appears to be the same as for air-conducted sound; cancellation experiments between AC, BC and TC show this

Cochlear stimulation through TC is commonly elicited using a vibrotactile transducer in contact with the skull; various body locations have also been shown effective [6]. Monaural and binaural presentation in contact with the mastoid, condyle or forehead have provided common TC stimulation sites; many additional locations on the skull have featured in

conversation, music listening and television, becomes scarce.

Rhodes used mechanical transduction of sound to assist hearing.

2. Tissue conduction of sound

2017

198

to be the case [3, 7, 8].

The prototype array uses five BCT-1 8 Ω 90 dB 1 W/1 m tactile transducers held in a tensioned framework exerting contact force with skull through a hemi-spherical plastic medium on each transducer. For reference, the transducer locations are numbered left to right: 1—left mastoid, 2—left temporal region above the zygomatic arch, 3—forehead, 4—right temporal region above the zygomatic arch, and 5—right mastoid. Principal design considerations were that of transducer location, contact force and surface area that would work with considerable variations in head size and shape. Signal sets were processed using Reaper DAW on mac, interfaced through a Focusrite PRO 26i/o and sent discretely to each transducer through individual 1 W amplifiers; the array has a frequency range of 200 Hz–16 kHz. A set of banded style 3M Ear Plugs were available for listeners to use and compare the experience with the plugs in vs. out (Figure 1).

Figure 1. Prototype 5 transducer array, amplifiers, interface and DAW.

#### 3.2. Signals

Signals were processed using Reaper DAW and spatially encoded using WigWare 1st order ambisonic panning; FX Plugins were used to construct early and late reflections and then decoded through a WigWare 1st order periphonic ambisonic decoder patched to the transducer array.

A 1st order ambisonic recording of a country park captured using a Soundfield™ microphone provided the ambient background; stereo recordings of bird sounds, voices, a steam train and music alongside mono FX clips were used to create the soundscape in which 1st order ambisonic recordings of a motorbike and aeroplane were placed.

#### 3.3. Method

In this study, we used 100 naïve (i.e. inexperienced in TC listening) and untutored (subjects receive no instructions on target attributes) listeners, who were then invited to offer observations and comment on the experience; of the 100 listeners non-reported previous experience of tissue conducted sound. 24 female and 76 male participants took part; each was asked to record their age, sex, occupation and whether or not they were a musician alongside their comments on the experience. 24 female's age range 16–61 years, 14 musicians and 10 non-musicians, 76 male's age range 16–66 years, 48 musicians and 28 non-musicians; occupations were recorded for use in future analysis. For discussions of elicitation problems and techniques, see [13–16]. This openended approach does not presuppose noteworthy attributes but is used to elicit them.

The listening tests took place across three days under non-ideal conditions at PLASA London, as part of the Exploratorium exhibit we shared the space with four other exhibitors. The Exploratorium was located on the upper level of the large exhibition hall, a large footfall and other exhibitors using amplified sound produced a considerable noise floor (see Section 5.1).

Participants were self-selecting; when any interest was shown they were invited to take in the listening test before any discussion could take place. Once seated, the headset was placed on the participants head and a short piece of music played while they were shown how to increase the overall amplitude to a comfortable level; banded ear plugs were cleaned and given to the participant to use at their discretion. Each audition lasted five minutes and the volunteers were invited to record brief details and their observations on prepared forms immediately afterward. The method of recording responses proved to be suboptimal, as many volunteers went on to describe the experience in greater detail verbally during post-test discussion than subsequently on paper.

### 4. Responses and analysis

After the auditions, the data were collated into a spreadsheet for analysis; the transcribed responses were examined for frequently occurring descriptive terms and related synonyms. This resulted in a collection of themes, the dominant theme was 'positive' at 78% and this was then correlated with other descriptive themes to elicit accompanying qualia that might contribute to the overall impression of 'positive'.

#### 4.1. Participant comment samples

1. Male, 23, DJ, Musician

3.2. Signals

2017

200

3.3. Method

discussion than subsequently on paper.

tribute to the overall impression of 'positive'.

4. Responses and analysis

Signals were processed using Reaper DAW and spatially encoded using WigWare 1st order ambisonic panning; FX Plugins were used to construct early and late reflections and then decoded through a WigWare 1st order periphonic ambisonic decoder patched to the transducer array.

Proceedings of the Conference on Design and Semantics of Form and Movement - Sense and Sensitivity, DeSForM

A 1st order ambisonic recording of a country park captured using a Soundfield™ microphone provided the ambient background; stereo recordings of bird sounds, voices, a steam train and music alongside mono FX clips were used to create the soundscape in which 1st order

In this study, we used 100 naïve (i.e. inexperienced in TC listening) and untutored (subjects receive no instructions on target attributes) listeners, who were then invited to offer observations and comment on the experience; of the 100 listeners non-reported previous experience of tissue conducted sound. 24 female and 76 male participants took part; each was asked to record their age, sex, occupation and whether or not they were a musician alongside their comments on the experience. 24 female's age range 16–61 years, 14 musicians and 10 non-musicians, 76 male's age range 16–66 years, 48 musicians and 28 non-musicians; occupations were recorded for use in future analysis. For discussions of elicitation problems and techniques, see [13–16]. This open-

The listening tests took place across three days under non-ideal conditions at PLASA London, as part of the Exploratorium exhibit we shared the space with four other exhibitors. The Exploratorium was located on the upper level of the large exhibition hall, a large footfall and other exhibitors using amplified sound produced a considerable noise floor (see Section 5.1). Participants were self-selecting; when any interest was shown they were invited to take in the listening test before any discussion could take place. Once seated, the headset was placed on the participants head and a short piece of music played while they were shown how to increase the overall amplitude to a comfortable level; banded ear plugs were cleaned and given to the participant to use at their discretion. Each audition lasted five minutes and the volunteers were invited to record brief details and their observations on prepared forms immediately afterward. The method of recording responses proved to be suboptimal, as many volunteers went on to describe the experience in greater detail verbally during post-test

After the auditions, the data were collated into a spreadsheet for analysis; the transcribed responses were examined for frequently occurring descriptive terms and related synonyms. This resulted in a collection of themes, the dominant theme was 'positive' at 78% and this was then correlated with other descriptive themes to elicit accompanying qualia that might con-

ended approach does not presuppose noteworthy attributes but is used to elicit them.

ambisonic recordings of a motorbike and aeroplane were placed.

'Very interesting, new experience of sound. Vibrations feel slightly unusual but also add a new dimension to the sound experience. Very cool' (positive, interesting, vibrations, weird).

2. Male, 28, Theatre Tech, Non-Musician

'Loved the vibrations of the aeroplane flying over and in general how the sound felt all around' (positive, spatial, surround, vibrations, external).

3. Male, 37, Equipment Sales, Musician

'I enjoyed the vibrations on the pressure points. Sound has remarkable stereo/surround perception. With ear plugs in, it felt like listening to headphones. A pleasant experience' (positive, surround, vibrations, feel, external, headphones).

4. Female, 18, Student, Musician

'Occasionally you could feel the deeper sounds as physical vibrations especially in the front central point. The higher sounds like bird song were easier to pick up what direction it was coming from. The music sounded better than it would through regular headphones as you felt surrounded by the sound as you would in a realistic setting of an orchestra' (positive, spatial, surround, clarity, vibrations, feel, external, headphones).


#### 4.2. Emergent themes

Table 1. Emergent themes and descriptors used for each.

Figure 2. Emergent themes from 100 participants' comments.

#### 4.3. Co-occurring themes

Themes were cross-correlated to elicit what impressions might contribute to overall positivity (or not) of the experience. So, for instance, 'interesting' mapped significantly to 'positive'; Figure 3 shows themes mapped against positive and Figure 4 shows themes mapped against vibrations and positive combined as this forms an area of future interest.

Figure 3. Emergent themes mapped against positive.

Figure 4. Themes present when mapped against vibrations + positive.

### 5. Discussion

4.3. Co-occurring themes

2017

202

Figure 2. Emergent themes from 100 participants' comments.

Figure 3. Emergent themes mapped against positive.

Themes were cross-correlated to elicit what impressions might contribute to overall positivity (or not) of the experience. So, for instance, 'interesting' mapped significantly to 'positive'; Figure 3 shows themes mapped against positive and Figure 4 shows themes mapped against

Proceedings of the Conference on Design and Semantics of Form and Movement - Sense and Sensitivity, DeSForM

vibrations and positive combined as this forms an area of future interest.

The high incidence of 'positive', 'spatial' and 'interesting' descriptors indicates the technique is worth investigating further, while the significant correlations between 'spatial' and other categories suggest that the potential informativeness may extend beyond that for single or twin displays.

Some listeners (19%) specifically commented in terms of 'clarity', which was interesting in the context of the background noise in the listening venue.

It is plausible that the experience is unfamiliar and not completely understandable in the short time frame, 58% (3% overlap) of listeners comments contain reference to 'weird' or 'interesting'. Another, commensurate explanation is that this constitutes a different kind of experience, an artificial exaptation [17].

In respect of reports of spatial impressions, we are currently reviewing the question of whether we are actually presenting signals that are physically equivalent (to the spatial signal set one would normally apprehend via air conduction), or whether we are presenting something which achieves a degree of perceptual equivalence through more abstract relationships. In the first case, what would be implied is that we are, inadvertently, providing equivalents to those aspects of the head-related transfer functions (HRTFs) that would bear strong relationships with perceptions of externalisation and elevation. Because of the multiplicity of signal paths inside the cranium, which vary with frequency and transducer locations, differences in frequency component arrival times within grouped and segregated sensory data [3–6, 18]

may induce interaural time difference fluctuations and hence elicit some sense of spacious envelopment [19]. We are not yet able to model the complex signal arriving at the cochleae, and in any event, it is exceedingly unlikely that we are precisely mimicking the individualised HRTFs of all listeners.

If, on the other hand, more abstract perceptual equivalence is in evidence, it is puzzling that spatial impressions are elicited in such short exposures. Informal listening tests of prolonged and repeated exposures do seem to indicate that spatial judgements improve; small sample size and uncontrolled test circumstances constrain conclusions.

An intriguing alternative possibility is that we are actually generating non-audible cues that are perceptually interpreted as auditory spatial cues. A significant (and unanticipated) contributor to the experience is that of vibration, which, in 24% of comments appears in a neutral or positive context, and in 12% is positively associated with comments on spatial impression. The vibrations are due to listening-circumstance inadequacies; the ambient noise floor was high, the transducers have limited dynamic range and modest frequency range. We assumed that vibration would be strongly associated with negative terms, but this only proves to be the case in 3% of responses. The tentative inference is that coherent (i.e. covariant with modulated auditory input) tactile input is potentially perceptually assimilable; this would exemplify multimodal perception.

Multimodality of perception has received increasing interest in the last four decades. The ubiquity of multisensory neurons (that can receive inputs from two or more sensory domains) in the brain indicates that multisensory integration is not limited to 'higher' cognitive processes but can occur at more fundamental levels. For instance, neurons in the primary visual cortex receive inputs from the primary auditory cortex [20]. Multimodality can be discussed in terms of cross-modal effects (where stimulus input to one modality alters the perceptual conclusions in another), for examples, the motion bounce illusion, see [21] and the McGurk effect [22]. It can also be discussed in terms of super-additive effects, where application of concurrent multimodal stimuli produces more disproportionately more robust perceptual conclusions than for unimodal stimuli [23]. For discussion of multisensory interplay, see [24].

The key observation in the question of unimodal and multimodal perception is that, while brain-region specialisation is well documented and unimodal perception is known to occur, perception in one modality can be affected by input to another and further, if stimuli to one modality are impoverished, input via another can be effectively cognitively utilised [25].

#### 5.1. Limitations

The transducers have restricted frequency response of 200 Hz to 16 KHz and component matching is problematic.

The generic headset could not be calibrated for consistency of transducer location and contact force for each individual in such a large cohort, leading to inconsistencies in the sensory experience across the cohort.

High ambient noise levels in the listening area probably interfered with subtlety of detail in the programme material.

Although listeners were unsolicited, some arguably had prior knowledge and possible expectations borne from previous listeners' comments. The 'interesting' category should be considered with caution, since, by definition, volunteers were interested enough to come forward.

Respondents were generally more fluent in their verbal descriptions than their written responses; only the written responses were recorded for analysis.

Variations in descriptions were noted; for instance, while some commented on a startling degree of clarity, others observed the opposite. Such variation may stem from variations in 'degree of fit' of the prototype apparatus, variations in physiology (skull thickness, for instance) or variations in biomechanical and/or neurological auditory processing.

An important limitation is that we did not categorise responses in terms of degree of emphasis, due to intrinsic uncertainties of use of language; for example, 'very spacious' and 'spacious' were categorised similarly.

### 6. Conclusions and further work

may induce interaural time difference fluctuations and hence elicit some sense of spacious envelopment [19]. We are not yet able to model the complex signal arriving at the cochleae, and in any event, it is exceedingly unlikely that we are precisely mimicking the individualised

Proceedings of the Conference on Design and Semantics of Form and Movement - Sense and Sensitivity, DeSForM

If, on the other hand, more abstract perceptual equivalence is in evidence, it is puzzling that spatial impressions are elicited in such short exposures. Informal listening tests of prolonged and repeated exposures do seem to indicate that spatial judgements improve; small sample

An intriguing alternative possibility is that we are actually generating non-audible cues that are perceptually interpreted as auditory spatial cues. A significant (and unanticipated) contributor to the experience is that of vibration, which, in 24% of comments appears in a neutral or positive context, and in 12% is positively associated with comments on spatial impression. The vibrations are due to listening-circumstance inadequacies; the ambient noise floor was high, the transducers have limited dynamic range and modest frequency range. We assumed that vibration would be strongly associated with negative terms, but this only proves to be the case in 3% of responses. The tentative inference is that coherent (i.e. covariant with modulated auditory input) tactile input is potentially perceptually assimilable; this would exemplify

Multimodality of perception has received increasing interest in the last four decades. The ubiquity of multisensory neurons (that can receive inputs from two or more sensory domains) in the brain indicates that multisensory integration is not limited to 'higher' cognitive processes but can occur at more fundamental levels. For instance, neurons in the primary visual cortex receive inputs from the primary auditory cortex [20]. Multimodality can be discussed in terms of cross-modal effects (where stimulus input to one modality alters the perceptual conclusions in another), for examples, the motion bounce illusion, see [21] and the McGurk effect [22]. It can also be discussed in terms of super-additive effects, where application of concurrent multimodal stimuli produces more disproportionately more robust perceptual conclusions

The key observation in the question of unimodal and multimodal perception is that, while brain-region specialisation is well documented and unimodal perception is known to occur, perception in one modality can be affected by input to another and further, if stimuli to one modality are impoverished, input via another can be effectively cognitively utilised [25].

The transducers have restricted frequency response of 200 Hz to 16 KHz and component

The generic headset could not be calibrated for consistency of transducer location and contact force for each individual in such a large cohort, leading to inconsistencies in the sensory

High ambient noise levels in the listening area probably interfered with subtlety of detail in the

than for unimodal stimuli [23]. For discussion of multisensory interplay, see [24].

size and uncontrolled test circumstances constrain conclusions.

HRTFs of all listeners.

2017

204

multimodal perception.

5.1. Limitations

matching is problematic.

experience across the cohort.

programme material.

The initial qualitative investigation indicates that the use of multiple transducers, decoded to so as to display spatial and musical information, is worth further exploration. The areas of interest are: externalisation (i.e. not 'in the head'), control of spaciousness, range perception and tactile augmentation.

In terms of informational bandwidth, there seem to be several justifications for using multiple transducers: dynamic range and frequency response of the apparatus is improved simply because more moving mass (in the transducers) and power are deployed. A general improvement in sense of spatiality is indicated, though the spatial impressions evinced are not precisely the same as for real environments, or other artificial means of depicting spatial sound. This is uncontroversial, since spatial qualia for headphone, in-earphone and loudspeaker presentations also differ. Headphones can give impressions of 'in the head' intimate sound fields but are correspondingly poor in producing the impression of externalisation and rangeperception, while the reverse is true for loudspeaker presentations. The tissue-conducted sound fields do, reportedly, convey some sense of spaciousness, envelopment or immersiveness, indicating externalisation, though it is unclear whether range perception can be coherently controlled. Directional localisation of sources appears to be imprecise; while some remarks suggest impressions of elevation, this requires more precise investigation.

The question of whether (and how) tactile stimuli interact with auditory stimuli requires elucidation as this has valuable implications for normal and hearing-impaired listeners; auditory spatial perception might actually be augmented with coherent tactile input. There is evidence that tactile stimuli can affect auditory conclusions [26, 27] and visual perceptions [28]. To investigate this, we shall have to improve the transduction of audio signals to an extent where spurious vibrations are minimised, while additionally utilising transducers specifically manage tactile input.

In the present context, multimodality has this implication: in conditions of suboptimal conditions such as hearing deficits, background noise and display limitations, it might be possible to

utilise cross-modal and super-additive effects to enhance auditory perception. Enhancements could include improved source segregation and intelligibility, along with more holistic qualia such as immersiveness and enjoyability.

### Author details

Peter Lennox\* and Ian McKenzie

\*Address all correspondence to: p.lennox@derby.ac.uk

University of Derby, UK

### References


[10] Stanley R, Walker BN. Lateralization of sounds using bone-conduction headsets. In: Proceedings of the Annual Meeting of the Human Factors and Ergonomics Society (HFES 2006); 16-20 October 2006; San Francisco, CA. p. 1571-1575

utilise cross-modal and super-additive effects to enhance auditory perception. Enhancements could include improved source segregation and intelligibility, along with more holistic qualia

Proceedings of the Conference on Design and Semantics of Form and Movement - Sense and Sensitivity, DeSForM

[1] Lin FR, Metter EJ, O'Brien RJ, Resnick SM, Zonderman AB, Ferrucci L. Hearing loss and incident dementia. Archives of Neurology. 2011;68(2):214-220. DOI: 10.1001/archneurol.

[2] Lin FR, Niparko JK, Ferrucci L. Hearing loss prevalence in the United States. Archives of Internal Medicine. 2011;171(20):1851-1852. DOI: 10.1001/archinternmed.2011.506 171/20/

[3] Stenfelt S, Goode RL. Transmission properties of bone conducted sound: Measurements in cadaver heads. The Journal of the Acoustical Society of America. 2005;118:2373-2391.

[4] Sohmer H. Reflections on the role of a traveling wave along the basilar membrane in view of clinical and experimental findings. European Archives of Oto-Rhino-Laryngology.

[5] Jun KC. Objective measurements of skull vibration during bone conduction audiometry

[6] Lenhardt ML, Shulman A, Goldstein BA. Bone-conduction propagation in the human body: Implications for high-frequency therapy. The International Tinnitus Journal. 2007;13(2):81-86

[7] Dietz AJ, May BS, Knaus DA, Greeley HP. Hearing protection for bone-conducted sound. In: New Directions for Improving Audio Effectiveness; Meeting Proceedings RTO-MP-

[8] Chordekar S, Kriksunov L, Kishon-Rabin L, Adelman C, Sohmer H. Mutual cancellation between tones presented by air conduction, by bone conduction and by non-osseous (soft tissue) bone conduction. Hearing Research. 2012;283(1–2):180-184. DOI: 10.1016/j.heares.

[9] McBride M, Letowski TR, Tran PK. Bone Conduction Head Sensitivity Mapping: Bone Vibrator. ARL-TR-3556. MD: U.S. Army Research Laboratory: Aberdeen Proving Ground; 2005

HFM-123, Paper 14. Neuilly-sur-Seine, France: RTO; 2005; p. 1–18

such as immersiveness and enjoyability.

\*Address all correspondence to: p.lennox@derby.ac.uk

2015;272:531. DOI: 10.1007/s00405-014-3045-z

[thesis]. Medical Faculty: Zürich University; 2009

Peter Lennox\* and Ian McKenzie

University of Derby, UK

References

2010.362

1851 [pii]

2011.10.004

DOI: 10.1121/1.2005847

Author details

2017

206

	- [25] Stein BE, Meredith MA. The Merging of the Senses. Cambridge, MA: MIT Press; 1993
	- [26] Tajadura-Jiménez A, Väljamäe A, Kitagawa N, Ho H. Whole-Body Vibration Influences Sound Localisation in the Median Plane. In: Proceedings of the 10th Annual International Workshop on Presence. Barcelona; Spain; October 2007
	- [27] Caclin A, Soto-Faraco S, Kingstone A, Spence C. Tactile "capture" of audition. Perception & Psychophysics. 2002;64(4):616-630
	- [28] Violentyev A, Shimojo S, Shams L. Touch-induced visual illusion. NeuroReport. 2005;16(10): 1107-1110

**Provisional chapter**

**SLAAP: Silencing Loud Alarms to Attenuate PTSD: Frequency-Selective Silencing Device for Digital Filtering of Alarm Sounds to Enhance ICU Patient Recovery Frequency-Selective Silencing Device for Digital Filtering of Alarm Sounds to Enhance ICU Patient Recovery**

**SLAAP: Silencing Loud Alarms to Attenuate PTSD:** 

DOI: 10.5772/intechopen.71112

Alyna Pradhan, Elizabeth Reynolds, Brittany Sweyer and Joseph J. Schlesinger Brittany Sweyer and Joseph J. Schlesinger Additional information is available at the end of the chapter

Alyna Pradhan, Elizabeth Reynolds,

Additional information is available at the end of the chapter

http://dx.doi.org/10.5772/intechopen.71112

#### **Abstract**

[25] Stein BE, Meredith MA. The Merging of the Senses. Cambridge, MA: MIT Press; 1993

Proceedings of the Conference on Design and Semantics of Form and Movement - Sense and Sensitivity, DeSForM

Workshop on Presence. Barcelona; Spain; October 2007

& Psychophysics. 2002;64(4):616-630

1107-1110

2017

208

[26] Tajadura-Jiménez A, Väljamäe A, Kitagawa N, Ho H. Whole-Body Vibration Influences Sound Localisation in the Median Plane. In: Proceedings of the 10th Annual International

[27] Caclin A, Soto-Faraco S, Kingstone A, Spence C. Tactile "capture" of audition. Perception

[28] Violentyev A, Shimojo S, Shams L. Touch-induced visual illusion. NeuroReport. 2005;16(10):

Free-field auditory medical alarms, although widely present in intensive care units, have created a number of hazards for both patients and clinicians in this environment. The harsh characteristics of the alarm noise profile combined the frequency at which they sound throughout the ICU have created discomfort for the patients and contribute to psychological problems, like post-traumatic stress disorder (PTSD) and delirium. Thus, this frequency-selective silencing device seeks to attenuate these problems by removing the alarm sounds from the patient perspective. Patients do not need to hear these alarms as the alarms primarily serve to alert clinicians; therefore, this device, through the use of a Raspberry Pi and digital filters, removes the alarm sounds present in the environment while passing all other sounds to the patient without distortion. This allows patients to hear everything occurring around them and to communicate effectively without experiencing the negative consequences of audible alarms.

**Keywords:** digital filtering, in-ear device, clinical alarms, wearable technology, ICU patient recovery

### **1. Introduction**

Major issues inhibiting successful patient recovery in intensive care units (ICUs) are the frequent occurrence of clinical alarms and the harsh, shrill noises that generally characterize these sounds. Alarms sound frequently to alert clinicians of physiological aberrancy that exceeds a threshold,

© 2016 The Author(s). Licensee InTech. This chapter is distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/3.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited. © 2017 The Author(s). Licensee InTech. This chapter is distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/3.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

yet many alarms have low-positive predictive value [1]. As stated by Edworthy and colleagues, multiparameter auditory warnings can be combined to create varying degrees of urgency [2]. Although the implementation of these results has proven useful to alert clinicians of possible danger, the potential negative consequences from the piercing alarm sounds were not considered from the patient perspective. While clinicians can suffer from alarm fatigue and desensitization, in this project, the patient-specific consequences are of the utmost concern, as patients commonly experience sleep deprivation, post-traumatic stress disorder (PTSD) anchored to critical illness, and delirium after a stay in the ICU [3]. Despite surviving an ICU stay, 88% of ICU patients experience hallucinatory/delusional intrusive memories related to ICU care for up to 8 months after hospital discharge [3], and the incidence of cognitive impairment as a function of ICU stay increases from 6 to 25% of patients [4].

While the underlying causes of these neuropsychological outcomes are not determined, the frequent, loud noises produced by clinical alarms often disturb patients' sleep patterns and sound for extended lengths of time with no explanation to the patient, the reason behind the alarm. Compared to other high-consequence industries, health care suffers from poor positive predictive value alarms, as 67.2% of the alarms of the ICUs are false positives [5].

Our approach of sheltering the patients from alarms is accomplished by the creation of a wearable frequency-selective silencing device which silences the frequencies corresponding to the alarm noises (primarily patient monitor red/crisis alarm) and will allow the passage of all normal sounds (speech and other environmental stimuli), while maintaining their quality to reduce the likelihood of delirium.

#### **1.1. Subtypes of PTSD anchored to critical illness: impact of sound**

Research is still ongoing to determine the specific sound exposure level of sound and the impact on neuropsychological outcomes in the ICU; specifically, the fractionation of sound into alarm and nonalarm contributions and the psychoacoustic features of sound (e.g., roughness, sharpness, and amplitude envelope) that may be deleterious to the patient. In the case of passive noise cancellation, sound source localization from a point different from the patient's ears could lead to spatial disorientation. Within the DSM-5 types of PTSD, there is a dissociative subtype of PTSD that is defined by symptoms of derealization and depersonalization [6]. The depersonalization experience could be an "out-of-body" experience, which could exacerbate the PTSD symptomatology. In an effort to not solve one problem and make manifold problems in the process, our approach and design will build from a single microphone passive cancellation process to a microphone array active cancellation process as described below.

### **2. Device design needs**

For patient-specific needs, the wearable technology must be user-friendly and comfortable to allow for continuous patient wear, especially while the patient is asleep. It must block alarm sounds while allowing the passage of all other environmental noise, such as speech and TV sounds. It is important to note that overstimulation of the auditory sense as well as a complete lack of stimulation of the auditory sense can contribute to PTSD and delirium, which is why noise-canceling headphones and/or simple earplugs that dampen all environmental noise entirely are *not* the desired solution.

#### **2.1. Wearer comfort and sound reproduction**

yet many alarms have low-positive predictive value [1]. As stated by Edworthy and colleagues, multiparameter auditory warnings can be combined to create varying degrees of urgency [2]. Although the implementation of these results has proven useful to alert clinicians of possible danger, the potential negative consequences from the piercing alarm sounds were not considered from the patient perspective. While clinicians can suffer from alarm fatigue and desensitization, in this project, the patient-specific consequences are of the utmost concern, as patients commonly experience sleep deprivation, post-traumatic stress disorder (PTSD) anchored to critical illness, and delirium after a stay in the ICU [3]. Despite surviving an ICU stay, 88% of ICU patients experience hallucinatory/delusional intrusive memories related to ICU care for up to 8 months after hospital discharge [3], and the incidence of cognitive impairment as a function

Proceedings of the Conference on Design and Semantics of Form and Movement - Sense and Sensitivity, DeSForM

While the underlying causes of these neuropsychological outcomes are not determined, the frequent, loud noises produced by clinical alarms often disturb patients' sleep patterns and sound for extended lengths of time with no explanation to the patient, the reason behind the alarm. Compared to other high-consequence industries, health care suffers from poor positive

Our approach of sheltering the patients from alarms is accomplished by the creation of a wearable frequency-selective silencing device which silences the frequencies corresponding to the alarm noises (primarily patient monitor red/crisis alarm) and will allow the passage of all normal sounds (speech and other environmental stimuli), while maintaining their quality

Research is still ongoing to determine the specific sound exposure level of sound and the impact on neuropsychological outcomes in the ICU; specifically, the fractionation of sound into alarm and nonalarm contributions and the psychoacoustic features of sound (e.g., roughness, sharpness, and amplitude envelope) that may be deleterious to the patient. In the case of passive noise cancellation, sound source localization from a point different from the patient's ears could lead to spatial disorientation. Within the DSM-5 types of PTSD, there is a dissociative subtype of PTSD that is defined by symptoms of derealization and depersonalization [6]. The depersonalization experience could be an "out-of-body" experience, which could exacerbate the PTSD symptomatology. In an effort to not solve one problem and make manifold problems in the process, our approach and design will build from a single microphone passive cancellation process to a microphone array active cancellation process as described below.

For patient-specific needs, the wearable technology must be user-friendly and comfortable to allow for continuous patient wear, especially while the patient is asleep. It must block alarm sounds while allowing the passage of all other environmental noise, such as speech and TV sounds. It is important to note that overstimulation of the auditory sense as well as a complete

predictive value alarms, as 67.2% of the alarms of the ICUs are false positives [5].

**1.1. Subtypes of PTSD anchored to critical illness: impact of sound**

of ICU stay increases from 6 to 25% of patients [4].

2017

210

to reduce the likelihood of delirium.

**2. Device design needs**

Although patients can have full degree-of-freedom head movement, patients and clinicians may be concerned that wearing headphones or earbuds would be uncomfortable to wear for prolonged periods of time; thus, future iterations of our design will incorporate the work by Voix and colleagues to develop comfortable wearable devices [7]. With wearable devices, there is additional concern of microphone placement and sound localization. With respect to the concern of the microphone being overly sensitive and amplifying environmental sounds that would be otherwise filtered by the human ear, we attenuated this difference by the application of an Audio-Technica (Tokyo, Japan) AT8131 windscreen. An additional concern was the quality of the audio path and exacerbation of spatial disorientation. As an initial step, a single microphone was used. However, future iterations of our design will use a microphone array with digital signal processing (DSP) tools for the real-time synthesis of a 3D sound pressure field using Ambisonics technologies to achieve the spatialization of monophonic signal or the reconstruction of natural 3D recorded sound pressure fields as guided by the work of Gauthier and colleagues [8]. As the focus on a "ground-up" ICU design from a multisensory aspect flourishes, ICU rooms are made to be quieter and more anechoic. As that is achieved, wave field synthesis (WFS), an open-loop technology, can be explored in concordance with environmental design. Specifically, adaptive wave field synthesis, combining WFS and active control to reproduce the spatial character of natural hearing, will ameliorate concerns of patient dissociative subtypes of PTSD symptomatology [9].

### **3. Digital signal processing**

To remove the alarm sound, MATLAB (MathWorks, Natick MA) digital signal processing was utilized to initially implement and test our digital filters. A spectral analysis was performed on a single alarm sound to obtain its frequency components. Then, an Infinite Impulse Response (IIR) Elliptic bandstop filter was created to block the frequency that specifically dominated in the spectral analysis. The width of the stopband had to be optimized so that the alarm component was completely blocked, yet the effect on environmental noise was minimized. This led to the creation of filters targeting the common red/patient crisis alarm with the most important ones focused at 960, 1920, 2880 and 3840 Hz.

The dynamic digital filter was then generated in Simulink (MathWorks, Natick MA) using the filter specifications determined in MATLAB. The design is two-fold in that it contains both a detector and a series of filters. The detector continuously processes all incoming environmental sounds and determines the power present in the unfiltered environmental noise as compared to the power present in the filtered version. If this difference exceeds a predetermined threshold, this serves to indicate that an alarm is present in the environment. If the alarm sound is detected, the detector switches on the digital filter, and the filtered version of the noise is passed to the patient.

This switching mechanism is critical to the design as it ensures that unnecessary processing and potential distortion will not occur for the patient if no alarms are sounding in the environment.

#### **3.1. Auditory masking**

When auditory filtering is used, there is a concern of inadvertently filtering desired auditory stimuli. This is especially important when one needs to respond to the auditory stimulus. A relatively understudied source of response failures deals with simultaneous masking, a condition where concurrent sounds interact in ways that make one or more imperceptible due to physical limitations on perception. Bolton and colleagues have developed a novel combination of psychophysical modeling and formal verification with model checking to detect masking in a modeled configuration of medical alarms. This builds on previous work by adding the ability to detect additive masking while concurrently improving method usability and scalability [10]. The psychoacoustics used to describe masking represent frequency on the Bark scale, which maps a frequency (in Hz) to a location on the basilar membrane where the sound stimulates the receptors the strongest. Frequency to Bark conversion is calculated as *zsound =* 13\*arctan (0.00076\**f sound*) + 3.5\*arctan ((*f sound*/7500)<sup>2</sup> ). As the alarm we filtered, the "red" high acuity patient alarm has a spectral component of the peak frequency of its narrow bandwidth outside the range of other typical environmental stimuli (e.g., speech), we did not pursue further model checking for additive masking. However, this approach can be used as other patient alarms (e.g., ventilator and infusion pump) are added to the alarm filtering schemata.

### **4. Device components**

The hardware portion of this device continuously completes the digital filtering task during the device's operation. To do this, the Simulink code for the detector and filter has been uploaded onto a Raspberry Pi (Raspberry Pi Foundation, Cambridge, UK) to allow for alarm filtration. A microphone connected to the Raspberry Pi obtains and passes the environmental sound to the digital detector (**Figure 1**).

**Figure 1.** Depiction of design prototype.

### **5. Proof of concept**

**Figure 1.** Depiction of design prototype.

**3.1. Auditory masking**

2017

212

(0.00076\**f*

*sound*) + 3.5\*arctan ((*f*

sound to the digital detector (**Figure 1**).

**4. Device components**

*sound*/7500)<sup>2</sup>

(e.g., ventilator and infusion pump) are added to the alarm filtering schemata.

This switching mechanism is critical to the design as it ensures that unnecessary processing and potential distortion will not occur for the patient if no alarms are sounding in the environment.

Proceedings of the Conference on Design and Semantics of Form and Movement - Sense and Sensitivity, DeSForM

When auditory filtering is used, there is a concern of inadvertently filtering desired auditory stimuli. This is especially important when one needs to respond to the auditory stimulus. A relatively understudied source of response failures deals with simultaneous masking, a condition where concurrent sounds interact in ways that make one or more imperceptible due to physical limitations on perception. Bolton and colleagues have developed a novel combination of psychophysical modeling and formal verification with model checking to detect masking in a modeled configuration of medical alarms. This builds on previous work by adding the ability to detect additive masking while concurrently improving method usability and scalability [10]. The psychoacoustics used to describe masking represent frequency on the Bark scale, which maps a frequency (in Hz) to a location on the basilar membrane where the sound stimulates the receptors the strongest. Frequency to Bark conversion is calculated as *zsound =* 13\*arctan

alarm has a spectral component of the peak frequency of its narrow bandwidth outside the range of other typical environmental stimuli (e.g., speech), we did not pursue further model checking for additive masking. However, this approach can be used as other patient alarms

The hardware portion of this device continuously completes the digital filtering task during the device's operation. To do this, the Simulink code for the detector and filter has been uploaded onto a Raspberry Pi (Raspberry Pi Foundation, Cambridge, UK) to allow for alarm filtration. A microphone connected to the Raspberry Pi obtains and passes the environmental

). As the alarm we filtered, the "red" high acuity patient

#### **5.1. Experimental design and testing**

To prove objectively that the device accomplished our aims, an experiment was performed to prove that the frequency components specific to the alarms were missing from the filtered sound. In the initial stages of the project, a Fast Fourier Transform (FFT) was performed using MATLAB on the unfiltered alarm sound sample and the filtered alarm in order to compare the magnitudes of the frequency components present between the two sounds (**Figures 2** and **3**).

**Figure 2.** FFT of a single unfiltered alarm (left) and the same alarm filtered by a series of bandstop filters (Note: The Y-axis values are different to display the present spectral waveform after filtering).

**Figure 3.** Alarm filtered by a series of bandstop filters in Simulink (Note: The Y-axis values are different to display the present spectral waveform after filtering).

#### **5.2. Results**

In the objective testing using MATLAB, the series of bandstop filters created on MATLAB dampened the magnitudes of the frequencies present in the alarm in the order of 10<sup>3</sup> , as seen in **Figure 2**.

Once the filtering on MATLAB proved successful, Simulink (Mathworks) was used to compile the software and deploy the data onto a Raspberry Pi device. By inputting a file (.wave) with both the alarm sound and environmental noise present, it was proven that the Simulink software was able to successfully filter the alarm frequencies as shown in **Figure 3**.

### **6. Limitations and future directions**

As of now, this device relies on the use of noise-canceling headphones to transmit the filtered sound to the patient. Future designs will incorporate a wireless, in-ear device that can perform all the necessary filtering functions and transmission of the filtered sound in the device itself. With an aggressive goal to reduce cost and time of development, our first model uses passive filtering; however, further developments will incorporate active noisecancellation which will obviate the need for the passive device to activate (~100–300 ms) and avoid a slight perceived auditory click of activation. With evolving design, this technologycentered initial approach will require FDA exemption status, so it can be studied in the clinical environment.

### **7. Conclusion**

Audible medical alarms are the cause of a number of hazards in hospital and ICU settings. Their shrill acoustic features and the frequency at which they alarm (both in sheer number and frequency spectrum) are responsible for a number of negative consequences, especially for patients. Patients can experience PTSD and delirium secondary to sleep disturbance from alarms and health care providers' divided and diminished attentional resources allocated to alarms. This frequency-selective silencing device was created to alleviate these problems and create a more comfortable environment for the patients during their length of stay in the ICU and promote patient safety.

### **Acknowledgements**

We would like to thank the Departments of Anesthesiology and Hearing and Speech Sciences at Vanderbilt University, particularly Dr. Ben Hornsby. We would also like to thank the Departments of Electrical Engineering and Computer Science at Vanderbilt University, especially Dr. A. B. Bonds, Dr. Dean Wilkes, and Garrett Hoffman for their assistance. We would also like to acknowledge Dr. Matthew Walker III and the Department of Biomedical Engineering at Vanderbilt University for their support.

### **Author details**

, as seen

**5.2. Results**

2017

214

in **Figure 2**.

clinical environment.

and promote patient safety.

**Acknowledgements**

**7. Conclusion**

In the objective testing using MATLAB, the series of bandstop filters created on MATLAB

Proceedings of the Conference on Design and Semantics of Form and Movement - Sense and Sensitivity, DeSForM

Once the filtering on MATLAB proved successful, Simulink (Mathworks) was used to compile the software and deploy the data onto a Raspberry Pi device. By inputting a file (.wave) with both the alarm sound and environmental noise present, it was proven that the Simulink soft-

As of now, this device relies on the use of noise-canceling headphones to transmit the filtered sound to the patient. Future designs will incorporate a wireless, in-ear device that can perform all the necessary filtering functions and transmission of the filtered sound in the device itself. With an aggressive goal to reduce cost and time of development, our first model uses passive filtering; however, further developments will incorporate active noisecancellation which will obviate the need for the passive device to activate (~100–300 ms) and avoid a slight perceived auditory click of activation. With evolving design, this technologycentered initial approach will require FDA exemption status, so it can be studied in the

Audible medical alarms are the cause of a number of hazards in hospital and ICU settings. Their shrill acoustic features and the frequency at which they alarm (both in sheer number and frequency spectrum) are responsible for a number of negative consequences, especially for patients. Patients can experience PTSD and delirium secondary to sleep disturbance from alarms and health care providers' divided and diminished attentional resources allocated to alarms. This frequency-selective silencing device was created to alleviate these problems and create a more comfortable environment for the patients during their length of stay in the ICU

We would like to thank the Departments of Anesthesiology and Hearing and Speech Sciences at Vanderbilt University, particularly Dr. Ben Hornsby. We would also like to thank the Departments of Electrical Engineering and Computer Science at Vanderbilt University, especially Dr. A. B. Bonds, Dr. Dean Wilkes, and Garrett Hoffman for their assistance. We

dampened the magnitudes of the frequencies present in the alarm in the order of 10<sup>3</sup>

ware was able to successfully filter the alarm frequencies as shown in **Figure 3**.

**6. Limitations and future directions**

Alyna Pradhan<sup>1</sup> , Elizabeth Reynolds<sup>1</sup> , Brittany Sweyer<sup>1</sup> and Joseph J. Schlesinger<sup>2</sup> \*


### **References**

	- [9] Gauthier PA, Berry A. Adaptive wave field synthesis with independent radiation mode control for active sound field reproduction. The Journal of the Acoustical Society of America. 2006;**119**:2721-2735
	- [10] Hasanain B, Boyd AD, Edworthy J, Bolton ML. A formal approach to discovering simultaneous additive masking between auditory medical alarms. Applied Ergonomics. 2017;**58**:500-514

**The Perceptual and Experiential Gap - Sensory Engagement**

[9] Gauthier PA, Berry A. Adaptive wave field synthesis with independent radiation mode control for active sound field reproduction. The Journal of the Acoustical Society of

Proceedings of the Conference on Design and Semantics of Form and Movement - Sense and Sensitivity, DeSForM

[10] Hasanain B, Boyd AD, Edworthy J, Bolton ML. A formal approach to discovering simultaneous additive masking between auditory medical alarms. Applied Ergonomics.

America. 2006;**119**:2721-2735

2017;**58**:500-514

2017

216

**Chapter 17** Provisional chapter

#### **Modified Social Benches: Exploring the Role of Aesthetic Interaction to Placemaking** Modified Social Benches: Exploring the Role of Aesthetic Interaction to Placemaking

DOI: 10.5772/intechopen.71118

## Andre G. Afonso

Additional information is available at the end of the chapter Andre G. Afonso

http://dx.doi.org/10.5772/intechopen.71118 Additional information is available at the end of the chapter

#### Abstract

This paper discusses some aesthetic and social aspects that involve interactions with urban art installations. The aim is to better understand how, and to what extent, aesthetic interactions with art installations can transform an urban space into a place. The discussion is based on a case study of the Modified Social Benches, a series of outdoor, interactive artworks that provide different types of bodily engagement, social encounters and aesthetic experiences. A detailed empirical analysis is carried out, emphasising the social roles around the installations as well as the most salient aspects regarding the bodily, the spatial and the experiential qualities of the interactions. The results suggest that urban installations affording playful, action-oriented and sensorimotor encounters are more effective to placemaking than installations that encourage static modalities of social activities.

Keywords: aesthetic interaction, placemaking, embodied interaction, interactive installation, social roles, engagement, play, urban space, place

#### 1. Introduction

In the last decade the theme of embodiment has received a growing attention among researches and practitioners from design and related fields. Research on embodiment is primarily concerned with how we use our bodies to shape the ways we perceive, feel and think [1]. A variety of disciplines, spanning from Philosophy to Human-Computer Interaction (HCI), have addressed different aspects of embodiment, and this has led to the emergence of more specific approaches. One of these approaches, known as Aesthetic Interaction, focuses on the experiential aspects of people's interactions with artefacts [2–5]. Emphasising the experience and the aesthetic dimension of interactions means adopting a holistically-oriented approach that considers the rich and complex nature of the whole human body—including the sensorimotor skills, the cognitive and emotional dispositions—that people use to make sense and produce meaning about an artefact, its context and ultimately about themselves.

© The Author(s). Licensee InTech. This chapter is distributed under the terms of the Creative Commons © 2017 The Author(s). Licensee InTech. This chapter is distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/3.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

Attribution License (http://creativecommons.org/licenses/by/3.0), which permits unrestricted use, distribution, and eproduction in any medium, provided the original work is properly cited.

The present discussion takes Aesthetic Interaction as a conceptual and methodological tool to analyse and better understand the process of placemaking. As a general notion, the term place refers to a physical space imbued with social and cultural meanings, values and traditions [6–8]. Unlike an abstract space, a place has its specific life, character, and identity, although all these qualities are more or less dynamic, that is, they may change according to the set of cultural, social, economic and political forces at play [9]. Then, placemaking can be interpreted as the process of giving life to a place, or infusing some qualities in a specific space so that it becomes a place: a spatial setting where people feel stimulated to come and to stay, where they can live a particular and enjoyable experience that cannot be lived elsewhere in the same way.

In order to analyse the relationships between Aesthetic Interaction and placemaking, this paper presents an empirical study on the series of interactive artworks Modified Social Benches, designed by Danish artist Jeppe Hein and installed in Southbank Centre, Central London during the Summer of 2016. The next section comprises a literature review on Aesthetic Interaction, placemaking and related concepts, bringing some relevant aspects as well as potentially fruitful connections between them.

### 2. Background

As the concepts of Embodied Interaction, Aesthetic Interaction and placemaking are essentially interdisciplinary, this section draws on different disciplines to outline the main characteristics of these concepts and their contributions to the study of urban interactive installations. Within the field of HCI, Dourish authored a seminal work, Where the action is, where he defines Embodied Interaction as "the creation, manipulation, and sharing of meaning through engaged interaction with artefacts" ([10], p. 126). For Dourish, Embodied Interaction is a valuable approach "to illuminate not just how we act on technology, but how we act through it" ([10], p. 154, emphases on original). Dourish's position highlights the experiential and meaningful dimension of interactions: we act by means of our own bodies, and this encounter between ourselves and technological artefacts is the source of meaning. This idea of interaction as meaningful action also underpins the concept of Aesthetic Interaction. According to Petersen et al. [3], Aesthetic Interaction is about aesthetics of use, rather than aesthetics of appearance. They contend that Aesthetic Interaction involves the entire human body, that is, intellect and all the senses, aiming to create involvement, experience, surprise and serendipity in interaction [3].

The growing interest on the experiential and corporeal realms of human-artefact relationships has inspired a range of subdisciplines to emerge [5]. Especially related to the case study of this paper are the fields of embodied engagement, whole body interaction and kinesthetic interaction (e.g., [11–13]). What these fields have in common is the consideration of the interacting, living, active human body as the main source of analysis and design; such idea is opposed to Cartesian and cognitivist approaches that tend to reduce users as disembodied information processors [1, 14].

Interestingly, the main ideas and concepts outlined above have also informed research and design in Architecture. While Interaction Design takes the notion of embodiment from the perspective of a user in action through technology [10], some architectural thinkers have focused on embodiment as a source of meaningful experiences of places [15, 16]. In particular, Phenomenology and Neuroscience have provided important insights to both architects and interaction designers, leading to more holistic accounts of interactions. According to these accounts, people's interactions with objects and places are as much about corporeal, tangible encounters as they are affective, symbolic ones [5, 17–21].

The present discussion takes Aesthetic Interaction as a conceptual and methodological tool to analyse and better understand the process of placemaking. As a general notion, the term place refers to a physical space imbued with social and cultural meanings, values and traditions [6–8]. Unlike an abstract space, a place has its specific life, character, and identity, although all these qualities are more or less dynamic, that is, they may change according to the set of cultural, social, economic and political forces at play [9]. Then, placemaking can be interpreted as the process of giving life to a place, or infusing some qualities in a specific space so that it becomes a place: a spatial setting where people feel stimulated to come and to stay, where they can live a

Proceedings of the Conference on Design and Semantics of Form and Movement - Sense and Sensitivity, DeSForM

In order to analyse the relationships between Aesthetic Interaction and placemaking, this paper presents an empirical study on the series of interactive artworks Modified Social Benches, designed by Danish artist Jeppe Hein and installed in Southbank Centre, Central London during the Summer of 2016. The next section comprises a literature review on Aesthetic Interaction, placemaking and related concepts, bringing some relevant aspects as well as

As the concepts of Embodied Interaction, Aesthetic Interaction and placemaking are essentially interdisciplinary, this section draws on different disciplines to outline the main characteristics of these concepts and their contributions to the study of urban interactive installations. Within the field of HCI, Dourish authored a seminal work, Where the action is, where he defines Embodied Interaction as "the creation, manipulation, and sharing of meaning through engaged interaction with artefacts" ([10], p. 126). For Dourish, Embodied Interaction is a valuable approach "to illuminate not just how we act on technology, but how we act through it" ([10], p. 154, emphases on original). Dourish's position highlights the experiential and meaningful dimension of interactions: we act by means of our own bodies, and this encounter between ourselves and technological artefacts is the source of meaning. This idea of interaction as meaningful action also underpins the concept of Aesthetic Interaction. According to Petersen et al. [3], Aesthetic Interaction is about aesthetics of use, rather than aesthetics of appearance. They contend that Aesthetic Interaction involves the entire human body, that is, intellect and all the senses, aiming to

The growing interest on the experiential and corporeal realms of human-artefact relationships has inspired a range of subdisciplines to emerge [5]. Especially related to the case study of this paper are the fields of embodied engagement, whole body interaction and kinesthetic interaction (e.g., [11–13]). What these fields have in common is the consideration of the interacting, living, active human body as the main source of analysis and design; such idea is opposed to Cartesian and cognitivist approaches that tend to reduce users as disembodied information

Interestingly, the main ideas and concepts outlined above have also informed research and design in Architecture. While Interaction Design takes the notion of embodiment from the

create involvement, experience, surprise and serendipity in interaction [3].

particular and enjoyable experience that cannot be lived elsewhere in the same way.

potentially fruitful connections between them.

2. Background

2017

220

processors [1, 14].

Several disciplines have discussed the concept of place—its definition, its qualities, and how it is manifested and lived. What seems to unify the understanding of place is its human dimension, both in an individual, subjective level, and as a shared, communal experience. Places are regarded as spatial settings that provide people with feelings of well-being, safety, security and orientation [22]. This general notion of place, and the related concept of placemaking, gained traction in the 1960s, through the work of Jane Jacobs and, some years later, William Whyte. For Jacobs, successful places are characterised by intense and diverse social encounters, like casual meetings and chattings, which help enhance the security of urban communities by encouraging their members to naturally engage in a dynamics of mutual surveillance [23]. For Whyte, lively places are those where people find urban amenities—such as sitting spaces, food, water features, varied shops and businesses—that make them stay and enjoy a public space in an everyday basis [24]. It follows that placemaking is about turning segregated, lifeless environments into lively, satisfying ones.

More recently, placemaking has been regarded as a collaborative process which aims to maximise the "shared value" of public spaces, by focusing on its social, cultural and physical identities [25]. The two core principles of placemaking have been summarised as (a) focus on designing cities for people; and (b) inclusion of citizens in the decision making process of design [22]. Such principles highlight the role of local communities in the planning and design of places. What makes communities key elements in the process of placemaking are their unique attributes—for example, trust and support networks, common experiences and interests, types of transactions, history and proximity [7]—attributes that, together, help to constitute and qualify places over time. In a certain sense, communities make their own places by imbuing their everyday environments with shared values, habits and identities; when "material and spatial elements are given life by the meanings, associations and experiences people inject into them during daily life" ([26], p. 141).

Urban art installations can arguably contribute to the process of placemaking by enticing a broad variety of interactions. People may interact not only with the installations themselves, but also with other people, for example, by starting a conversation prompted by the installation—the process known as triangulation [24]. Moreover, interactive installations can make people stay longer in the area for different reasons: directly engaging with the installation, observing it from a distance, photographing or filming the actions around them or simply talking to people nearby. Altogether, these social activities or "social buzz" [27] tend to create a very particular and lively atmosphere of enjoyment, which helps to characterise a place.

Not surprisingly, embodied, playful and aesthetic modes of interaction have been employed for many years in urban spaces, especially in the context of parks, playgrounds or seasonal initiatives such as funfairs, action- or adventure-themed events and, more recently, urban

Figure 1. Left: Charlotte Ammundsens square, image courtesy of Danish architecture centre; right: Rock Rock Around the Block, a prototype presented at the market street prototyping festival 2016, image courtesy of market street prototyping festival.

prototyping festivals. What seems to be singular in the current turn to embodiment in Urban Design is an emphasis on the hybridization of concepts and typologies. Instead of discrete, selfcontained facilities designed for specific publics and activities—for example, a fenced playground where children play while adults watch—a range of contemporary urban projects have assumed a multifunctional character, fostering physical activity, playful behaviour, community participation and aesthetic interactions in a more fluid, open-ended way [28].

Part of these new initiatives has been built as permanent additions to the city: such as the Superkilen park and the Charlotte Ammundsens square, in Denmark, which features a rocky landscape that can be used for climbing, skating, biking or simply sitting or lying [29] (Figure 1, left). Other projects have been temporarily installed as part of events, such as Portland's City Repair Project, with its strong emphasis on community engagement for placemaking, and San Francisco's Market Street Prototyping Festival, in which artists and designers reinvented the concept of the urban bench by employing playful and striking forms to entice social encounters and new ways of experiencing the urban space (Figure 1, right).

However multifunctional, playful or enticing these contemporary projects are, they all seem to share an underlying principle: they seek to bring life to the city by proposing spatial and social experiences that are fundamentally rooted in the human body. In other words, these projects explore Aesthetic Interaction as a platform of placemaking. The Modified Social Benches, object of the present study, can be approached as part of these contemporary creative efforts. By extending the concept of Aesthetic Interaction to the urban scale, the following analysis aims to clarify how the bodily engagement with outdoor installations can contribute to the process of placemaking.

### 3. Modified social benches: a case study

This section presents an empirical study of the Modified Social Benches, a series of temporary, outdoor and interactive installations designed by Danish artist Jeppe Hein. The installations borrow their basic form from well-known park benches and alter their design to various degrees, transforming the act of sitting into a "conscious physical endeavour" [30] (Figure 2).

Modified Social Benches: Exploring the Role of Aesthetic Interaction to Placemaking http://dx.doi.org/10.5772/intechopen.71118 223

Figure 2. Two versions of the modified social benches.

Besides their sculptural quality that sparks people's attention, the Modified Social Benches can also be approached as a playful and performative experiment on bodily engagement and social behaviour in the urban space. By blurring the boundaries between Art, Interaction Design and Architecture, Jeppe Hein's installations offer a valuable context for analysing the process of placemaking in light of Aesthetic Interaction.

#### 3.1. Methodology

prototyping festivals. What seems to be singular in the current turn to embodiment in Urban Design is an emphasis on the hybridization of concepts and typologies. Instead of discrete, selfcontained facilities designed for specific publics and activities—for example, a fenced playground where children play while adults watch—a range of contemporary urban projects have assumed a multifunctional character, fostering physical activity, playful behaviour, commu-

Figure 1. Left: Charlotte Ammundsens square, image courtesy of Danish architecture centre; right: Rock Rock Around the Block, a prototype presented at the market street prototyping festival 2016, image courtesy of market street prototyping

Proceedings of the Conference on Design and Semantics of Form and Movement - Sense and Sensitivity, DeSForM

Part of these new initiatives has been built as permanent additions to the city: such as the Superkilen park and the Charlotte Ammundsens square, in Denmark, which features a rocky landscape that can be used for climbing, skating, biking or simply sitting or lying [29] (Figure 1, left). Other projects have been temporarily installed as part of events, such as Portland's City Repair Project, with its strong emphasis on community engagement for placemaking, and San Francisco's Market Street Prototyping Festival, in which artists and designers reinvented the concept of the urban bench by employing playful and striking forms to entice social encounters

However multifunctional, playful or enticing these contemporary projects are, they all seem to share an underlying principle: they seek to bring life to the city by proposing spatial and social experiences that are fundamentally rooted in the human body. In other words, these projects explore Aesthetic Interaction as a platform of placemaking. The Modified Social Benches, object of the present study, can be approached as part of these contemporary creative efforts. By extending the concept of Aesthetic Interaction to the urban scale, the following analysis aims to clarify how the bodily engagement with outdoor installations can contribute to the process of placemaking.

This section presents an empirical study of the Modified Social Benches, a series of temporary, outdoor and interactive installations designed by Danish artist Jeppe Hein. The installations borrow their basic form from well-known park benches and alter their design to various degrees, transforming the act of sitting into a "conscious physical endeavour" [30] (Figure 2).

nity participation and aesthetic interactions in a more fluid, open-ended way [28].

and new ways of experiencing the urban space (Figure 1, right).

3. Modified social benches: a case study

festival.

2017

222

After a bottom-up, exploratory approach of the installations in Southbank Centre, 4 out of 10 different versions of the Modified Social Benches were selected for this case study. The principal criteria for selecting the four benches were the following:


Figure 3. Overview of the area highlighting the four art installations.

Figure 4. Different behaviours towards the benches, from playing (left) to chatting (right).

The primary methodological tools used for this study were on-site observations and time-lapse photography. Observations are an important tool because they allow a gathering of data that might be considered as an objective view of human behaviour. By observing we can learn about the environment without taking account of people's intentions—something important, given that social activities in urban contexts give rise to patterns of use and movement that are independent of the intentions of individuals [31]. Time-lapse photography was used to record and analyse the interactions. Since the study did not focus on fine details of discrete interactions, but rather on patterns of bodily engagement and social behaviour on the broader urban scale, time-lapse photography seemed a suitable alternative to video recording, while being a technique that has been adopted by influential studies on human behaviour in outdoor settings [24].

The data collection was mainly conducted in August and September 2016, with a total of 15 recording sessions. Each recording session lasted between 30 and 35 min and employed a small action camera. The camera was positioned at the same spot in all sessions, and it was set to take one photograph at intervals of 60 s. The sessions took place at different times of the day and different days of the week, so as to capture the varying conditions of the urban and social setting.

#### 3.2. Social roles around the installations

In order to analyse the bodily, social and spatial interactions around the benches, the people recorded in the area of study were classified in three main categories: players, participants and spectators. Each of these categories tries to address the distinct ways in which people seemingly experience the art installations, both from a bodily, social and spatial perspective.

#### 3.2.1. Players

The category of players describes people who were observed engaging in any kind of performative movements with Jeppe Hein's benches. Playing, in this case, denotes a conspicuously active, corporeal and dynamic engagement of one's own body with the installations. For players, the bench is first and foremost an urban sculpture to be physically and sensorily explored and revealed. More than sparking visual attraction, for players the meaning of the sculptural bench can only be utterly realised through an engagement of their whole bodies, in a playful, physical, performative inspection of the bench's surfaces, structure and balance. Players seem to be driven by an instinct to bodily and sensorily unveil the bench's affordances for action, movement or pure play.

The images captured in Southbank Centre suggest that this playful behaviour is motivated by two main reasons: first, one's own curiosity to explore and to test different movement possibilities, that is, the bench's affordances for action and play. In this group were recorded the most performative people, who engaged in all sorts of acrobatic movements and postures, such as climbing, hanging or sliding on the benches. The second important motivation for adopting a playful behaviour is to pose for photographs or videos. Playing with the benches to have one's portrait taken was more frequent among adults (Figure 5).

Unlike conventional sitters who pose on conventional benches, most of the people being photographed amid Jeppe Hein's artworks deliberately adopted an unusual or "funny" body posture, so as to reflect the playful and liberal character of the installations. In either cases playing for the sake of playing or playing to be photographed—the interaction of players was markedly dynamic and fleeting.

#### 3.2.2. Participants

The primary methodological tools used for this study were on-site observations and time-lapse photography. Observations are an important tool because they allow a gathering of data that might be considered as an objective view of human behaviour. By observing we can learn about the environment without taking account of people's intentions—something important, given that social activities in urban contexts give rise to patterns of use and movement that are independent of the intentions of individuals [31]. Time-lapse photography was used to record and analyse the interactions. Since the study did not focus on fine details of discrete interactions, but rather on patterns of bodily engagement and social behaviour on the broader urban scale, time-lapse photography seemed a suitable alternative to video recording, while being a technique that has been adopted by influential studies on human behaviour in outdoor

Proceedings of the Conference on Design and Semantics of Form and Movement - Sense and Sensitivity, DeSForM

Figure 4. Different behaviours towards the benches, from playing (left) to chatting (right).

The data collection was mainly conducted in August and September 2016, with a total of 15 recording sessions. Each recording session lasted between 30 and 35 min and employed a small action camera. The camera was positioned at the same spot in all sessions, and it was set to take one photograph at intervals of 60 s. The sessions took place at different times of the day and different days of the week, so as to capture the varying conditions of the urban and

In order to analyse the bodily, social and spatial interactions around the benches, the people recorded in the area of study were classified in three main categories: players, participants and spectators. Each of these categories tries to address the distinct ways in which people seemingly experience the art installations, both from a bodily, social and spatial perspective.

The category of players describes people who were observed engaging in any kind of performative movements with Jeppe Hein's benches. Playing, in this case, denotes a conspicuously

settings [24].

2017

224

social setting.

3.2.1. Players

3.2. Social roles around the installations

Some people were recorded at the benches for a relatively long time—typically more than 3 min—though they were not regarded as players. Unlike players, the category of participants describes those who did not use the benches to engage in performative or exploratory movements, but rather were there primarily to chat and/or to rest. The term participant is borrowed from literature on HCI and public interfaces, where this category has been used to describe subtler forms of engagement between an individual and an interactive installation [32, 33].

Figure 5. Different motivations among players: playing for the sake of playing (left) and playing to be photographed (right).

Participants' behaviour points to a significantly different relationship to the art installations and the space around them, with respect to the bodily, the social and the experiential realms. If, for players, the benches stand out as urban sculptures calling for sensorimotor explorations, participants seem to regard Jeppe Hein's artful installations as a set of conveniently located urban furniture, where one can sit to wait for a friend, use the mobile or simply to enjoy the panorama of river Thames while resting the legs or chatting with another participant.

Nevertheless the distinction between players and participants is not always straightforward. Part of the recorded people manifested postures and behaviours seemingly located somewhere between the performative actions of players and the more relaxed forms of engagement commonly found among participants (Figure 6).

In such cases, time was the decisive factor to characterise a person as a player or as a participant: for the sake of the analysis, interactions sustaining the same body position for more than 1 min were regarded as participation, whereas the more fleeting bodily interactions, that is, those lasting less than 1 min were classified as play. The threshold of 1 min to distinguish players from participants is methodologically informed, as it coincides with the intervals of the time-lapse photography used in the study.

#### 3.2.3. Spectators

The category of spectators includes people who were observed in the immediate vicinity of a bench (up to around 4 m) while satisfying the following criteria: (a) they were not captured in a walking position; (b) their attention is visibly turned to the bench or, alternatively and (c) they form part of a pair or a group, of which at least one other member is physically engaged with the bench. During the analysis of the photographs a specific class of spectators stood out: those who were noticeably capturing images of people engaging with the installations. Even though this latter class of spectators could form a category of their own—arguably that of "active spectators", since they are assigned to a functional, specific social role in the context—they were subsumed under the category of spectators so as to better fit the broader scale of the study, namely the urban scale of place (Figure 7).

For the purposes of this study, the most important characteristic shared by spectators is that, although referring to people who are not physically engaged with the art installations at a given time, spectators are potential players, that is, they may become active players, either in a

Figure 6. Bodily engagements at the boundary between participation and play.

Figure 7. Multiple social roles in a snapshot: two persons sit (participants); a kid climbs on the right of the bench (player); four spectators stand nearby (two at the front and a couple behind the bench). Four passers-by can also be seen on their way.

matter of seconds or in their possible next visits to the area. What distinguishes spectators from mere passers-by are the bodily, temporal and social relationships between the person, the art installation and the place. Firstly, beholding implies some duration—for this study, at least a few seconds. It also entails bodily and social dimensions: it is a physical posture, a gesture which signals to nearby people that something interesting might be going on. By gazing at the benches, spectators not only stimulate passers-by to do the same. In fact, spectators help feed the whole cycle of social roles around the installations: some of the passers-by may stop around to observe or photograph the benches, and part of these people may go one step further and become players, eventually creating a "social buzz" in the place [27].

Especially during busy recording sessions, several people were observed at the vicinities of the benches, either leaning against the railings or sitting on the ledge beneath them (Figure 8).

Figure 8. People standing and sitting by the railings.

Participants' behaviour points to a significantly different relationship to the art installations and the space around them, with respect to the bodily, the social and the experiential realms. If, for players, the benches stand out as urban sculptures calling for sensorimotor explorations, participants seem to regard Jeppe Hein's artful installations as a set of conveniently located urban furniture, where one can sit to wait for a friend, use the mobile or simply to enjoy the

Proceedings of the Conference on Design and Semantics of Form and Movement - Sense and Sensitivity, DeSForM

Nevertheless the distinction between players and participants is not always straightforward. Part of the recorded people manifested postures and behaviours seemingly located somewhere between the performative actions of players and the more relaxed forms of engagement

In such cases, time was the decisive factor to characterise a person as a player or as a participant: for the sake of the analysis, interactions sustaining the same body position for more than 1 min were regarded as participation, whereas the more fleeting bodily interactions, that is, those lasting less than 1 min were classified as play. The threshold of 1 min to distinguish players from participants is methodologically informed, as it coincides with the intervals of the

The category of spectators includes people who were observed in the immediate vicinity of a bench (up to around 4 m) while satisfying the following criteria: (a) they were not captured in a walking position; (b) their attention is visibly turned to the bench or, alternatively and (c) they form part of a pair or a group, of which at least one other member is physically engaged with the bench. During the analysis of the photographs a specific class of spectators stood out: those who were noticeably capturing images of people engaging with the installations. Even though this latter class of spectators could form a category of their own—arguably that of "active spectators", since they are assigned to a functional, specific social role in the context—they were subsumed under the category of spectators so as to better fit the broader scale of the

For the purposes of this study, the most important characteristic shared by spectators is that, although referring to people who are not physically engaged with the art installations at a given time, spectators are potential players, that is, they may become active players, either in a

panorama of river Thames while resting the legs or chatting with another participant.

commonly found among participants (Figure 6).

time-lapse photography used in the study.

study, namely the urban scale of place (Figure 7).

Figure 6. Bodily engagements at the boundary between participation and play.

3.2.3. Spectators

2017

226

Importantly, most of these individuals remained at the same spot for several minutes, and they were not identified as part of a group engaged with the benches. In such cases, these individuals were not included as spectators, because their behaviours seem to best characterise the act of resting or waiting—but one that would apparently take place irrespective of the presence of the benches.

#### 3.3. Main findings

In order to present and analyse the data collected on-site, the four versions of the "Modified Social Benches" are identified as A, B, C and D, as shown in Figure 9:

A first glance at the striking shapes above suggests that each design affords different types of bodily and social behaviour. Likewise, each version of the bench may also result in particular forms and levels of aesthetic interaction. This section aims to clarify how these different forms of aesthetic interaction may affect the dynamics of placemaking. Chart 1 presents a synthesis of the findings, with the distribution of players, participants and spectators around each bench throughout the 15 recording sessions of the study.

The data above reveal that benches A and D share a similar character: in addition to encouraging playful, performative movements, these benches also succeeded in attracting a significant number of spectators around them. An average of 22.67 players per session were registered on bench A, and in only one session (7th September) no players attended. Bench D achieved a slightly higher average of players (22.87 per session), who were also consistently present throughout the study. As for spectators, bench A attracted the largest number of them —an average of 52.87 spectators per session—which, in most occasions, surpassed the number

Figure 9. The four versions of the benches analysed in the study.

of players and participants altogether. Bench D saw an average of 32.40 spectators, and their distribution was not so dominant as that verified in bench A.

Importantly, most of these individuals remained at the same spot for several minutes, and they were not identified as part of a group engaged with the benches. In such cases, these individuals were not included as spectators, because their behaviours seem to best characterise the act of resting or waiting—but one that would apparently take place irrespective of the presence of

Proceedings of the Conference on Design and Semantics of Form and Movement - Sense and Sensitivity, DeSForM

In order to present and analyse the data collected on-site, the four versions of the "Modified

A first glance at the striking shapes above suggests that each design affords different types of bodily and social behaviour. Likewise, each version of the bench may also result in particular forms and levels of aesthetic interaction. This section aims to clarify how these different forms of aesthetic interaction may affect the dynamics of placemaking. Chart 1 presents a synthesis of the findings, with the distribution of players, participants and spectators around each bench

The data above reveal that benches A and D share a similar character: in addition to encouraging playful, performative movements, these benches also succeeded in attracting a significant number of spectators around them. An average of 22.67 players per session were registered on bench A, and in only one session (7th September) no players attended. Bench D achieved a slightly higher average of players (22.87 per session), who were also consistently present throughout the study. As for spectators, bench A attracted the largest number of them —an average of 52.87 spectators per session—which, in most occasions, surpassed the number

Social Benches" are identified as A, B, C and D, as shown in Figure 9:

throughout the 15 recording sessions of the study.

Figure 9. The four versions of the benches analysed in the study.

the benches.

2017

228

3.3. Main findings

There is an important qualitative difference that helps explain the quantitative data outlined above. Among the players who engaged with bench A, most of them were children, who found in the slope-shaped installation a particularly amusing affordance for sliding. Thus, for the larger part of the players on bench A, the installation was literally a toy, a piece of

Chart 1. Distribution of social roles for each version of the benches.

playground. And, as one would expect to find in an ordinary urban playground, these young players were accompanied by their parents, relatives or friends, most of whom added up to the number of spectators around the bench (Figure 10).

Modified Social Benches: Exploring the Role of Aesthetic Interaction to Placemaking http://dx.doi.org/10.5772/intechopen.71118 231

Figure 10. Social gatherings around bench A as a result of children playing.

Bench D, on the other hand, presented a rather distinct profile of players. Children, adolescents and young adults likewise experienced bench D by performing a wide variety of corporeal movements and postures. And, while the steep shape of bench A imposed serious difficulties for more than one person to play at a time, the circular design of bench D allowed two, three or even more players to explore the installation simultaneously (Figure 11).

All these factors contributed to a smaller proportion of spectators around bench D if compared to bench A. In fact, with regard to the social roles, the design of bench D proved to be the most versatile one among the four versions of benches analysed here, as, in most sessions, the three social roles described in this study—players, participants and spectators—were recorded around bench D (Chart 1).

Benches B and C also function as platforms for social encounters, but they share a different character if compared to benches A and D. As shown in Chart 1B and C, the large majority of people recorded at benches B and C were qualified as participants. With regard to Aesthetic Interaction, this finding points to a very different experience, in terms of its bodily, social and spatial aspects. Firstly, the participants recorded at benches B and C found themselves in relaxed sitting positions: their bodies were laying on Jeppe Hein's art installations as they would be on any other conventional bench. Secondly, many of these participants remained sat for several minutes, while resting alone or talking to other people in different social formations (Figure 12).

Thirdly, the spatial and social logic of the behaviours observed around benches B and C can be described as self-contained and contemplative: people did not feel compelled to engage in any

Figure 11. A young couple (left) and a group of children (right) playing simultaneously with bench D.

playground. And, as one would expect to find in an ordinary urban playground, these young players were accompanied by their parents, relatives or friends, most of whom added up to the

Proceedings of the Conference on Design and Semantics of Form and Movement - Sense and Sensitivity, DeSForM

2017

230

number of spectators around the bench (Figure 10).

Chart 1. Distribution of social roles for each version of the benches.

Figure 12. A mother rests on bench B (left) while the "social character" is evident at bench C (right).

sort of play with benches B and C, and only a few of them were recorded as spectators, with averages of 8.27 spectators for bench B and 12.33 for bench C (Chart 1B and C).

### 4. Discussion

The main findings summarised above suggest that some correlations between Aesthetic Interaction and placemaking might exist. It was argued that a place consists of a spatial setting where people feel stimulated to go and to stay. Then, based solely on Chart 1A–D, one is tempted to conclude that bench C is the most successful in terms of placemaking, simply because bench C attracted the largest number of people (average of 110.80 people per session). However, the data compiled on Chart 1A–D refers to discrete occurrences of interaction, and not discrete persons, which means that the same individual recorded for, say, 8 min resting on a bench corresponds to eight participants. Considering that many participants captured at bench C remained sat for several minutes, in fact the total number of different participants on bench C was significantly lower than that presented on Chart 1C. Hence the placemaking qualities of bench C are twofold: on the one hand, by allowing several people to sit in a variety of social formations, bench C does contribute to placemaking, with a design that encourage people to stay at the place often for a relatively long time while resting and/or chatting (Figure 13).

Figure 13. A group of ladies with a baby settle on bench C: from 12.03 pm (left) to 12.30 pm (right) these participants remained unchanged.

On the other hand, precisely because bench C encourages people to stay on it, other people often find no space to sit, interact or socialise on bench C, thus limiting the potential of this bench to create more fleeting modalities of social encounters.

Bench B shares with bench C the elementary character of "a bench to sit on". Nonetheless, the study shows that, compared to bench C, bench B presents a rather distinct quality with regard to social gatherings. The sitting surfaces of bench B are curved and discontinuous; as a result, bench B is not as comfortable as bench C, and it accommodates only two individuals at a time —whereas up to six people were found simultaneously on bench C. Encouraging neither extended permanence nor different social arrangements, the design of bench B turns out to be the weakest of the four analysed versions with respect to placemaking. In addition, Bench B resists easy classification: although visually striking, its design makes bench B something inbetween urban sculpture and urban furniture. In light of Aesthetic Interaction, this generates a problematic situation where people do not feel sufficiently compelled to engage their bodies in playful, movement-based experiences, and, at the same time, they do not feel so much compelled to sit and to stay.

The case study suggests that benches A and D are the most effective in terms of placemaking. These benches attracted high numbers of people (averages of 84.2 and 68.53 people per session, respectively). Most importantly, benches A and D revealed the highest distribution of players, participants and spectators across the recording sessions, which means that benches A and D provided more variegated experiences of bodily and social interactions around them. Not displaying a highly dominant presence of participants—such as those verified on benches B and C—means that more people were given the opportunity to experience benches A and D in qualitatively different manners, such as playing, resting or observing the activities around the benches. Unlike bench B, the striking shapes of benches A and D are unambiguously aimed at playful, exploratory sensorimotor encounters. This versatile, playful and action-oriented character makes benches A and D work in favour of placemaking, by imbuing the urban space with a very particular identity and life through social activities.

### 5. Conclusion

sort of play with benches B and C, and only a few of them were recorded as spectators, with

Proceedings of the Conference on Design and Semantics of Form and Movement - Sense and Sensitivity, DeSForM

The main findings summarised above suggest that some correlations between Aesthetic Interaction and placemaking might exist. It was argued that a place consists of a spatial setting where people feel stimulated to go and to stay. Then, based solely on Chart 1A–D, one is tempted to conclude that bench C is the most successful in terms of placemaking, simply because bench C attracted the largest number of people (average of 110.80 people per session). However, the data compiled on Chart 1A–D refers to discrete occurrences of interaction, and not discrete persons, which means that the same individual recorded for, say, 8 min resting on a bench corresponds to eight participants. Considering that many participants captured at bench C remained sat for several minutes, in fact the total number of different participants on bench C was significantly lower than that presented on Chart 1C. Hence the placemaking qualities of bench C are twofold: on the one hand, by allowing several people to sit in a variety of social formations, bench C does contribute to placemaking, with a design that encourage people to stay at the place often for a relatively long time while resting and/or chatting (Figure 13).

Figure 13. A group of ladies with a baby settle on bench C: from 12.03 pm (left) to 12.30 pm (right) these participants

averages of 8.27 spectators for bench B and 12.33 for bench C (Chart 1B and C).

Figure 12. A mother rests on bench B (left) while the "social character" is evident at bench C (right).

4. Discussion

2017

232

remained unchanged.

This paper presented a case study of the Modified Social Benches, a set of sculptural benches designed by Jeppe Hein and installed in Central London through the Summer of 2016. After an extensive fieldwork employing time-lapse photography and observations, three main social roles were identified to describe people's behaviour around the art installations: players, participants and spectators. Each of these social roles refers to a specific level of bodily and aesthetic engagement with the benches, ranging from the performative actions of players to the more constrained attitudes of spectators.

The mapping and analysis of the diverse social encounters and bodily actions triggered by the art installations suggest a correlation between Aesthetic Interaction and placemaking. Drawing from the main findings, it is possible to argue that urban installations encouraging playful, action-oriented and sensorimotor encounters tend to be more effective to placemaking than

installations affording static modalities of social activities. If place is characterised as a lively setting offering different modalities of social encounters and meaningful experiences, then urban artefacts that provide people with a broader variety of social and sensorimotor experiences (like benches A and D presented in this study) seem to better define lively places in comparison to other types of artefacts (like benches B and C).

### Acknowledgements

The case study presented in this paper is part of a research funded by CNPq.

### Author details

Andre G. Afonso

Address all correspondence to: andre.afonso.13@ucl.ac.uk

The Bartlett School of Architecture, University College London, UK

### References


[8] Hauge A. Identity and place: A critical comparison of three identity theories. Architectural Science Review. 2007;50(1)

installations affording static modalities of social activities. If place is characterised as a lively setting offering different modalities of social encounters and meaningful experiences, then urban artefacts that provide people with a broader variety of social and sensorimotor experiences (like benches A and D presented in this study) seem to better define lively places in

Proceedings of the Conference on Design and Semantics of Form and Movement - Sense and Sensitivity, DeSForM

[1] Marshall P, Hornecker E. Theories of embodiment in HCI. In: Jewitt C, Price S, Brown B, editors. The SAGE Handbook of Digital Technology Research. SAGE Publications; 2013.

[2] Bacigalupi M. The craft of movement in interaction design. In: Proceedings of the Working Conference on Advanced Visual Interfaces (AVI'98); 25–27 May 1998; L'Aquila, Italy.

[3] Petersen M G et al. Aesthetic interaction—A pragmatist's aesthetics of interactive systems. In: Proceedings of the 5th Conference on Designing Interactive Systems (DIS '04);

[4] Wright P, Wallace J, McCarthy J. Aesthetics and experience-centered design. ACM Trans-

[5] Levisohn A, Schiphorst T. Embodied engagement: Supporting movement awareness in ubiquitous computing systems. Ubiquit. Learn. International Journal. 2011;3(4):97-112 [6] Casey E. How to get from space to place in a fairly short stretch of time. In: Feld S, Basso

[7] McCullough M. Digital ground: Architecture, pervasive computing, and environmental

01–04 August 2004; Cambridge, MA, USA. New York: ACM; 2004. p. 269–276.

actions on Computer-Human Interaction. 2008;15(4): Article 18

K, editors. Senses of Place. Santa Fe, NM: SAR Press; 1996. p. 13-52

comparison to other types of artefacts (like benches B and C).

Address all correspondence to: andre.afonso.13@ucl.ac.uk

New York: ACM; 1998. p. 174–184.

knowing. Cambridge, MA: MIT Press; 2004

The Bartlett School of Architecture, University College London, UK

The case study presented in this paper is part of a research funded by CNPq.

Acknowledgements

Author details

2017

234

Andre G. Afonso

References

p. 144-158

	- [23] Jacobs J. The Death and Life of Great American Cities. New York: Vintage; 1961
	- [24] Whyte WH. The social life of small urban spaces. Washington: Conservation Foundation; 1980
	- [25] Project for Public Spaces. What is placemaking? [Internet]. 2009. Available from: https:// www.pps.org/reference/what\_is\_placemaking/ [Accessed: 2017-05-30].
	- [26] Thwaites K, Simkins I. Experiential landscape: Revealing hidden dimensions of peopleplace relations. In: Thwaites K, et al, editors. Urban Sustainability through Environmental Design: Approaches to Time-People-Place Responsive Urban Spaces. London; New York: Routledge; 2007. p. 141-145
	- [27] Brignull H, Rogers Y. Enticing people to interact with large public displays in public spaces. In: Proceedings of IFIP International Conference on Human Computer Interaction (INTERACT '03); Zurich, Switzerland. IOS Press, IFIP; 2003. p. 17–24.
	- [28] Danish Architecture Center. Let's Play [Exhibition] 23 June-30. Copenhagen, Denmark: Danish Architecture Center; September 2016
	- [29] Danish Architecture Center. Charlotte Ammundsens Plads [Internet]. 2014. Available from: http://www.dac.dk/en/dac-life/danish-architecture-guide/copenhagen/charlotte-ammundsensplads/ [Accessed: 2017–05-31].
	- [30] Jeppe H. Modified Social Bench [Internet]. 2017. Available from: http://www.jeppehein. net/pages/project\_id.php?path=publics&id=232 [Accessed: 2017–03-03].
	- [31] Al Sayed K et al. Space Syntax Methodology. 4th ed. London: Bartlett School of Architecture, UCL; 2014
	- [32] Reeves S. Designing interfaces in public settings: Understanding the role of the spectator in human-computer interaction. Berlin: Springer Science & Business Media; 2011
	- [33] Wouters N et al. Uncovering the honeypot effect: How audiences engage with public interactive systems. In: Proceedings of the 11th Conference on Designing Interactive Systems (DIS '16); 04–08 June 2016; Brisbane, Australia. New York: ACM; 2016. p. 5–16.

**Provisional chapter**

### **MindFull: Tableware to Manipulate Sensory Perception and Reduce Portion Sizes and Reduce Portion Sizes**

**MindFull: Tableware to Manipulate Sensory Perception** 

DOI: 10.5772/intechopen.71115

Helen Andreae Helen Andreae Additional information is available at the end of the chapter

[23] Jacobs J. The Death and Life of Great American Cities. New York: Vintage; 1961

www.pps.org/reference/what\_is\_placemaking/ [Accessed: 2017-05-30].

(INTERACT '03); Zurich, Switzerland. IOS Press, IFIP; 2003. p. 17–24.

net/pages/project\_id.php?path=publics&id=232 [Accessed: 2017–03-03].

1980

2017

236

Routledge; 2007. p. 141-145

plads/ [Accessed: 2017–05-31].

ture, UCL; 2014

Danish Architecture Center; September 2016

[24] Whyte WH. The social life of small urban spaces. Washington: Conservation Foundation;

Proceedings of the Conference on Design and Semantics of Form and Movement - Sense and Sensitivity, DeSForM

[25] Project for Public Spaces. What is placemaking? [Internet]. 2009. Available from: https://

[26] Thwaites K, Simkins I. Experiential landscape: Revealing hidden dimensions of peopleplace relations. In: Thwaites K, et al, editors. Urban Sustainability through Environmental Design: Approaches to Time-People-Place Responsive Urban Spaces. London; New York:

[27] Brignull H, Rogers Y. Enticing people to interact with large public displays in public spaces. In: Proceedings of IFIP International Conference on Human Computer Interaction

[28] Danish Architecture Center. Let's Play [Exhibition] 23 June-30. Copenhagen, Denmark:

[29] Danish Architecture Center. Charlotte Ammundsens Plads [Internet]. 2014. Available from: http://www.dac.dk/en/dac-life/danish-architecture-guide/copenhagen/charlotte-ammundsens-

[30] Jeppe H. Modified Social Bench [Internet]. 2017. Available from: http://www.jeppehein.

[31] Al Sayed K et al. Space Syntax Methodology. 4th ed. London: Bartlett School of Architec-

[32] Reeves S. Designing interfaces in public settings: Understanding the role of the spectator in human-computer interaction. Berlin: Springer Science & Business Media; 2011

[33] Wouters N et al. Uncovering the honeypot effect: How audiences engage with public interactive systems. In: Proceedings of the 11th Conference on Designing Interactive Systems (DIS '16); 04–08 June 2016; Brisbane, Australia. New York: ACM; 2016. p. 5–16.

Additional information is available at the end of the chapter

http://dx.doi.org/10.5772/intechopen.71115

#### **Abstract**

Rising obesity levels across the world are a major threat to health, well-being and the economy. Reducing the amount we eat is difficult. This is partly because consciously controlling our eating typically increases the amount we eat. The paper presents the MindFull tableware—a new design for tableware to help people to reduce portion sizes effectively and unconsciously. MindFull designs exploit a range of features of our sensory perception identified from psychological research literature. Initial experiments show encouraging results for the design and suggest several directions for future development, research and applications for the design findings.

**Keywords:** perception, portion size, tableware, sensory illusions, design psychology

### **1. Introduction**

Psychology's understanding of sensory errors and our eating experiences when combined with design can be used to help people control their portion sizes. This paper reports on some relevant aspects of psychology research, how they can be applied to the design of tableware, along with the results of preliminary tests of the tableware (**Figure 1**).

The worldwide prevalence of obesity has more than doubled since 1980 due to the increased availability of calorie dense food and the decreased need for physical activity. The increase in obesity is pulling on our society's resources both directly and indirectly. Cawley and Meyerhoefer [1] estimate just over 20% of US health expenditures are on obesity-related illness costing \$209.7 billion. Also, those who are obese are more likely to miss days of work, costing \$4.3 billion in absentee-related costs [2]. Many also struggle to perform some job demands, resulting in 4.2% loss in productivity in jobs such as manufacturing [2]. Obesity and depression are also strongly linked [3] adding to individuals' pain.

© 2016 The Author(s). Licensee InTech. This chapter is distributed under the terms of the Creative Commons

**Figure 1.** MindFull tableware, photo credit to Jessica Noone, Daphnee Belleil, Isla Davies and Lucy McMaster.

There are many strategies to lose weight: various exercise programmes, diets including low fat and low carbohydrate diets and reductions in meal frequency, to name a few. In the end, weight loss works if energy intake is less than energy output. One of the simplest and most effective ways of doing this is through the reduction of portion sizes [4]. Our standard portion sizes have been consistently increasing which makes it easy to mindlessly overeat [5]. Portion control can be hard, particularly when adherence requires attention. Unfortunately, consciously restraining eating makes you more likely to eat the food you are purposely avoiding and increases the risk of bingeing [5, 6]. However, when people are unaware, smaller portions have very little effect on reported satiety [7]. Portion size reduction lends itself to long-term adherence, particularly for people who are not actively restricting their food intake.

Importantly, outside extreme levels of hunger, satiety is more strongly influenced by our expectations and our memory than physiological signals [8]. There is a strong body of evidence supporting the impact of memory on our appetite and subsequent food intake. Simply remembering a recent meal can reduce how much we eat during the next meal [9, 10]. Our implicit memory of recent eating strongly affects satiety. Patients with anterograde amnesia would eat their normal meal up to three times in a row, when the control group ate one [11]. In addition, many studies have confirmed the effectiveness of recalling a recent meal, see Refs. [10, 12] for meta-analyses. The same was found in Wansink's bottomless soup bowl study, where bowls could self fill and empty [13]; regardless of the amount participants consumed, satiety was governed by how much they perceived they had eaten.

Expectations are equally as important as memory: if we expect a meal will satisfy then usually it will [14]. Labelling food as high calorie lets people feel fuller and eat less than the same food with no labelling [15]. Furthering this line of inquiry, Brunstrom et al. [16] tested whether expected satiety could be manipulated without drawing attention to it. Participants were shown either a large or small quantity of fruit under the guise of checking for allergies. They were given identical-sized smoothies. However, those who saw the larger amount of fruit felt more satisfied and reported lower hunger levels throughout the 3 hours of testing. Our expectations of satiety have little relation to actual calorie content: high-fat or dense-calorie foods are expected to be less filling per calorie than carbohydrates. For example, potatoes were rated as five times more filling than cashew nuts [17]. Because the volume-to-calorie ratio differs between foods, judging how filling food will be is difficult. Past experiences make it easier to accurately judge food that we are familiar with and have eaten to a level of fullness [17]. For unfamiliar foods and meals with several different foods, we tend to rely on a volume heuristic to judge quantity [18]. There is also a high correlation between familiar foods and satiety [17, 19].

These findings led the team of four undergraduate students, supervised by the author, to focus on creating designs which would manipulate users' expectations of the amount of food they were eating and help them pay attention when eating. To drive design decisions, the team chose to investigate common sensory perception errors.

Our sensory perception of volume is important when judging portion size. To enable faster thinking, our cognitive processes rely on heuristics of many kinds to quickly understand the world around us. From an early age, people estimate volume using height, width and length [20]. For young children to be accurate, the forms must be simple, as shown through Piaget's work with volume consistency [21, 22]. Surprisingly, our volume estimation of complex forms only becomes slightly better with experience [20]. Volume accuracy is much better with rectilinear and cylindrical forms [22, 23]; people find it much harder to judge the volume of complex and curvilinear forms we are unfamiliar with. The design implications are to cause perception errors in volume estimation by using non-standard, complex forms.

There are many strategies to lose weight: various exercise programmes, diets including low fat and low carbohydrate diets and reductions in meal frequency, to name a few. In the end, weight loss works if energy intake is less than energy output. One of the simplest and most effective ways of doing this is through the reduction of portion sizes [4]. Our standard portion sizes have been consistently increasing which makes it easy to mindlessly overeat [5]. Portion control can be hard, particularly when adherence requires attention. Unfortunately, consciously restraining eating makes you more likely to eat the food you are purposely avoiding and increases the risk of bingeing [5, 6]. However, when people are unaware, smaller portions have very little effect on reported satiety [7]. Portion size reduction lends itself to long-term adherence, particularly for people who are not actively restricting their food intake. Importantly, outside extreme levels of hunger, satiety is more strongly influenced by our expectations and our memory than physiological signals [8]. There is a strong body of evidence supporting the impact of memory on our appetite and subsequent food intake. Simply remembering a recent meal can reduce how much we eat during the next meal [9, 10]. Our implicit memory of recent eating strongly affects satiety. Patients with anterograde amnesia would eat their normal meal up to three times in a row, when the control group ate one [11]. In addition, many studies have confirmed the effectiveness of recalling a recent meal, see Refs. [10, 12] for meta-analyses. The same was found in Wansink's bottomless soup bowl study, where bowls could self fill and empty [13]; regardless of the amount participants consumed,

**Figure 1.** MindFull tableware, photo credit to Jessica Noone, Daphnee Belleil, Isla Davies and Lucy McMaster.

Proceedings of the Conference on Design and Semantics of Form and Movement - Sense and Sensitivity, DeSForM

2017

238

Expectations are equally as important as memory: if we expect a meal will satisfy then usually it will [14]. Labelling food as high calorie lets people feel fuller and eat less than the same food with no labelling [15]. Furthering this line of inquiry, Brunstrom et al. [16] tested whether expected satiety could be manipulated without drawing attention to it. Participants were shown either a large or small quantity of fruit under the guise of checking for allergies. They were given identical-sized smoothies. However, those who saw the larger amount of fruit felt more satisfied and reported lower hunger levels throughout the 3 hours of testing. Our expectations of satiety have little relation to actual calorie content: high-fat or dense-calorie foods are expected to be less filling per calorie than carbohydrates. For example, potatoes were rated as five times

satiety was governed by how much they perceived they had eaten.

Nonvisual senses are important. When containers are tall and thin visually, people tend to misjudge them as having a greater volume. But when using haptic senses, the bias is reversed [24]. This could be because, in the hand, the width dimension is more prominent compared to the height [25]. This same reversal was found in Pechey et al.'s investigations into the effect of 'glass shape' on volume judgments of wine [26]. Designs must consider both visual and haptic cues, using forms that feel wide, yet are visually perceived as large.

Weight impacts on volume perception. Heavier containers are perceived as larger [30]. Piqueras-Fiszman and Spence [27] found participants expected food to be more filling when presented in heavier but visually identical containers. Intriguingly, it is not only just straight mass that changes perception but also the effort exerted in our muscles [28]. It is important to note different manipulations allowed different cues to become more salient; holding an object to throw lets people feel more than just lifting it up and down, thus allowing the brain to account for the sensory input [28]. There are some odd aspects to weight heuristics: even when we have experience with an object, if it is large but light, we will still exert extra force when manipulating it [29]. If weight is being used to manipulate perception, the handling of the object must be considered.

The size-contrast illusion is also important: our size judgement is altered by comparisons with contextual cues. When applied to eating, participants (even nutrition experts) serve themselves more ice cream in a larger bowl than a small bowl [30]. In Mindless Eating [5], Wansink discussed experiments showing larger plate size can affect perception of portion size. However, Penaforte et al. [31] did an experiment with identical portion sizes of plain pasta on different sized flat plates and found plate size did not make any difference. Our team found these findings provocative, as they did not fit with previous literature. We postulated two possible explanations for the results. Firstly, pasta is a very familiar food to many people, making judging the portions easier. Secondly, a pile of pasta on a flat plate is a simple shape, so simple cognitive judgments can accurately estimate portion size. This highlights the need for complex and deceptive forms.

Three other relevant points emerged from the research. A smooth cube is perceived as being larger than a rough one [22]. Reducing eating speed results in lower food intake [12]. A cute/ baby schema helps to narrow and focus attention during unrelated tasks [32]. All of these could be factored into a design.

### **2. Design**

Many current designs overtly address the issue of portion control. For example, they simply visualise how big portions should be or create segmented plates (reminiscent toddler plates; **Figure 2**). The image in the bottom right is a piece of strategic design by Fajar Kurnia, Jeremy Chia and

**Figure 2.** Current portion control plates. Photo or Design Credit to: (from top left to bottom right) Health.com, Precise Portions: Nutrition control system, Calorie Queens: 3D dinner Divider, Meal Measure: portion control plate, Zak's Moso: bamboo divided dinner plate, Halved by Fajar Kurnia, Jeremy Chia and Jo Djauhari.

Jo Djauhari which has a lovely playful approach but still does not address the psychological aspects of portion control. Problematically, these approaches remind people they are being limited in what they are eating; which makes it harder for people to sustain their portion control. Rather than this explicit attention-enhancing approach, Wansink argued [14] that we should seek.

*"…small changes in the eating environment (such as package downsizing, smaller dinnerware and reduced visibility and convenience) that can be easily implemented … to help solve mindless overeating."*

#### and claimed

Three other relevant points emerged from the research. A smooth cube is perceived as being larger than a rough one [22]. Reducing eating speed results in lower food intake [12]. A cute/ baby schema helps to narrow and focus attention during unrelated tasks [32]. All of these

Proceedings of the Conference on Design and Semantics of Form and Movement - Sense and Sensitivity, DeSForM

Many current designs overtly address the issue of portion control. For example, they simply visualise how big portions should be or create segmented plates (reminiscent toddler plates; **Figure 2**). The image in the bottom right is a piece of strategic design by Fajar Kurnia, Jeremy Chia and

**Figure 2.** Current portion control plates. Photo or Design Credit to: (from top left to bottom right) Health.com, Precise Portions: Nutrition control system, Calorie Queens: 3D dinner Divider, Meal Measure: portion control plate, Zak's Moso:

bamboo divided dinner plate, Halved by Fajar Kurnia, Jeremy Chia and Jo Djauhari.

could be factored into a design.

**2. Design**

2017

240

*"It is easier to change our food environment than to change our mind."*

The team created a range of concepts, each with the aim of making the food on the plate or bowl seem larger (**Figure 3**). The idea of a humped base to make the pile of food seem larger was developed further into the MindFull bowl and plate. The MindFull solution addresses the design implications from the literature review (**Figure 4**). It uses heavy materials to create a false sense of density, and the design is wide to activate the haptic cues for size. The round curved form and asymmetry makes it harder to judge the volume accurately. The shape of the bowl forces users to hold the bowl flat on their hand or stretch their grasp, which enhances the sensory cues for size. The lack of a raised rim obliges the user to scoop their food slowly to stop the food from spilling. Similarly, the angles of the bowl's interior make it easy to chase food around in circles; slowing consumption. The surface finish is smooth rather than textured, to add to the impression of size. Finally, the rounded forms have association with a baby schema, hopefully, activating the attention bias to avoiding overeating due to distraction (**Figure 5**).

**Figure 3.** Development. Photo credit to Jessica Noone, Daphnee Belleil, Isla Davies and Lucy McMaster.

**Figure 4.** Design solutions to manipulate sensory perception. Photo credit to Jessica Noone, Daphnee Belleil, Isla Davies and Lucy McMaster.

**Figure 5.** MindFull details. Photo credit to Jessica Noone, Daphnee Belleil, Isla Davies and Lucy McMaster.

### **3. Testing**

To see if there were indications that any of our assumptions might work, we put the design through two preliminary user-testing sessions.

#### **3.1. Test 1**

The first test was a between-subjects test comparing the portion sizes of self-served portions of rice in the MindFull bowl and plate with portion sizes in a standard bowl and plate.

#### *3.1.1. Participants*

A total of 44 undergraduate students between 18 and 21 years were approached at the University between 10 and 11:30 am to volunteer for 2 minutes in a food experiment. They were asked their current hunger levels: hungry, content or not hungry. Each group had an even split of gender and were matched on hunger levels.

#### *3.1.2. Apparatus*

**Figure 4.** Design solutions to manipulate sensory perception. Photo credit to Jessica Noone, Daphnee Belleil, Isla Davies

Proceedings of the Conference on Design and Semantics of Form and Movement - Sense and Sensitivity, DeSForM

**Figure 5.** MindFull details. Photo credit to Jessica Noone, Daphnee Belleil, Isla Davies and Lucy McMaster.

and Lucy McMaster.

2017

242

The testing environment consisted of a partitioned-off area within a studio environment (**Figure 6**). A video camera was set up to record the process. One table held a large bowl of rice with a serving spoon. The MindFull bowl and plate and a standard bowl and plate were placed out of sight behind a box. On the second table, a sheet of plastic was laid out with a camera on a tripod above to photograph the portion sizes.

**Figure 6.** Photograph during test one. Photo credit to Jessica Noone, Daphnee Belleil, Isla Davies and Lucy McMaster.

#### *3.1.3. Method*

Participants were asked their hunger level ("hungry", "content" or "not hungry"), handed a plate or bowl from either the standard set or the MindFull set and asked to serve themselves what they considered their normal portion of rice. Once they had served themselves, the rice was tipped out and photographed from above with the chosen hunger level label and a small note of what plate or bowl it came from. The participant was thanked and given a small sweet. This process was repeated for each participant.

#### *3.1.4. Results*

The photographs of the rice portions were collated and processed by dish type to create **Figure 7**. The number of spoonfuls each participant served was recorded and independent sample t-tests were conducted to compare. There was a significant difference between the amount of food served in the MindFull bowl (M = 2.5, SD = 1.11) and the standard bowl (M = 4.8, SD = 1.09) conditions; t [15] = 3.23, p = 0.003. However, there was no significant difference between the amount of food served in the MindFull plate (M = 2.62, SD = 1.09) and the standard plate (M = 3.5, SD = 1.65) conditions; t [12]=1.60, p = 0.12.

#### *3.1.5. Limitations*

These results indicate aspects of the MindFull bowl may be effective; however, there are many limitations to study. It would have been much better to conduct within-subject experiments or to have matched the participants in each group by size and BMI. More accurate measurements of the servings also would have given more accurate results.

**Figure 7.** Infographic showing the largest, smallest and average portion sizes served in each dish—made from photographs of the rice portions.

#### **3.2. Test two**

*3.1.3. Method*

2017

244

*3.1.4. Results*

*3.1.5. Limitations*

photographs of the rice portions.

This process was repeated for each participant.

standard plate (M = 3.5, SD = 1.65) conditions; t [12]=1.60, p = 0.12.

ments of the servings also would have given more accurate results.

Participants were asked their hunger level ("hungry", "content" or "not hungry"), handed a plate or bowl from either the standard set or the MindFull set and asked to serve themselves what they considered their normal portion of rice. Once they had served themselves, the rice was tipped out and photographed from above with the chosen hunger level label and a small note of what plate or bowl it came from. The participant was thanked and given a small sweet.

Proceedings of the Conference on Design and Semantics of Form and Movement - Sense and Sensitivity, DeSForM

The photographs of the rice portions were collated and processed by dish type to create **Figure 7**. The number of spoonfuls each participant served was recorded and independent sample t-tests were conducted to compare. There was a significant difference between the amount of food served in the MindFull bowl (M = 2.5, SD = 1.11) and the standard bowl (M = 4.8, SD = 1.09) conditions; t [15] = 3.23, p = 0.003. However, there was no significant difference between the amount of food served in the MindFull plate (M = 2.62, SD = 1.09) and the

These results indicate aspects of the MindFull bowl may be effective; however, there are many limitations to study. It would have been much better to conduct within-subject experiments or to have matched the participants in each group by size and BMI. More accurate measure-

**Figure 7.** Infographic showing the largest, smallest and average portion sizes served in each dish—made from

This study took a qualitative approach. It used a within-subjects design to compare people's experiences with the MindFull dishes versus standard dishes.

#### *3.2.1. Participants*

Participants consisted of two males and two females all aged 19 years, who were not aware of the details of the project. The four participants were told they would eat two lunches on separate days and be asked about their satisfaction levels. They were asked to choose 2 days where they would wake up at the same time and to have similar breakfasts each of the days. They were also asked about any food allergies and preferences. In return for their time, they received two free lunches, which were eaten in the course of the testing.

#### *3.2.2. Apparatus*

The bowl and plate, designed by the MindFull team, were used during the first test, and a standard white bowl and plate set were used in the second. A questionnaire asking about food consumption that morning, their satiety and satisfaction levels and their thoughts on the experience was given at the end of the testing process. A video camera was used in order to film the participant during the experiment. A knife, fork, table and chair were provided in a portioned-off section of a studio space. The meal provided was a fresh vegetable pasta salad with sliced cucumber and carrot offering a variety of flavours to reduce the fullness being reached due to flavour fatigue [33] (**Figure 8**).

#### *3.2.3. Method*

Participants were asked what time they had woken up, what they had for breakfast and what time they had eaten it. They were also asked to rate their hunger levels on a scale of

**Figure 8.** Food served in test two. Photo credit to Jessica Noone, Daphnee Belleil, Isla Davies and Lucy McMaster.

0–10: 0 = full, 4 = content, 8 = hungry and would eat if food was available, 10 = extremely hungry and would go out of the way to find food. The food was laid out on a table as shown in **Figure 8**, and they were invited to start eating. Participants were engaged in conversation while eating and were encouraged to verbalise how they felt during the process, employing aspects of the 'think out loud' method [34] (**Figure 9**). The first day, they ate from the MindFull bowl and plate. The second day, they use the standard set. The tests were 2 days apart. Once the participant finished the meal, they were asked to rate their fullness levels from 0 to 10, 0 = hungry, 10 = over full, 7 = comfortably full, their satisfaction levels, 0 = unsatisfied, 10 = very satisfied, 5 = neutral, and to add any comment about their experience. After completing the second test, they took part in a semi-structured interview asking their impressions of the design, how they would compare the two experiences?, would they like to have it in their home?, could they see themselves purchasing the design? and any frustrations or joys they experienced while eating.

#### *3.2.4. Results*

There was no difference in the satisfaction or fullness levels between MindFull and the standard tableware.

A deductive thematic analysis was done of the conversations, questionnaire comments and semi-structured interviews looking for comments related to the perception-based design features and opinions on the design. The videos of the sessions were also analysed for behaviour and body language.

**Figure 9.** Photographs taken during test two. Photo credit to Jessica Noone, Daphnee Belleil, Isla Davies and Lucy McMaster.

Generally people found the MindFull set more exciting and special.

*"Oh it's [Standard set] not as exciting this time"*

0–10: 0 = full, 4 = content, 8 = hungry and would eat if food was available, 10 = extremely hungry and would go out of the way to find food. The food was laid out on a table as shown in **Figure 8**, and they were invited to start eating. Participants were engaged in conversation while eating and were encouraged to verbalise how they felt during the process, employing aspects of the 'think out loud' method [34] (**Figure 9**). The first day, they ate from the MindFull bowl and plate. The second day, they use the standard set. The tests were 2 days apart. Once the participant finished the meal, they were asked to rate their fullness levels from 0 to 10, 0 = hungry, 10 = over full, 7 = comfortably full, their satisfaction levels, 0 = unsatisfied, 10 = very satisfied, 5 = neutral, and to add any comment about their experience. After completing the second test, they took part in a semi-structured interview asking their impressions of the design, how they would compare the two experiences?, would they like to have it in their home?, could they see themselves purchasing the design? and any frustrations or

Proceedings of the Conference on Design and Semantics of Form and Movement - Sense and Sensitivity, DeSForM

There was no difference in the satisfaction or fullness levels between MindFull and the stan-

A deductive thematic analysis was done of the conversations, questionnaire comments and semi-structured interviews looking for comments related to the perception-based design features and opinions on the design. The videos of the sessions were also analysed for behaviour

**Figure 9.** Photographs taken during test two. Photo credit to Jessica Noone, Daphnee Belleil, Isla Davies and Lucy

joys they experienced while eating.

*3.2.4. Results*

2017

246

dard tableware.

and body language.

McMaster.

*"It is interesting how a meal is displayed makes such a different to how much you want it"*

*"I really like the other plate [MindFull]… I would just want to have it displayed on my bench.., I particularly like the wood"*

*"The other one is too special for every day, I only get it out for special things."*

*"Funny, the food seemed more interesting in the other plate [MindFull]"*

There were only a few comments on the portion sizes, two participants thought the MindFull set had more food, one could not tell and the last thought it had less.

*"Is it more food this time? It feels like more" [Standard set]*

*"Do I have to finish all of it?" [MindFull]*

*"The bowl was quite fat looking I thought it was going to be too much" [MindFull]*

*"Is it the same portion sizes?"*

In their written comments about MindFull they said

*"I thought the bowl was super full and I wouldn't be able to finish, but it was actually a good serving size."*

*"It was nice, filled me up. Took my time and enjoyed it."*

Two participants mentioned how the different tableware changed what you could see of the food.

*"It was sooo round and cute, the food was more 'in' the bowl so you couldn't see how much there was."*

*"the food is more on top of this bowl, you know, you can see it."*

Interestingly, these were the two participants who had strongest opposing perceptions of portion size. The one who said you could not see how much food thought MindFull had a big portion size and the one who noted you could see the food in the standard set thought MindFull had a smaller portion size. This could indicate one read the portion size only as the food he could see, while the other added the bowl into the equation.

When looking at the behaviour, participants very seldom touched the MindFull set, but they often repositioned and held the edge of the standard set. They also tended to take more time getting the food onto the fork in the MindFull condition, and the amount of food on each forkful was generally less. This could be because the design features were effective, or it may be because they were more concerned about damaging the design. Three of the four participants commented they had to be careful not to spill the food when eating with MindFull.

*"There is a knack to eating with that one, but for me I don't mind 'cause it feels so much nicer"*

Participants also seemed more relaxed with the standard set; however, this may have been due to having completed the test before. A larger sample size controlled for order effects

would have been beneficial to address these issues. When commenting on the design two participants, expressed storage could be an issue as MindFull does not stack tightly. Three of them mentioned if they had the set they would want it on display. All of these findings only offer initial insights due to the small sample size.

### **4. Discussion**

People use a variety of heuristics to estimate size from a range of visual and haptic perceptions. These heuristics are not accurate in all circumstances and can lead to very inaccurate estimates at times. The MindFull bowl and plate were designed with a number of features that exploited the limitations of these heuristics to present an illusion of larger portion sizes in order to assist people to unconsciously reduce and control their eating.

The experiments reported in this paper give a preliminary indication that the design may be successful. In particular, the MindFull bowl merits further investigation, as it significantly alters the perceived portion sizes in comparison to a normal bowl. However, more rigorous testing in a greater range of situations and with larger sample sizes would be needed to confidently validate the design.

The MindFull plate did not appear to be as effective. We postulate this may be because the mound on the MindFull plate is visually prominent, resulting in users being more aware of it and not deceived by the illusory perceptual consequences. The MindFull bowl, on the other hand, has a less visually prominent mound, making it much harder for the user to be aware the perceptual manipulation. The food also hid the shape of the internal walls making it difficult to judge the wall thickness and know how much was food and how much was bowl.

Furthermore, the shape of the MindFull bowl meant less of the food portion was visible than on the MindFull plate, so the container itself had a more dominant effect on the perception. The bowl also incorporated more identified design implications than the plate. It had a more unconventional and complex shape, which may have directly defeated many of the standard heuristics for estimating size. Unlike the plate, the bowl gave a haptic sensation of a larger container, as it was unwieldy when held in one hand, adding to the illusion of a big portion size. The difference between the weight of the MindFull bowl and the standard bowl was much higher than the difference in weight between the MindFull plate and the standard plate. This could mean the illusion of the weight-to-size ratio was stronger in the bowl making it more successful. Since the weight-to-size heuristic is particularly robust against cognitive awareness, we suggest this factor was particularly important in preventing people from using slower (but more accurate) thinking processes to overcome the deceptive sensory signals. To determine which of the strategies were most effective, the variables would need to be isolated in further testing.

The way participants served their own food in the first study was a significant limitation because they could feel the weight of the food in the serving spoon as well as count the number of spoonfuls. This process probably reduced the effectiveness of the sensory manipulations by the bowl or the plate. To truly test the effects of the design features, future experiments should modify the way the food is served to remove this limitation. However, serving food would happen in regular use of the tableware, which suggests a further design opportunity to create a weighted and/or complex shaped serving spoon that employs similar sensory illusions. Additionally, after eating, the shape of the bowl is made visible, so the perceptual illusions are more obvious, possibly inducing the user to reassess their expectations. This may have been a factor in one of the comments in the second study: "I thought the bowl was super full, and I wouldn't be able to finish, but it was actually a good serving size". It is unclear what effect this post-eating change in expectations would have on satiety, but it could be a severe design limitation.

The results of the second study were not as rigorous but did give indications for design improvements. Generally, the responses and satiety levels did not contradict the effectiveness of the MindFull approach. A big flaw in this study was not having participants pick up and handle the tableware, since many of the features targeted haptic feedback. A study with more participants and a setup where tableware is handled would be beneficial.

Overall, we conclude it could still be worth exploring MindFull and additions to the MindFull range of items, such as serving spoons, cutlery, glasses and serving bowls. The MindFull range of tableware is but one design approach to helping people control their eating; yet it demonstrates the potential of good design in this area. Due to the enormous cost of obesity to our society, investment into design which could help us eat less is of immense worth.

### **Acknowledgements**

would have been beneficial to address these issues. When commenting on the design two participants, expressed storage could be an issue as MindFull does not stack tightly. Three of them mentioned if they had the set they would want it on display. All of these findings only

Proceedings of the Conference on Design and Semantics of Form and Movement - Sense and Sensitivity, DeSForM

People use a variety of heuristics to estimate size from a range of visual and haptic perceptions. These heuristics are not accurate in all circumstances and can lead to very inaccurate estimates at times. The MindFull bowl and plate were designed with a number of features that exploited the limitations of these heuristics to present an illusion of larger portion sizes

The experiments reported in this paper give a preliminary indication that the design may be successful. In particular, the MindFull bowl merits further investigation, as it significantly alters the perceived portion sizes in comparison to a normal bowl. However, more rigorous testing in a greater range of situations and with larger sample sizes would be needed to con-

The MindFull plate did not appear to be as effective. We postulate this may be because the mound on the MindFull plate is visually prominent, resulting in users being more aware of it and not deceived by the illusory perceptual consequences. The MindFull bowl, on the other hand, has a less visually prominent mound, making it much harder for the user to be aware the perceptual manipulation. The food also hid the shape of the internal walls making it difficult to judge the wall thickness and know how much was food and how much was bowl.

Furthermore, the shape of the MindFull bowl meant less of the food portion was visible than on the MindFull plate, so the container itself had a more dominant effect on the perception. The bowl also incorporated more identified design implications than the plate. It had a more unconventional and complex shape, which may have directly defeated many of the standard heuristics for estimating size. Unlike the plate, the bowl gave a haptic sensation of a larger container, as it was unwieldy when held in one hand, adding to the illusion of a big portion size. The difference between the weight of the MindFull bowl and the standard bowl was much higher than the difference in weight between the MindFull plate and the standard plate. This could mean the illusion of the weight-to-size ratio was stronger in the bowl making it more successful. Since the weight-to-size heuristic is particularly robust against cognitive awareness, we suggest this factor was particularly important in preventing people from using slower (but more accurate) thinking processes to overcome the deceptive sensory signals. To determine which of the strategies were most effective, the variables would need to be isolated

The way participants served their own food in the first study was a significant limitation because they could feel the weight of the food in the serving spoon as well as count the number of spoonfuls. This process probably reduced the effectiveness of the sensory manipulations by the bowl or the plate. To truly test the effects of the design features, future experiments should

in order to assist people to unconsciously reduce and control their eating.

offer initial insights due to the small sample size.

**4. Discussion**

2017

248

fidently validate the design.

in further testing.

I would like to thank the students who developed this project: Jessica Noone, Daphnee Belleil, Isla Davies and Lucy McMaster.

### **Author details**

Helen Andreae

Address all correspondence to: helen.andreae@vuw.ac.nz

Victoria University of Wellington, Wellington, New Zealand

### **References**

	- [3] Luppino FS, de Wit LM, Bouvy PF, Stijnen T, Cuijpers P, Penninx BWJH, et al. Overweight, obesity, and depression: A systematic review and meta-analysis of longitudinal studies. Archives of General Psychiatry. 2010 Mar;**67**(3):220-229
	- [4] Clark A, Franklin J, Pratt I, McGrice M. Overweight and obesity: Use of portion control in management. Australian Family Physician. 2010 Jun;**39**(6):407
	- [5] Wansink B. Mindless Eating: Why We Eat More Than We Think. New York: Bantam trade pbk, Bantam Books; 2007 x+292
	- [6] Polivy J, Coleman J, Herman CP. The effect of deprivation on food cravings and eating behavior in restrained and unrestrained eaters. The International Journal of Eating Disorders. 2005 Dec;**38**(4):301-309
	- [7] Rolls BJ, Roe LS, Meengs JS. Reductions in portion size and energy density of foods are additive and lead to sustained decreases in energy intake. The American Journal of Clinical Nutrition. 2006 Jan;**83**(1):11-17
	- [8] Herman CP, Polivy J. A boundary model for the regulation of eating. Research Publications-Association for Research in Nervous and Mental Disease. 1984;**62**:141-156
	- [9] Polivy J, Herman CP, Hackett R, Kuleshnyk I. The effects of self-attention and public attention on eating in restrained and unrestrained subjects. Journal of Personality and Social Psychology. 1986 Jun;**50**(6):1253-1260
	- [10] Higgs S. Memory and its role in appetite regulation. Physiology & Behavior. 19 May 2005;**85**(1):67-72
	- [11] Rozin P, Dow S, Moscovitch M, Rajaram S. What causes humans to begin and end a meal? A role for memory for what has been eaten, as evidenced by a study of multiple meal eating in amnesic patients. Psychological Science. 1998 Sep 1;**9**(5):392-396
	- [12] Robinson E, Almiron-Roig E, Rutters F, de Graaf C, Forde CG, Tudur Smith C, et al. A systematic review and meta-analysis examining the effect of eating rate on energy intake and hunger. The American Journal of Clinical Nutrition 2014 Jul;100(1):123-151
	- [13] Wansink B, Painter JE, North J. Bottomless bowls: Why visual cues of portion size may influence intake. Obesity Research. 2005 Jan;**13**(1):93-100
	- [14] Wansink B, Chandon P. Slim by design: Redirecting the accidental drivers of mindless overeating. Journal of Consumer Psychology. 2014;**24**(3):413-431
	- [15] Wooley SC. Physiologic versus cognitive factors in short term food regulation in the obese and nonobese. Psychosomatic Medicine. 1972 Feb;**34**(1):62-68
	- [16] Brunstrom JM, Brown S, Hinton EC, Rogers PJ, Fay SH. "Expected satiety" changes hunger and fullness in the inter-meal interval. Appetite. 2011 Apr;**56**(2):310-315
	- [17] Irvine MA, Brunstrom JM, Rogers PJ. Perceptions of the satiating efficacy of a range of common foods. Appetite. 2010 Aug;**55**(1):168-169
	- [18] Keenan GS, Brunstrom JM, Ferriday D. Effects of meal variety on expected satiation: Evidence for a "perceived volume" heuristic. Appetite. 2015 Jun 1;**89**:10-15

[19] Irvine MA, Brunstrom JM, Rogers PJ. Perceptions of the satiating efficacy of a range of common foods. Appetite. 2008 Nov;**51**(3):761

[3] Luppino FS, de Wit LM, Bouvy PF, Stijnen T, Cuijpers P, Penninx BWJH, et al. Overweight, obesity, and depression: A systematic review and meta-analysis of longitudinal studies.

Proceedings of the Conference on Design and Semantics of Form and Movement - Sense and Sensitivity, DeSForM

[4] Clark A, Franklin J, Pratt I, McGrice M. Overweight and obesity: Use of portion control

[5] Wansink B. Mindless Eating: Why We Eat More Than We Think. New York: Bantam

[6] Polivy J, Coleman J, Herman CP. The effect of deprivation on food cravings and eating behavior in restrained and unrestrained eaters. The International Journal of Eating

[7] Rolls BJ, Roe LS, Meengs JS. Reductions in portion size and energy density of foods are additive and lead to sustained decreases in energy intake. The American Journal of

[8] Herman CP, Polivy J. A boundary model for the regulation of eating. Research Publications-Association for Research in Nervous and Mental Disease. 1984;**62**:141-156

[9] Polivy J, Herman CP, Hackett R, Kuleshnyk I. The effects of self-attention and public attention on eating in restrained and unrestrained subjects. Journal of Personality and

[10] Higgs S. Memory and its role in appetite regulation. Physiology & Behavior. 19 May

[11] Rozin P, Dow S, Moscovitch M, Rajaram S. What causes humans to begin and end a meal? A role for memory for what has been eaten, as evidenced by a study of multiple

[12] Robinson E, Almiron-Roig E, Rutters F, de Graaf C, Forde CG, Tudur Smith C, et al. A systematic review and meta-analysis examining the effect of eating rate on energy intake

[13] Wansink B, Painter JE, North J. Bottomless bowls: Why visual cues of portion size may

[14] Wansink B, Chandon P. Slim by design: Redirecting the accidental drivers of mindless

[15] Wooley SC. Physiologic versus cognitive factors in short term food regulation in the

[16] Brunstrom JM, Brown S, Hinton EC, Rogers PJ, Fay SH. "Expected satiety" changes hunger and fullness in the inter-meal interval. Appetite. 2011 Apr;**56**(2):310-315

[17] Irvine MA, Brunstrom JM, Rogers PJ. Perceptions of the satiating efficacy of a range of

[18] Keenan GS, Brunstrom JM, Ferriday D. Effects of meal variety on expected satiation:

Evidence for a "perceived volume" heuristic. Appetite. 2015 Jun 1;**89**:10-15

meal eating in amnesic patients. Psychological Science. 1998 Sep 1;**9**(5):392-396

and hunger. The American Journal of Clinical Nutrition 2014 Jul;100(1):123-151

influence intake. Obesity Research. 2005 Jan;**13**(1):93-100

common foods. Appetite. 2010 Aug;**55**(1):168-169

overeating. Journal of Consumer Psychology. 2014;**24**(3):413-431

obese and nonobese. Psychosomatic Medicine. 1972 Feb;**34**(1):62-68

Archives of General Psychiatry. 2010 Mar;**67**(3):220-229

trade pbk, Bantam Books; 2007 x+292

Disorders. 2005 Dec;**38**(4):301-309

Clinical Nutrition. 2006 Jan;**83**(1):11-17

Social Psychology. 1986 Jun;**50**(6):1253-1260

2005;**85**(1):67-72

2017

250

in management. Australian Family Physician. 2010 Jun;**39**(6):407


Provisional chapter

#### **Felted Terrain: Interactive Textile Landscape; Transforming the Experience of Knitted Textile with Computation and Soft Electronics** Felted Terrain: Interactive Textile Landscape; Transforming the Experience of Knitted Textile with Computation and Soft Electronics

DOI: 10.5772/intechopen.71124

Yihyun Lim

Additional information is available at the end of the chapter Yihyun Lim

http://dx.doi.org/10.5772/intechopen.71124 Additional information is available at the end of the chapter

#### Abstract

This paper presents Felted Terrain, an interactive textile with embedded soft electronics, that creates a sensorial experience with tactility, sound, and esthetics. The project takes the traditional craft of knitting and applies computation at different points of processes, from pattern generation with parametric scripting to integration of conductive and flexible electronics for creating user interactivity. With digital design and fabrication tools, the sensor-embedded textile is produced to be experienced at the spatial level of the interior. The paper discusses the design processes of the project and the potentials of embedding unexpected interactivity to the everyday object of the knitted fabric to provide opportunities for multi-sensorial experiences.

Keywords: Interaction design, design methodology, computation design, esthetics of interaction, tangible interaction, haptic interaction, sensible interface, flexible electronics, soft computation

### 1. Introduction

Knitting, creating fabric from weaving of worsted fibers, has been with us since the old days. As a material at the scale of our body, we are familiar with its touch, use, and experience. Textiles have also been used at the scale of the interior as wall tapestries and affect our experience with the space through esthetics. Through its interlaced yarn and colors, textile wall hangings were important elements of story-telling, communicating stories to the inhabitants of the space [6].

With the advent of soft electronics, there is an opportunity to take the experience of the textiles to the next level of multi-sensoriality. This project aims to explore two aspects of 'craft research' in interactive textiles; the generative design/craft process of textile and creation of spatial experience

© 2017 The Author(s). Licensee InTech. This chapter is distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/3.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

© The Author(s). Licensee InTech. This chapter is distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/3.0), which permits unrestricted use, distribution, and eproduction in any medium, provided the original work is properly cited.

through public exhibition (reflective-on-action). The first part discusses the production process of an interactive textile using embedded soft electronics, computation, and generative design that "focus on aesthetics, personal expression, and the idea of play, as opposed to the prevalent utilitarian focus of wearable technology design on universal connectivity and productivity applications" [2]. The slow-paced craft process of the felted textile embraces the practice of'reflective-inaction', where final interaction design of the project was continuously designed and iterated throughout the process of making [11]. The second part of the project focuses on the experience of interactive textile in a spatial (exhibition) setting where it becomes a mediator of experiential elements, that transforms an everyday space into a multi-sensorial 'practice space' [5]. The combination of haptic, visual, and auditory experience that is placed throughout the exhibition space provides users to 'reflect-on-action' [11] and discover the various interactive elements of the textile.

### 2. Design, computation, and production of Felted Terrain: an interactive non-woven textile

Felted Terrain attempts to subvert the notion of primitive handcraft in knitted/felted textile through its integration of soft electronics, computation design, and fabrication method. Using traditional textile techniques such as knitting, embroidery, and felting, the project aims to create textile of an 'ambient display' [14] that sense, transmit data, and create spatial sensory effect by presenting information within a space through subtle changes in the background of awareness (in this case, sound) (Figure 1).

There are two parts to the creation of 'Felted Terrain'—designing of knit pattern to create threedimensional knit structures with parametric computation design tools, and making of the electronic textile "that incorporates capabilities for sensing, communication, and interconnection technology" [2] with soft computation. Felted Terrain is a result of this two-part process and aims to present a seamless integration of technology and interactive experience in a knitted woven textile. As a handcraft process, the slow pace of the design and production of the textile enables 'reflection-in-action' [11], to continuously reflect on the project as a whole and also at every stage of the process to ensure the integration of design, craft, and intended user interaction in the production of the interactive textile. The following documents every step of the design and making process, that involves both digital and manual methods.

Figure 1. Felted Terrain—an interactive sensorial textile that generates sound and visual graphics upon touch.

#### 2.1. Creating the pattern

through public exhibition (reflective-on-action). The first part discusses the production process of an interactive textile using embedded soft electronics, computation, and generative design that "focus on aesthetics, personal expression, and the idea of play, as opposed to the prevalent utilitarian focus of wearable technology design on universal connectivity and productivity applications" [2]. The slow-paced craft process of the felted textile embraces the practice of'reflective-inaction', where final interaction design of the project was continuously designed and iterated throughout the process of making [11]. The second part of the project focuses on the experience of interactive textile in a spatial (exhibition) setting where it becomes a mediator of experiential elements, that transforms an everyday space into a multi-sensorial 'practice space' [5]. The combination of haptic, visual, and auditory experience that is placed throughout the exhibition space provides users to 'reflect-on-action' [11] and discover the various interactive elements of the textile.

Proceedings of the Conference on Design and Semantics of Form and Movement - Sense and Sensitivity, DeSForM

2. Design, computation, and production of Felted Terrain: an interactive

Felted Terrain attempts to subvert the notion of primitive handcraft in knitted/felted textile through its integration of soft electronics, computation design, and fabrication method. Using traditional textile techniques such as knitting, embroidery, and felting, the project aims to create textile of an 'ambient display' [14] that sense, transmit data, and create spatial sensory effect by presenting information within a space through subtle changes in the background of

There are two parts to the creation of 'Felted Terrain'—designing of knit pattern to create threedimensional knit structures with parametric computation design tools, and making of the electronic textile "that incorporates capabilities for sensing, communication, and interconnection technology" [2] with soft computation. Felted Terrain is a result of this two-part process and aims to present a seamless integration of technology and interactive experience in a knitted woven textile. As a handcraft process, the slow pace of the design and production of the textile enables 'reflection-in-action' [11], to continuously reflect on the project as a whole and also at every stage of the process to ensure the integration of design, craft, and intended user interaction in the production of the interactive textile. The following documents every step of the design and

Figure 1. Felted Terrain—an interactive sensorial textile that generates sound and visual graphics upon touch.

non-woven textile

2017

254

awareness (in this case, sound) (Figure 1).

making process, that involves both digital and manual methods.

#### 2.1.1. Generating dynamic three-dimensional knit patterns with computation

As the initial inspiration from the project came from the rolling mossy landscape of Iceland (Figure 2), creating a three-dimensional pattern to express the soft terrain was the first step in the process. To generate pattern, a circle packing Grasshopper script for Rhino 3D was used. The size of the circles varied from large to small (which corresponds to the pitch of the musical notes). The script also allowed easy planning in mapping of conductive areas, as shown in Figure 3 (the red circles indicate where the conductive thread will be added to create a capacitive sensor tips). The parametrically generated pattern was then overlaid on a grid to translated these into a knitting pattern. The pattern used in this project was one of many iterations generated from the computational design tool, which opened up a wide possibility to quickly generate variations for different visual effects.

#### 2.1.2. Designing soft circuitry embroidery patterns

Using the same pattern, a soft circuit diagram was made to plan out the embroidery of conductive circuits on felted textile. The curved lines indicated the circuit that links all of the conductive tips (gray-shaded area) to the Lilypad microcontroller (Figure 3). Soft circuits were

Figure 2. Interpreting the mossy terrain of Iceland to a knitted pattern.

Figure 3. (Left) Circle-packing pattern generated from parametric scripts. (Center) The pattern was overlaid on a grid to create a knitting pattern, and red circles indicate where the conductive thread should be added during the knitting process to create capacitive sensor tips. (Right) Gray-shaded area indicates the conductive tips, with circuit path showing its connection to the central microcontroller placed on the bottom left corner.

embroidered along the designated pattern using conductive stainless steel thread. The Lilypad microcontroller and x-Bee wireless modules were also hand sewn using conductive thread, creating a full e-textile.

#### 2.2. Embedding conductivity to knitted textiles

#### 2.2.1. Knitting with two types of fibers

The knitting process involved using two different types of yarn, a regular wool yarn and a conductive stainless steel yarn. As seen in Figure 4, stainless steel yarn was knitted together at the tips of the bumps to give "electro-mechanical properties" that will enable the fabrication of complex textile with interactivity [1]. This allowed integration of flexible sensors to build electronic circuits on soft substrates, and enables a move away from traditional electronics, of using PCB boards and hard materials, to an exploration of emergent flexible materials to create interactive physical designs [2]. When connected to the microcontroller, these tips can be programmed to become touch sensitive through capacitance. In this project, a low-profile stainless steel yarn was used instead of visible silver or gold metallic threads in creating soft circuits, so it could be blended in with the wool fiber and create a seamless look, paving its way for unexpected interaction in the final stage.

To produce a textile at the scale of the interior, yet using a domestic scale of the knitting machine, the project was made in small size patches of 1 m 1 m as shown in Figure 5. Knitted patches were stitched together to create a large wall-sized piece woven textile with three-dimensional 'bump' forms. The process produced a loosely knit woven textile with areas of embedded conductivity where stainless steel yarn was added to the wool yarn during the knitting process. Hand stitched seams and stainless steel yarn at tips of the three-dimensional bumps were still visible at this stage—in order to create a seamless esthetics of a non-woven textile, the completed knit textile went through the multi-step process of wet-felting.

Figure 4. Knitting with two different types of yarn to create a conductive 3D structure knit. Conductive fibers are embedded in each tip of the three-dimensional forms.

Felted Terrain: Interactive Textile Landscape; Transforming the Experience of Knitted Textile with… http://dx.doi.org/10.5772/intechopen.71124 257

Figure 5. Knitting in 1 m 1 m patches to produce a spatial-scale of knitted textile.

#### 2.2.2. Transforming woven textile to non-woven: felting

embroidered along the designated pattern using conductive stainless steel thread. The Lilypad microcontroller and x-Bee wireless modules were also hand sewn using conductive thread,

Proceedings of the Conference on Design and Semantics of Form and Movement - Sense and Sensitivity, DeSForM

The knitting process involved using two different types of yarn, a regular wool yarn and a conductive stainless steel yarn. As seen in Figure 4, stainless steel yarn was knitted together at the tips of the bumps to give "electro-mechanical properties" that will enable the fabrication of complex textile with interactivity [1]. This allowed integration of flexible sensors to build electronic circuits on soft substrates, and enables a move away from traditional electronics, of using PCB boards and hard materials, to an exploration of emergent flexible materials to create interactive physical designs [2]. When connected to the microcontroller, these tips can be programmed to become touch sensitive through capacitance. In this project, a low-profile stainless steel yarn was used instead of visible silver or gold metallic threads in creating soft circuits, so it could be blended in with the wool fiber and create a seamless look, paving its

To produce a textile at the scale of the interior, yet using a domestic scale of the knitting machine, the project was made in small size patches of 1 m 1 m as shown in Figure 5. Knitted patches were stitched together to create a large wall-sized piece woven textile with three-dimensional 'bump' forms. The process produced a loosely knit woven textile with areas of embedded conductivity where stainless steel yarn was added to the wool yarn during the knitting process. Hand stitched seams and stainless steel yarn at tips of the three-dimensional bumps were still visible at this stage—in order to create a seamless esthetics of a non-woven

Figure 4. Knitting with two different types of yarn to create a conductive 3D structure knit. Conductive fibers are

textile, the completed knit textile went through the multi-step process of wet-felting.

creating a full e-textile.

2017

256

2.2. Embedding conductivity to knitted textiles

way for unexpected interaction in the final stage.

embedded in each tip of the three-dimensional forms.

2.2.1. Knitting with two types of fibers

Wet felting shrinks the knitted textile by 30–40% from its original size (Figure 6). The felting process is a result of the 'tangling of wool fibers', due to the reptile-like scales on the surface of individual fibers. These scales are the main reasons in giving traditional wool its abrasive texture. When washed with hot water, these scales interlock and become lighter and tighter causing shrinkage, becoming a non-woven felted textile (Figure 7). Using this natural mechanism, the above knitted textile went through the wet-felting process, where it was agitated multiple times in the washing machine with hot water and soap, until the knitted structure was no longer visible and became a homogenous texture of felted fabric. Afterwards the entire piece was formed, flattened and then air dried to retain the shape of the bumps.

#### 2.3. Actuating sound and visual interactivity

The previous section explored the use of computation as a tool to generate design for the threedimensional pattern of the knitted textile. Computation can go beyond the role of design tool, and become 'part of the designed things themselves' [12].

Figure 6. Felting process shrinks the woven wool textile by 30–40%. Felting transforms a woven knitted fabric into a seamless non-woven fabric.

Figure 7. Diagram of wool fibers and the interlocking mechanism of wool scales in felting.

The three-dimensional form of the felted textile is designed with an intention to draw users touch, squeeze, and stroke each of the bumps. In addition to the apparent visual and tactile experience of the textile, auditory experience was programmed to the textile as an output of the touch interaction. In this project, a Lilypad microcontroller, an Arduino variant that is designed to be easily integrated with flexible circuits on textiles through sewing with conductive threads, was used along with a wireless x-Bee module to transform the experience of the everyday textile into an interactive e-textile.

#### 2.3.1. Creating intuitive interaction: textile keyboard

The interaction had two parts to the design. First was leveraging on the expected behavior of people with textiles (especially since the bumpy form intrigued users to touch) and second was designing an interaction that is intuitive for users to find out the rules of the game after a short engagement. In this project, a simple music notes were assigned to selected bumps on the felted textile. The size of the bumps corresponded to the pitch of the notes, for example, large bump played a lower octave note, and the note/pitch of the sound would go up the scale as the bump sizes get smaller (Figures 8 and 9).

#### 2.3.2. Real-time visual feedback of sound and touch

The three-dimensional form of the felted textile is designed with an intention to draw users touch, squeeze, and stroke each of the bumps. In addition to the apparent visual and tactile experience of the textile, auditory experience was programmed to the textile as an output of the touch interaction. In this project, a Lilypad microcontroller, an Arduino variant that is designed to be easily integrated with flexible circuits on textiles through sewing with conductive threads, was used along with a wireless x-Bee module to transform the experience of the

Figure 7. Diagram of wool fibers and the interlocking mechanism of wool scales in felting.

Figure 6. Felting process shrinks the woven wool textile by 30–40%. Felting transforms a woven knitted fabric into a

Proceedings of the Conference on Design and Semantics of Form and Movement - Sense and Sensitivity, DeSForM

The interaction had two parts to the design. First was leveraging on the expected behavior of people with textiles (especially since the bumpy form intrigued users to touch) and second was designing an interaction that is intuitive for users to find out the rules of the game after a short engagement. In this project, a simple music notes were assigned to selected bumps on the felted textile. The size of the bumps corresponded to the pitch of the notes, for example, large

everyday textile into an interactive e-textile.

seamless non-woven fabric.

2017

258

2.3.1. Creating intuitive interaction: textile keyboard

In order to enable the textile as a simple sound keyboard, Lilypad microcontroller was loaded with a modified Capsense Arduino Code. The received serial data was then transferred to a

Figure 8. Lilypad microcontroller was sewn onto the felted fabric along with x-Bee module for wireless transmission of data to the main computer.

Figure 9. Using Arduino and Processing, touching of the conductive felted tips produced individual sound of piano notes, at the same time visualizing the pitch through real-time projection of the sine wave curve on the wall.

Processing script (adapted from MIT Media Lab High-Low Tech Group's Piano Code). With these codes each touch on the tips were translated into a sound, which was then played through a speaker placed in the room. To create a visual connection to the sound the textile was producing, a real-time projection of the sound pitch was projected on the wall, in a form of a sine curve. Through this tri-part experience, a tactile touch could be both felt acoustically and visually.

### 3. Interactive textile as mediator of experience: creating opportunities for 'practiced space' through exhibition

Felted Terrain was installed at the MIT Keller Gallery for open interaction with visitors and passersby. The exhibition context could influence how one experiences the material of the exhibit [9]. With the design of the layout, the exhibitor can guide the visitor to engage with the material in a specific way and order, adding additional interaction element to the whole experience (Figure 10).

The exhibition was designed to invite visitors to reflect-on-action [11], to unveil the experiences of the interactive textile by engaging with it step by step. The square felted textile was placed on a clear table with a spotlight providing visual focus. A circular shiny mirrored film was placed beneath the table to enlarge the presence of the felted textile as well as providing additional view of the textile (negative space of the three-dimensional pattern). Upon entry, the three-dimensional form and fuzzy texture of the felted textile lures users to engage in touch. Each touch of the capacitive sensor tips, produced different musical notes, which was played through embedded sound system in the gallery space. By touching various sized tips, users could gradually understand the connection between bump sizes and sound notes, where the pitches of the notes correspond to the size of the bumps. Over time, one could make the parallel analogy of the textile as musical keyboard, and many of the users started to play a tune with the felted textile bumps. Various touch gestures were observed during the exhibition—in addition to lightly tapping the bumps, other gestures such as stroking, squeezing, pressing, and pulling of the bumps were made. Regardless of the types of gestures, in this version of the

Figure 10. Exhibition of the textile—it was placed on a custom designed clear table with reflective film on floor.

Felted Terrain: Interactive Textile Landscape; Transforming the Experience of Knitted Textile with… http://dx.doi.org/10.5772/intechopen.71124 261

Figure 11. The tactility and three-dimensional design of the textile invites users to touch, which generates sounds and visualizes the pitch of the sound on a nearby wall.

textile, the touch was accompanied by real-time visualization of the sounds, which was a wall projection of a sine wave curve.

The interactive textile also became mediator of experiential elements, and as a result it created an opportunity for a 'practiced space' by bringing meaning to a static space [5], where one's actions and engagement with the textile produced different perception of space (from space of sound, space of visual movement, and space of tacility). The interactive textile influenced how people experience their surroundings (Figure 11).

### 4. Potentials/future applications

Processing script (adapted from MIT Media Lab High-Low Tech Group's Piano Code). With these codes each touch on the tips were translated into a sound, which was then played through a speaker placed in the room. To create a visual connection to the sound the textile was producing, a real-time projection of the sound pitch was projected on the wall, in a form of a sine curve. Through this tri-part experience, a tactile touch could be both felt acoustically and visually.

Proceedings of the Conference on Design and Semantics of Form and Movement - Sense and Sensitivity, DeSForM

3. Interactive textile as mediator of experience: creating opportunities for

Felted Terrain was installed at the MIT Keller Gallery for open interaction with visitors and passersby. The exhibition context could influence how one experiences the material of the exhibit [9]. With the design of the layout, the exhibitor can guide the visitor to engage with the material in a specific way and order, adding additional interaction element to the whole

The exhibition was designed to invite visitors to reflect-on-action [11], to unveil the experiences of the interactive textile by engaging with it step by step. The square felted textile was placed on a clear table with a spotlight providing visual focus. A circular shiny mirrored film was placed beneath the table to enlarge the presence of the felted textile as well as providing additional view of the textile (negative space of the three-dimensional pattern). Upon entry, the three-dimensional form and fuzzy texture of the felted textile lures users to engage in touch. Each touch of the capacitive sensor tips, produced different musical notes, which was played through embedded sound system in the gallery space. By touching various sized tips, users could gradually understand the connection between bump sizes and sound notes, where the pitches of the notes correspond to the size of the bumps. Over time, one could make the parallel analogy of the textile as musical keyboard, and many of the users started to play a tune with the felted textile bumps. Various touch gestures were observed during the exhibition—in addition to lightly tapping the bumps, other gestures such as stroking, squeezing, pressing, and pulling of the bumps were made. Regardless of the types of gestures, in this version of the

Figure 10. Exhibition of the textile—it was placed on a custom designed clear table with reflective film on floor.

'practiced space' through exhibition

experience (Figure 10).

2017

260

The Felted Terrain project explored the creation of different sensorial experiences to the everyday surface of textile through computation-based design and integration of soft electronics. The exploration described in this paper is just the beginning, as many variations, effects and experiences can be created. For simplification, there are three components to the project for further exploration. First is in the design of the esthetics through pattern making. Through generative computation design, many dynamic and diverse designs can be made for application on textile. When combined with traditional techniques in knitting (such as creating "bumps" in this project by adding and decreasing stitches) the parametric design tool can quickly generate iterations of design forms.

Second, design of the interaction can be explored further. The project engaged few sensorial experiences, from auditory to visual, however, multitudes of combinations of experiences can be designed. The sensorial output from the touching of felted bumps are unlimited; further exploration can be made to produce other outputs such as changes in lighting of the space, temperature, haptic vibration, etc. Interaction modes are another part of the interaction design that can be modified. Additional touch behaviors on fabrics, such as stroking, squeezing, pushing can be studied to embed actuating sensors (conductive areas) to encourage other modes of engagement with the textile.

Lastly, the scale of the textile experience can be varied, from a scale of the body to the scale of the interior and building, to produce different emotional effects and affect understanding of the space. Other production methods and tools, such as thermo-forming of non-woven textile on digitally produced forms (CNC milled form base as an example), or using an industrial scale knitting machine to scale up and automate the process can produce the soft, responsive, sculptural textile surface at a scale of the interior. The e-textile surface can also be modularized in a form of e-textile for application on larger surface. The placement of tiles can produce different overall visual patterns as well as sensory outputs (Figure 12).

Figure 12. Diagram of components to be further explored.

### 5. Conclusion

Second, design of the interaction can be explored further. The project engaged few sensorial experiences, from auditory to visual, however, multitudes of combinations of experiences can be designed. The sensorial output from the touching of felted bumps are unlimited; further exploration can be made to produce other outputs such as changes in lighting of the space, temperature, haptic vibration, etc. Interaction modes are another part of the interaction design that can be modified. Additional touch behaviors on fabrics, such as stroking, squeezing, pushing can be studied to embed actuating sensors (conductive areas) to encourage other

Proceedings of the Conference on Design and Semantics of Form and Movement - Sense and Sensitivity, DeSForM

Lastly, the scale of the textile experience can be varied, from a scale of the body to the scale of the interior and building, to produce different emotional effects and affect understanding of the space. Other production methods and tools, such as thermo-forming of non-woven textile on digitally produced forms (CNC milled form base as an example), or using an industrial scale knitting machine to scale up and automate the process can produce the soft, responsive, sculptural textile surface at a scale of the interior. The e-textile surface can also be modularized in a form of e-textile for application on larger surface. The placement of tiles can produce

different overall visual patterns as well as sensory outputs (Figure 12).

modes of engagement with the textile.

2017

262

Figure 12. Diagram of components to be further explored.

The Felted Terrain project is an exploration of creating a soft textile surface that creates rich interaction and activities between people, computer, and the physical space to be expressive, unexpected, and enjoyable.

The slow crafting process of designing and making the interactive multi-sensorial textile provided many moments of 'reflection-in-action' [9], where the designer-maker can reflect in each action of the process to inform the experience of the whole and design decisions of parts. The exhibition context provided opportunities for 'reflection-on-action' [9], by allowing visitors to unveil layers of experiences and figuring out the pattern (rule) of interaction by reflecting on past experiences, knowledge and actions.

Textiles have a "uniquely intimate relationship with the human body" [1]. We wear them as clothing and also live around them as interior furnishing. We are naturally drawn to touching and feeling the tactility of the textile. What further explorations can be made to create soft surfaces that intuitively draw people to feel, respond, and interact? Would production at various scales, from the body-scale wearable to the level of interior and building produce different experiential effect of the interactive textile surface?

Textiles, whether knit or woven, worn on body or hung in space, is ubiquitous in our lives. The everyday presence of textile surfaces and its application to create soft, fabric-based computers embody Mark Weiser's vision of ubiquitous computing [13]: providing functionality while disappearing discreetly into the soft surface of the 'textiled' space. The esthetics, material qualities, and flexibility of the textiles present large possibility for embedded computation [4], or as a medium to form 'computational composite' [9] to create a pervasive, playful, and theatrical interactive experience for all.

### Acknowledgements

Thanks to MIT Department of Architecture and Council for Arts at MIT for funding and support.

### Author details

Yihyun Lim Address all correspondence to: yihyun@mit.edu Massachusetts Institute of Technology, Cambridge, MA, USA

### References

[1] Berzowska J, Bromley M. Soft computation through conductive textiles. In: . Proceedings of the International Foundation of Fashion Technology Institutes Conference; 2007. pp. 12-15

	- [2] Berzowska J. Electronic textiles: Wearable computers, reactive fashion, and soft computation. Textile. 2005;3(1):58-75
	- [3] Buechley L, Hill BM. LilyPad in the wild: How hardware's long tail is supporting new engineering and design communities. In: Proceedings of the 8th ACM Conference on Designing Interactive Systems. 2010 Aug 16. ACM. pp. 199-207
	- [4] Buechley L, Eisenberg M. The LilyPad Arduino: Toward wearable engineering for everyone. IEEE Pervasive Computing. 2008;7(2):
	- [5] De Certeau M. The Practice of Everyday Life, translated by Steven Rendell. Berkeley: CA; 1980
	- [6] Dumitrescu D, Landin H, Vallgårda A. An interactive textile hanging: Textile, context, and interaction. Thinking. 2012;7:
	- [7] Hallnäs L, Redström J. Interaction Design: Foundations, Experiments. Textile Research Centre: Swedish School of Textiles, University College of Borås and Interactive Institute; 2006
	- [8] Niedderer K, Townsend K. Craft research: Joining emotion and knowledge. Design and Emotion 2010 (Proceedings). Chicago, USA: IIT; 2010. pp. 5-7
	- [9] Nimkulrat N. Hands-on intellect: Integrating craft practice into design research; 2012
	- [10] Perner-Wilson H, Buechley L, Satomi M. Handcrafting textile interfaces from a kit-of-noparts. In: Proceedings of the fifth international conference on Tangible, embedded, and embodied interaction, 2011 Jan 22, ACM. pp. 61-68
	- [11] Schön DA. Educating the Reflective Practitioner: Toward a New Design for Teaching and Learning in the Professions. Jossey-Bass; 1987
	- [12] Vallgårda A, Redström J. Computational composites. In: Proceedings of the SIGCHI conference on Human factors in computing systems, 2007 Apr 29, ACM. pp. 513-522
	- [13] Weiser M. Ubiquitous computing. In: ACM Conference on Computer Science. 1994. p. 418
	- [14] Wisneski C, Ishii H, Dahley A, Gorbet M, Brave S, Ullmer B, Yarin P. Ambient displays: Turning architectural space into an interface between people and digital information. In: International Workshop on Cooperative Buildings, 1998 Feb 25. Berlin, Heidelberg: Springer. pp. 22-32

**Provisional chapter**

### **Igeni: Reinforcing Hygiene Practices in Children Through Dynamic Products Through Dynamic Products**

**Igeni: Reinforcing Hygiene Practices in Children** 

DOI: 10.5772/intechopen.71122

Marta Taverna, Sara Colombo and Lucia Rampino Marta Taverna, Sara Colombo and Lucia Rampino Additional information is available at the end of the chapter

Additional information is available at the end of the chapter

http://dx.doi.org/10.5772/intechopen.71122

#### **Abstract**

[2] Berzowska J. Electronic textiles: Wearable computers, reactive fashion, and soft compu-

Proceedings of the Conference on Design and Semantics of Form and Movement - Sense and Sensitivity, DeSForM

[3] Buechley L, Hill BM. LilyPad in the wild: How hardware's long tail is supporting new engineering and design communities. In: Proceedings of the 8th ACM Conference on

[4] Buechley L, Eisenberg M. The LilyPad Arduino: Toward wearable engineering for every-

[5] De Certeau M. The Practice of Everyday Life, translated by Steven Rendell. Berkeley: CA;

[6] Dumitrescu D, Landin H, Vallgårda A. An interactive textile hanging: Textile, context,

[7] Hallnäs L, Redström J. Interaction Design: Foundations, Experiments. Textile Research Centre: Swedish School of Textiles, University College of Borås and Interactive Institute; 2006 [8] Niedderer K, Townsend K. Craft research: Joining emotion and knowledge. Design and

[9] Nimkulrat N. Hands-on intellect: Integrating craft practice into design research; 2012

[10] Perner-Wilson H, Buechley L, Satomi M. Handcrafting textile interfaces from a kit-of-noparts. In: Proceedings of the fifth international conference on Tangible, embedded, and

[11] Schön DA. Educating the Reflective Practitioner: Toward a New Design for Teaching and

[12] Vallgårda A, Redström J. Computational composites. In: Proceedings of the SIGCHI conference on Human factors in computing systems, 2007 Apr 29, ACM. pp. 513-522 [13] Weiser M. Ubiquitous computing. In: ACM Conference on Computer Science. 1994. p. 418 [14] Wisneski C, Ishii H, Dahley A, Gorbet M, Brave S, Ullmer B, Yarin P. Ambient displays: Turning architectural space into an interface between people and digital information. In: International Workshop on Cooperative Buildings, 1998 Feb 25. Berlin, Heidelberg:

Designing Interactive Systems. 2010 Aug 16. ACM. pp. 199-207

Emotion 2010 (Proceedings). Chicago, USA: IIT; 2010. pp. 5-7

embodied interaction, 2011 Jan 22, ACM. pp. 61-68

Learning in the Professions. Jossey-Bass; 1987

Springer. pp. 22-32

tation. Textile. 2005;3(1):58-75

1980

2017

264

one. IEEE Pervasive Computing. 2008;7(2):

and interaction. Thinking. 2012;7:

Igeni is a set of dynamic products designed to lead children towards autonomy in their personal hygiene practices. The set is composed of three objects: the Billy Brush toothbrush, the Wally Wash faucet-ring and the Fanny Flush toilet reminder. These interactive products are designed to enhance skills of personal hygiene in preschool children, thanks to sensors and actuators for multisensory communication. In this paper, we present the design process that led to the creation of this set of products and we describe the design outcome, consisting of a setoff objects aimed at exploiting the child's senses to convey messages, to create engaging experiences and to encourage healthy practices. We also present and discuss preliminary tests with users.

**Keywords:** dynamic products, non-verbal communication, sensory communication, health, hygiene habits, children

### **1. Introduction**

This paper describes Igeni, a set of dynamic products designed to educate children in preschool age (3–5 years) to be more autonomous in personal hygiene practices. The paper intends to add to the field of dynamic products through a practical design case, showing how dynamic sensory features can be designed to communicate with children and affect their behaviour. It identifies and explore a potential application field for dynamic products and provides design clues that can inspire the use of dynamic features in other contexts.

Parents daily struggle with reminding children of basic hygiene practices and a lot of time and effort is dedicated to teaching them how to perform these actions in an effective way. Results are difficult to check and different practices are carried out as control mechanisms

(e.g. visual inspection of hand and teeth, touching toothbrush to check if it is wet, etc.). These practices inspired us to explore how IoT and interaction design can be applied to the field of children's personal hygiene, in order to reinforce the educational process and to teach them to be more autonomous, while at the same time reducing parents' burdening.

We decided to explore this problem space by designing dynamic products that focus on sensory and non-verbal communication. In interaction design, diverse forms of non-verbal communication have started to be investigated recently, which regard alternative—more physical and sensory—languages for the communication of digital information to the user. Ambient interfaces [1, 2] and dynamic products are at the core of this design area. In particular, dynamic products are objects that show sensory features (visual, tactile, auditory and olfactory) that change proactively and in a reversible way over time, providing information to users by transforming their own sensory appearances (i.e. shape, colour, smell, light, sound, temperature, etc.) [3].

Dynamic products provide the possibility to convey information in an implicit and sensory way, and highlight the emergence of a new semantics, which understands the changing and dynamic appearances of products as communication means. Design can apply different strategies to effectively shape and map information into dynamic sensory features, like abstract disruption, translation, reproduction and metaphor, according to the context where the product will be used and to the communication goals ([4], p. 96). Moreover, communicating by senses seems to be a more engaging way to provide information to users and has the ability to create meaningful experiences and to affect the users' activities and practices ([4], p. 92).

However, we could not find any study in this area investigating how product's dynamic sensory features can be used to convey information or to encourage certain behaviours, having children as users, with a product and interaction design perspective. As dynamic products seem to be powerful means to emotionally engage users and affect their behaviour, we explored their potential in the field of personal hygiene in children, through a real case study.

This work intends to explore how dynamic products can be used to influence users' behaviour in a practical project, and it wants to expand their application field, by testing this approach in the area of design for children.
