**4. Discussion**

An effective analysis of the non-verbal behavior that accompanies turn-taking requires material that is authentic and rather informal than formal. The episode from the entertainment show *As ti tut not padu?* offers such a resource. As elaborated in the section *The EVA Corpus*, the notion that the material is highly spontaneous is corroborated by the video's statistics. The sheer amount of overlapping speech indicates a high frequency of turn management acts. The notion that the material is spontaneous and performed in a relaxed manner is further supported in Cooperrider's [27] foreground-background distinction, as the DAs are wellbalanced according to their nature. Overall, there are 1,897 foreground DAs and 2,020 background DAs. Therefore, more than half of the DAs are of background nature. This suggests that the material is not task-oriented, but instead serves the purpose of an entertainment show. Moreover, the most frequently observed NCI in the video were NCIs classified as regulators. The group of regulators represents 3/4 of all recognized NCIs in the entertainment episode. Regulators are followed by the group of deictic NCIs representing almost roughly 16 percent of the recognized NCIs. The remaining groups of illustrators, batons, symbols, and undetermined NCI each account for less than a tenth of the recognized NCIs. Again, the dominant NCI groups are of background nature, even if observing only regulators. These findings further support the notion that the material is highly spontaneous, relaxed, and entertaining. Therefore, it was a suitable choice for the analysis of natural turn-taking behavior and its accompanying non-verbal behavior.

Even though it seems relatively elementary to annotate turn management DAs, at times, the process proved to be complex. The acts of turn management are intertwined with stalling and instruction DAs. Examples 1 and 2 show how the stalling function (within the time management dimension) can also act as a turn-taking mechanism. In example 1, on the one hand, the co-host wants to take the floor since the guest suddenly released his turn. On the other hand, he does not know how to start. Hence, it is difficult to determine the primary DA, especially since the remainder of his response can be considered an information-providing DA. Example 2 illustrates how fillers, which generally pertain to the stalling function within the time dimension, act as turn-taking devices. They signal to the interlocutor that one wishes to speak but still requires additional time to properly verbalize one's thoughts. To further complicate the annotation of turntaking management, in a conversation, each utterance by another person can secondarily be considered a turn-taking DA. Even the act of posing a question, which would primarily be annotated within the information-seeking dimension, can secondarily be annotated with a turn-assign function (since person A, who is asking person B the question, wishes person B to respond). Nevertheless, we did not annotate such secondary cases of turn management as alternative DAs. Only DAs where turn management is key for an utterance were assigned the turn dimension. Consequently, the share of turn

**119**

*Can Turn-Taking Highlight the Nature of Non-Verbal Behavior: A Case Study*

management DAs could be significantly greater, and the ratio between foreground and background expressions notably tilted towards the background spectrum. Again, this only highlights the nature of the material, which is by no means primarily task-oriented. Since regulators and (partially) deictics are background expressions, we expected them to co-occur with turn management DAs. As noted, deictics are the elusive group of NCIs which can occur both in the foreground and the background. In Example 1, the explicit turn management DA is accompanied by a referential deictic. The semantic link with the word it refers to, however, is weak. The guest speaks of an abstract "this" in his utterance "well this is it". He does not refer to anything physically present in the room; he just points to a mental image. Therefore, the deictic is part of the background. This, in turn, is in line with the assigned DA,

A similar symmetry between the nature of the DA and the NCI is observed in Examples 2 and 3. In Example 2, the NCI can be considered both as a background and a foreground expression (see **Figure 3**: 1). Within the concept of DAs, it can also be considered as occurring in the background and in the foreground. In the background,

it is a turn-assigning DA with which the host hopes to receive a response to his nomenclature dilemma. In this case, the non-verbal behavior is perceived as spontaneous. Rather than to visualize the referential utterance *dajta* "come on" the speaker tries to emphasize his frustration with the interlocutors, i.e., if you know better then please explain, and thereby assigns the turn someone else. The 'open hand gesture' is observed to signal this. In the foreground, it has an instructing function, within the task dimension, since he demands a response. The host is referencing actual people in the room and due to the imperative use of the referential utterance *dajta* "come on you two", the observed conversational expression may be interpreted as instructing. *Dajta* is perceived as an explicit speech referent, and the non-verbal behavior seems to directly visualize it. Similarly, in Example 3, the NCI (see **Figure 4**: 3) is not a typical background referential deictic as the co-host physically leans towards the host while he nods towards him in order to prompt him to continue. The referent of the NCI is, therefore, an actual person (the host) in the room, and the NCI also occurs in the foreground. This duality is reflected in DAs as well. On the one hand, the co-host assigns the turn to the host (within the turn dimension); on the other, the co-host instructs the host to speak (within the task dimension). Again, this is an example of the difficulty in differentiating between background and foreground expressions. Finally, as hypothesized, regulators are the group of NCIs that co-occur with unambiguous turn management DAs. In Example 1, the secondary turn management DA co-occurs with the group of regulators, specifically a communication regulator. Whereas the group of deictic NCIs crosses the foreground-background barrier, regulators are background expressions. The fact that the accompanying NCI is a regulator and not a referent from the deictic group, which would be more typical for feedback elicitation, further endorses the assignment of a turn dimension DA and highlights the turn management intent. Example 3 illustrates a similar unambiguity when regulators are used for turn-taking. There is an underlying agreement of the nature of the DAs and the NCI in the last utterance "yes" (see **Figure 4**: 4). Namely, both take place in the background. And clearly, they were well understood by the interlocutors as no one else tried to take the turn. The opposite phenomenon is observed at the beginning of the same example (see **Figure 4**: 1) with the host's filler "uh". It is an example of a failed turn-take attempt since the co-host interrupts the host. There was no noticeable nonverbal behavior accompanying the host's filler, which is why no NCI could be assigned. However, one might argue that the fact that there is no NCI accompanying his turntake attempt, contributes to the reason of why the attempt failed. Consequently, this might be considered a supporting example of Birdwhistell's [22] findings that success-

ful communication requires both verbal and non-verbal components.

*DOI: http://dx.doi.org/10.5772/intechopen.95516*

since both concepts are background expressions.

#### *Can Turn-Taking Highlight the Nature of Non-Verbal Behavior: A Case Study DOI: http://dx.doi.org/10.5772/intechopen.95516*

*Types of Nonverbal Communication*

**4. Discussion**

co-host's interruption. As he turns to the co-host, he remains speechless and waits for him to elaborate. The co-host subsequent NCI, on the other hand, is a clear referential deictic (3) accompanying the DA of assigning the turn. His gaze towards the host was not enough to prompt a response, so he adds a firm head nod (see **Figure 4**: 3) towards him and verbalizes his intent of assigning the turn to him with "continue". This firm head nod is why the DA was not secondarily classified as a turn release, but as instructing within the task dimension. The host almost simultaneously responds to this NCI with a slight nod himself (see **Figure 4**: 4) and thereupon the verbal confirmation "yes". The nod was identified as a communication regulator NCI. The verbal confirmation functions as a turn accept DA. Both the DA and the NCI are of background nature.

An effective analysis of the non-verbal behavior that accompanies turn-taking requires material that is authentic and rather informal than formal. The episode from the entertainment show *As ti tut not padu?* offers such a resource. As elaborated in the section *The EVA Corpus*, the notion that the material is highly spontaneous is corroborated by the video's statistics. The sheer amount of overlapping speech indicates a high frequency of turn management acts. The notion that the material is spontaneous and performed in a relaxed manner is further supported in Cooperrider's [27] foreground-background distinction, as the DAs are wellbalanced according to their nature. Overall, there are 1,897 foreground DAs and 2,020 background DAs. Therefore, more than half of the DAs are of background nature. This suggests that the material is not task-oriented, but instead serves the purpose of an entertainment show. Moreover, the most frequently observed NCI in the video were NCIs classified as regulators. The group of regulators represents 3/4 of all recognized NCIs in the entertainment episode. Regulators are followed by the group of deictic NCIs representing almost roughly 16 percent of the recognized NCIs. The remaining groups of illustrators, batons, symbols, and undetermined NCI each account for less than a tenth of the recognized NCIs. Again, the dominant NCI groups are of background nature, even if observing only regulators. These findings further support the notion that the material is highly spontaneous, relaxed, and entertaining. Therefore, it was a suitable choice for the analysis of natural turn-taking behavior and its accompanying non-verbal behavior.

Even though it seems relatively elementary to annotate turn management DAs, at times, the process proved to be complex. The acts of turn management are intertwined with stalling and instruction DAs. Examples 1 and 2 show how the stalling function (within the time management dimension) can also act as a turn-taking mechanism. In example 1, on the one hand, the co-host wants to take the floor since the guest suddenly released his turn. On the other hand, he does not know how to start. Hence, it is difficult to determine the primary DA, especially since the remainder of his response can be considered an information-providing DA. Example 2 illustrates how fillers, which generally pertain to the stalling function within the time dimension, act as turn-taking devices. They signal to the interlocutor that one wishes to speak but still requires additional time to properly verbalize one's thoughts. To further complicate the annotation of turntaking management, in a conversation, each utterance by another person can secondarily be considered a turn-taking DA. Even the act of posing a question, which would primarily be annotated within the information-seeking dimension, can secondarily be annotated with a turn-assign function (since person A, who is asking person B the question, wishes person B to respond). Nevertheless, we did not annotate such secondary cases of turn management as alternative DAs. Only DAs where turn management is key for an utterance were assigned the turn dimension. Consequently, the share of turn

**118**

management DAs could be significantly greater, and the ratio between foreground and background expressions notably tilted towards the background spectrum. Again, this only highlights the nature of the material, which is by no means primarily task-oriented.

Since regulators and (partially) deictics are background expressions, we expected them to co-occur with turn management DAs. As noted, deictics are the elusive group of NCIs which can occur both in the foreground and the background. In Example 1, the explicit turn management DA is accompanied by a referential deictic. The semantic link with the word it refers to, however, is weak. The guest speaks of an abstract "this" in his utterance "well this is it". He does not refer to anything physically present in the room; he just points to a mental image. Therefore, the deictic is part of the background. This, in turn, is in line with the assigned DA, since both concepts are background expressions.

A similar symmetry between the nature of the DA and the NCI is observed in Examples 2 and 3. In Example 2, the NCI can be considered both as a background and a foreground expression (see **Figure 3**: 1). Within the concept of DAs, it can also be considered as occurring in the background and in the foreground. In the background, it is a turn-assigning DA with which the host hopes to receive a response to his nomenclature dilemma. In this case, the non-verbal behavior is perceived as spontaneous. Rather than to visualize the referential utterance *dajta* "come on" the speaker tries to emphasize his frustration with the interlocutors, i.e., if you know better then please explain, and thereby assigns the turn someone else. The 'open hand gesture' is observed to signal this. In the foreground, it has an instructing function, within the task dimension, since he demands a response. The host is referencing actual people in the room and due to the imperative use of the referential utterance *dajta* "come on you two", the observed conversational expression may be interpreted as instructing. *Dajta* is perceived as an explicit speech referent, and the non-verbal behavior seems to directly visualize it. Similarly, in Example 3, the NCI (see **Figure 4**: 3) is not a typical background referential deictic as the co-host physically leans towards the host while he nods towards him in order to prompt him to continue. The referent of the NCI is, therefore, an actual person (the host) in the room, and the NCI also occurs in the foreground. This duality is reflected in DAs as well. On the one hand, the co-host assigns the turn to the host (within the turn dimension); on the other, the co-host instructs the host to speak (within the task dimension). Again, this is an example of the difficulty in differentiating between background and foreground expressions.

Finally, as hypothesized, regulators are the group of NCIs that co-occur with unambiguous turn management DAs. In Example 1, the secondary turn management DA co-occurs with the group of regulators, specifically a communication regulator. Whereas the group of deictic NCIs crosses the foreground-background barrier, regulators are background expressions. The fact that the accompanying NCI is a regulator and not a referent from the deictic group, which would be more typical for feedback elicitation, further endorses the assignment of a turn dimension DA and highlights the turn management intent. Example 3 illustrates a similar unambiguity when regulators are used for turn-taking. There is an underlying agreement of the nature of the DAs and the NCI in the last utterance "yes" (see **Figure 4**: 4). Namely, both take place in the background. And clearly, they were well understood by the interlocutors as no one else tried to take the turn. The opposite phenomenon is observed at the beginning of the same example (see **Figure 4**: 1) with the host's filler "uh". It is an example of a failed turn-take attempt since the co-host interrupts the host. There was no noticeable nonverbal behavior accompanying the host's filler, which is why no NCI could be assigned. However, one might argue that the fact that there is no NCI accompanying his turntake attempt, contributes to the reason of why the attempt failed. Consequently, this might be considered a supporting example of Birdwhistell's [22] findings that successful communication requires both verbal and non-verbal components.

We can therefore confirm our hypothesis that turn management DAs co-occur with regulators. The case analysis further supports the hypothesis that turn management DAs particularly co-occur with communication regulators. Moreover, we can observe that during propositional content, i.e., task-oriented DAs, use of illustrators (foreground NCIs) is more common (see **Figure 2**: B in Example 1). In accordance with Cooperrider's [27] characteristics of foreground-background gestures, we observed the spatial prominence of each type of gesture. Example 1 shows how non-verbal behavior changes in parallel with the change of DAs. As the DAs changed from task with the function of informing to turn management with the function of turn release (see **Figure 2**, B and 1), so did the NCI. It shifted from foreground behavior to background behavior. Moreover, foreground NCIs are far more prominent than the background NCIs. It seems that, as this simultaneous shift in DAs and foreground-background behavior occurs, body behavior is decelerated and minimized. Our findings, therefore, corroborate Cooperrider's [27] the special hallmarks of foreground-background gestures.

There are, however, border cases. For example, the background DA of providing feedback during active listening, such as uttering the supportive "yes" or "mmhmm", can be accompanied by a slight nod of the head. Head nodding is generally considered a foreground gesture, if it signals a "yes" or "no" answer, since it can substitute speech altogether. However, in background use, one does not provide an answer, but merely signals to the interlocutor, that one is listening to them and wishes them to continue their turn. Hence, the act is clearly of background nature. Nevertheless, it is impossible to state that at the same time one does not also agree with what the interlocutor is saying. Agreement, however, is considered a foreground act. This is a typical case where the duality no longer applies. Hence, it is possible even for background DAs, such as feedback providing and eliciting, to cooccur with foreground NCIs. Moreover, even task-oriented DAs are often accompanied with batons, a representative background NCI, since they signal importance or set the rhythm but do not convey any propositional content. It is therefore difficult to extend the shared background-foreground nature hypothesis to other DAs. Despite this observation, the exploration of the shared nature in foreground DAs offers an interesting research question for future research.

A potential concept to further elaborate on the underlying nature is to observe whether the gesture is prominent (in its iteration or spatial dimensions) or subtle [27], as observed in Example 1. In accordance with this distinction, a subtle nod suggests background nature whereas a prominent nod suggest that the gesture is of foreground nature. Moreover, the relative timing may also, provide additional insight in the communicative intent. Although, not directly investigated in this research, is seems that when the stroke phase of the embodiment (especially a hand gesture) cooccurs with a specific speech referent (i.e. the gesture starts at the same time as the spoken articulation) the information provided is propositional, i.e., of foreground nature, whereas when the stroke phase occurs outside boundaries of the targeted referent (or without one) the information provided is of background nature. An example would be phrases "look over there!" and "what do you mean?". In general, deictics will accompany both phrases. On the one hand, the phrase "look over there" is clearly a task-oriented DA and will be accompanied by a pointer, the stroke of which will occur aligned with the verbal articulation of "there". On the other hand, the stroke phase of a similar gesture 'visualizing' the "you" in "what do you mean?" will co-occur with "mean" and will be recognized as a referential deictic in turn management (i.e., as turn offer). Thus, in our future investigations, we tend to analyze if the alignment of verbal structure with the prosody of non-verbal cues (i.e. the cues preceding verbal acts, cues following verbal acts, cues at the beginning or end of verbal cats) may shed further light on the true purpose of the shared nature.

**121**

**Author details**

**Acknowledgements**

**Conflicts of interest**

Maribor, Slovenia

*Can Turn-Taking Highlight the Nature of Non-Verbal Behavior: A Case Study*

In this chapter, we examined what kind of non-verbal behavior accompanies turn management DAs. For the annotation of turn management DAs, the ISO 24617-2 scheme's functions sufficed. Nevertheless, turn management DAs frequently overlap with other DAs, especially within the time management dimension. The fact that it is sometimes very difficult to decide which dimension and function is the most fitting shows the importance of multidimensional DA tagging. As a future endeavor, it would be more functional to create annotation schemes that, besides being multidimensional, denote the hierarchical order of the tags assigned, for example, the primary, secondary, tertiary, etc. dimensions and functions.

Cooperrider's [27] distinction between gestures that occur in the foreground or background proved an effective method within the concept of DAs. We hypothesized that there is an interlink between background NCI and background DAs. Since regulators, specifically, communication regulators, convey typical background NCI, we predicted their co-occurrence with turn management DAs. Indeed, the present case study confirms this hypothesis. Moreover, an interlink with deictic NCI was observed. As they can be of either background or foreground nature, the premise that background DAs co-occur with background NCI is maintained. This duality is not observed only within NCI but also within DAs. An utterance can have alternative expressions, one of background nature and one of foreground nature. However, the duality occurs simultaneously for NCI and for DAs. Hence, the fact that there is the same duality at the NCI level and at the DAs level strengthens the

Izidor Mlakar\*, Matej Rojc, Darinka Verdonik and Simona Majhenič

\*Address all correspondence to: izidor.mlakar@um.si

provided the original work is properly cited.

The authors declare no conflict of interest.

hypothesis of an interlink between the two concepts.

PERSIST (grant agreement No. 875406).

Faculty of Electrical Engineering and Computer Science, University of Maribor,

© 2021 The Author(s). Licensee IntechOpen. This chapter is distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/ by/3.0), which permits unrestricted use, distribution, and reproduction in any medium,

This paper is partially funded by the Slovenian Research Agency, project HUMANIPA (research core funding No. J2-1737 (B)). This paper is partially

funded by European Union's Horizon 2020 research an innovation program, project

*DOI: http://dx.doi.org/10.5772/intechopen.95516*

**5. Conclusions**

*Can Turn-Taking Highlight the Nature of Non-Verbal Behavior: A Case Study DOI: http://dx.doi.org/10.5772/intechopen.95516*
