*4.1.1 Description of three binaural headphone monitoring solutions*

**Table 1** highlights the binaural rendering pipelines and augmentation technologies that we chose to best adapt to performers' needs for each context. To enable conductors to monitor large ensembles on headphones, Soudoplatoff and Pras [7] designed a Binaural with Head Tracking (BHT) system that rendered a *JML* tree [53], that is, a main five-microphone array with specific dimensions, and integrated spot microphones. This system used *Bipan*<sup>6</sup> software [54] coupled with *Hedrot,*<sup>7</sup> that is, a head tracker located on the conductors' headphones. In Bipan, the LISTEN database [55] was used with the HRTF pair n°1040, as advised in a previous study [56], since this HRTF pair satisfied most users during public demonstrations of the software [57]. Bipan had a latency of 5.3 ms when used with a buffer size of 256 samples. According to previous research, a monitoring system latency below 42 ms should be acceptable [58]. Furthermore, Hedrot had a latency of 48.1 ± 4.3 ms [54], which should provide conductors with accurate localization cues since the head tracking latency does not hinder the stability of virtual sounds within complex auditory scenes under 71 ms [59], even if it could be noticeable when superior to 30 ms [60]. The assessment tests required the use of the *TotalMix* application, which has a meaningless latency of three samples (equal to about 68 μs at 44.1 kHz), to digitally convert the microphone signal and send it to the BHT via a *RME MADIface*. 8

<sup>6</sup> 3D audio technology developed in-house at the Paris Conservatoire in collaboration with IRCAM as part of the Bili project: http://www.bili-project.org/. More details can be found here: https://alexisbaskind.net/fr/bipan-binaural/

<sup>7</sup> https://abaskind.github.io/hedrot/

<sup>8</sup> https://www.manualslib.com/manual/1310692/Rme-Audio-Madiface-Usb.html?page=70


#### **Table 1.**

*Performance context, binaural rendering pipeline, and augmentation principles of the three binaural headphone monitoring technologies.*

To enhance the intelligibility of improvisers' subtle expressive gestures, Bauer et al. [8] developed a Binaural Mixed Reality (BMR) system that rendered close mono microphones through KLANG: fabrik (KF) hardware. KF was chosen for its convincing externalization of sources and sound quality<sup>9</sup> as well as its latency of less than 3 ms.10 Indeed, the set of KLANG-proprietary HRTFs was preferred to HRTFs from the LISTEN database that features a low sampling resolution, introduces noise artifacts, and present amplitude errors, for example, for the HRTF pair of subject IRC\_1034 [61]. The BMR had a total latency (KF latency plus ProTools latency) of 4 ms for the two trios. Regarding the world music performer, the technical setup between the microphone signal and the monitoring system included several digital devices, and the measured total latency of the chain was 14.1 ms. The musician specified that he did not notice it, and confirmed that the system latency did not hinder his performance.

To attempt acoustic transparency of the recording auditory space, Menon [9] built an Active Binaural Headphones (ABH) system with two 150°-angled small condenser microphones mounted on each earcup. Based on Bauer et al. [8]'s satisfying findings, the signal coming from the four mounted microphones was binaurally rendered through KLANG: vier (KV) hardware, which features the same sonic and latency properties than KF11. The ABH total latency was inferior to 16.8 ms. The assessment tests required the use of the *CueMix* application that has no latency to digitally convert the microphone signal and send it to the ABH via a *MOTU 896* 

<sup>10</sup> https://www.klang.com/en/products/klang\_fabrik

<sup>9</sup> The researchers were able to evaluate the quality of this equipment as they are experienced sound engineers, in both stereo and 3D audio production techniques.

<sup>11</sup> https://www.klang.com/en/products/klang\_vier

#### *Binaural Headphone Monitoring to Enhance Musicians' Immersion in Performance DOI: http://dx.doi.org/10.5772/intechopen.104845*

*mk3*12 that has a latency of under 13 ms, and the *Aviom* personal monitor mixer that has a latency of 0.88 ms to amplify the headphone signal.13

In summary, the BHT and ABH are two AAR systems with dynamic binaural because for both of their applied contexts, performers primarily needed to monitor sound sources while being in the same room as their peers, and thus required a technology that accurately conveyed source localization. On the other hand, the BMR is an AMR system with static binaural because improvisers primarily needed to monitor their previous recordings or their band members who were playing in separate rooms; thus, the re-creation of a virtual space that facilitated their immersion was more desirable than accurate source localization.

For all three technologies, closed-back headphones were used to minimize sound leakage into the microphones. Both Bipan and KLANG used anechoic HRTFs, and so enabled us as sound engineers to generate spatial images with re-created acoustics that fit the acoustics of the performance space.14 These HRTFs were also non-individualized and thus required performers' listening training [31] and/or dynamic binaural rendering [29] to optimize source externalization and mitigate timbre artifacts. Therefore, one week before conducting the case studies that assessed the BMR setup, which is static, the improviser participants were instructed to listen to three-to-five binaural audio productions over headphones that were selected from *Hyperradio* podcasts by Bauer (total duration of around 25 mn), to get used to the binaural rendering. All of them confirmed to Bauer at the beginning of their recording session that they had listened to at least three of these productions. This consists of a total listening experience of 15 mn at minimum for each participant.

### *4.1.2 Case study procedures for binaural solution assessment*

**Table 2** details the locations, genres, and instrument line-ups of the eight case studies in chronological order for testing our BHT, BMR, and ABH technologies in rehearsal or studio recording situations. Thirteen performers agreed to participate in these comparative tests without financial compensation. The first two tests that involved symphonic ensembles were organized at the institutional level as part of a pedagogical project. For the other five tests, Bauer and Menon volunteered to mix the recordings, which the performers could use to promote their music.

To assess the three headphone technologies that are described in the previous section, two conductors, seven improvisers, and four musicians who perform a range of musical genres compared binaural against *traditional* stereo headphones, that is, the monitoring systems commonly used in each of the performance venues. The experimental procedures for each case study are summarized in **Table 2**. Because "an experimental protocol is ecologically valid if the participants react […] as if they were in a natural situation" [10], Soudoplatoff organized the first two case studies during rehearsals of programmed productions with large ensembles. Specifically, for the last two days of a week of film-scoring rehearsals, Maestro Laurent Petitgirard agreed to swap headphone conditions five times during breaks that occurred every 90 min, which led him to test each condition three times. Unfortunately, the comparison could not be carried out with the jazz symphonic ensemble due to a conjunction of acoustic and organization issues (see Section 4.2

<sup>12</sup> https://motu.com/techsupport/technotes/what-is-the-latency-of-my-motu-audio-interface

<sup>13</sup> https://www.aviom.com/library/User-Guides/36\_A-16D-User-Guide.pdf

<sup>14</sup> Using non-anechoic HRTFs implies generating a binaural image that emulates the externalization of sources in specific room acoustics. This may be enjoyable for the listener and can be creative in the context of music production. However, it is likely to be confusing for the musician in the context of headphone monitoring when performing.


#### **Table 2.**

*Location, genre, instrumentation, and comparison procedure of the eight case studies—B refers to the binaural condition and S to the stereo condition.*

for explanations). Bauer and Menon ensured the ecological validity of their experiments by inviting performers to record in the studio with the incentive of getting a demo that they could use to promote their music. In this context, the world music performer and two improvisation/jazz trios accepted to test the BMR system in a counterbalanced order, and each switched conditions once, after 2 h and 45 min, respectively. Also, a singer-songwriter, a rock duo, and a pianist who performed with electronics accepted to test the ABH system once they were satisfied with their takes using the traditional stereo system of the studio.

For the seven case studies during which performers compared binaural and stereo headphones, the researchers took notes on users' behaviors and comments during the tests. Whereas Soudoplatoff asked Maestro Petitgirard to react spontaneously after each trial, Bauer conducted post-test focus group interviews, and Menon carried out

#### *Binaural Headphone Monitoring to Enhance Musicians' Immersion in Performance DOI: http://dx.doi.org/10.5772/intechopen.104845*

individual post-test written surveys with the performers at the end of the recording session. For all case studies, performers were asked to compare both types of headphones in terms of comfort, playfulness, benchmarks, and perception of the spatial image. For the recording sessions only, performers were asked to compare the perception of their own instrument in relation to others'. Moreover, a few weeks after the recording sessions of the world music performer and the two improvisation/jazz trios, Bauer sent stereo mixes of all the takes to the performers, and he asked them to select their favorite take for each piece (or their favorite improvisation). Based on a previous performance study in the recording studio in jazz [62], collecting musicians' choice of takes that were recorded in different conditions has the potential to inform the impact of the BMR on creativity and musical results. The context of Soudoplatoff's and Menon's tests did not allow for this additional collection of data.
