Gesture-sound causality from the audience's perspective: investigating the influence of mapping design on the reception of new digital musical instruments

Gina Emerson, Alexander Förstel, Hauke Egermann

Research output: Chapter in Book/Report/Conference proceedingConference contribution


1. Background In contrast to their traditional, acoustic counterparts, digital musical instruments (DMIs) rarely feature a clear, causal relationship between the performer's actions and the sounds produced. Instead, they often function simply as controllers, triggering sounds that are or have been synthesised elsewhere; they are not necessarily sources of sound in themselves (Miranda and Wanderley 2006). Consequently, the performer's physical interaction with the device frequently does not appear to correlate directly with the sonic output, thus making it difficult for spectators to discern how gestures and actions are translated into sounds. This relationship between input and output is determined by the mapping, the term for the process of establishing relationships of cause and effect between the control and sound generation elements of the instrument (Hunt et al. 2003). 2. Aims This two-part empirical study from the 3DMIN (Design, Development and Dissemination of New Musical Instruments) project at the Technical University, Berlin, aims to shed light on how the causality of mapping designs impacts spectator responses to new DMIs. The results will offer important feedback to those designing contemporary musical instruments, building on existing research on the evaluation of DMIs. 3. Method A preliminary questionnaire study was carried out at a 3DMIN concert in February 2015 with the aim of assessing audience reactions to performances with new DMIs in an explorative manner. The participants (n= 49) were asked 1) to describe in note form what they had paid most attention to during the performance and 2) to offer a more general description of their overall impressions of the performance, mentioning any emotional responses and their thoughts on the instrument design and music. 4. Results The respondents used a range of keywords to describe their experiences, which were grouped into three dimensions of response: emotional reactions (e.g. `tension' [n= 13], `calming' [n= 3], `threatening' [n= 3], `annoying' [n= 3]), statements of preference or liking (e.g. `good' [n= 21], `cool' [n= 8], `great' [n= 7]) and comments on attention (e.g. `interesting' [n= 27], `curiosity' [n= 3], `boring' [n= 5], `predicatable/repetitive' [n= 7]). The majority of participants reported attending to the gestures or movements of the performers (n= 28), with fourteen participants specifically mentioning either focusing on or trying to understand the gesture-sound causality. 5. Discussion The results from the preliminary investigation suggest that the lack of gesture-sound causality in some DMI designs is considered to be a noteworthy and to some extent, perplexing aspect by audience members. A controlled, computer-based study will be developed in order to isolate the influence of differing degrees of input-output causality on the spectator's reception of the instrument. The participants will watch short video clips of causal and uncausal DMIs and rate them on a number of parameters that correspond to the three reception dimensions of emotional reaction, preference and attention, as devised from the preliminary study results.
Original languageEnglish
Title of host publicationTagungsband Jahrestagung der deutschen Gesellschaft für Musikpsychologie
EditorsGunter Kreutz
Place of PublicationOldenburg, Germany
PublisherDeutsche Gesellschaft für Musikpsychologie
Number of pages2
Publication statusPublished - 2015

Cite this