Auditory Scene Synthesis using Virtual Acoustic Recording and Reproduction

TY - THES

T1 - Auditory Scene Synthesis using Virtual Acoustic Recording and Reproduction

AU - Kearney, Gavin

PY - 2010/3/30

Y1 - 2010/3/30

N2 - This thesis is concerned with recording of real acoustic events for spatial audio presentations in interactive Virtual Auditory Environments (VAEs). A VAE is an environment where auditory perception does not correspond to the physical environment of the listener, but rather to a virtual one. The current state of the art of spatial audio reproduction in VAEs focuses on real-time walk-through audio presentations using computational-based auralization, where the transfer functions describing the acoustic interaction between a sound source and a listener's ears in a reverberant room are computationally modeled. The purpose of such systems is that the end-listener will no longer be limited to single-perspective recordings of acoustic events such as musical performances, but can instead move around a VAE as if they were at an original performance. Whilst such methods can yield a reasonable spatial impression of an acoustic event, auralization using real acoustic measurements (i.e. data-based auralization) offers a level of authenticity not yet fully achieved with computational-based methods. On the other hand, significant challenges exist in the use of data-based auralization, since the incorporation of listener movement in the virtual space requires a dataset of room impulse responses to be measured. Furthermore, the plausibility of the reproduction of acoustic events in a VAE isalso dictated by the spatial audio reproduction method. This thesis hypothesizes that practical data-based auralization can be achieved for both individual and multi-listener scenarios using the same recording/measurement and reproduction paradigm. Of particular interest is the potential to utilize current commercially available technology to achieve this. In supporting this hypothesis, two main strands of research are defined, termed `virtual acoustic recording' and `virtual acoustic reproduction'. `Virtual acoustic recording' outlines the direct-field capture of the musical event, as well as its subsequent convolution with Spatial Room Impulse Responses (SRIRs). The process considers the perceptual differences between convolution-based recordings and real recordings, in particular the removal of coloration effects in the convolution chain, as well as the synthesis of the directional properties of the source usingcommercially available loudspeakers. A novel interpolation algorithm, as well as a perceptual analysis of source localization in reverberant environments is then used to reduce the number of measurements required in the SRIR dataset.`Virtual Acoustic Reproduction' then refers to the spatial audio presentation of an acoustic event in both single and multi-listener scenarios in a VAE. To this end, an in-depth study of spatial audio reproduction strategies is presented, where the localization accuracy of systems such as Vector Based Amplitude Panning, Ambisonics, and Wave Field Synthesis is assessed through both objective and subjective measures. The computational challenges involved inheadphone reproduction of spatial audio presentations are simplified using a novel factorization algorithm with reduces the length of head-related impulse response filters significantly, without loss of spatial information. Finally, conclusions from this thesis work are drawn pertaining to the research hypothesis, and future directions are presented, which include the incorporation of the signal processing techniques presented for computational-based auralization.

AB - This thesis is concerned with recording of real acoustic events for spatial audio presentations in interactive Virtual Auditory Environments (VAEs). A VAE is an environment where auditory perception does not correspond to the physical environment of the listener, but rather to a virtual one. The current state of the art of spatial audio reproduction in VAEs focuses on real-time walk-through audio presentations using computational-based auralization, where the transfer functions describing the acoustic interaction between a sound source and a listener's ears in a reverberant room are computationally modeled. The purpose of such systems is that the end-listener will no longer be limited to single-perspective recordings of acoustic events such as musical performances, but can instead move around a VAE as if they were at an original performance. Whilst such methods can yield a reasonable spatial impression of an acoustic event, auralization using real acoustic measurements (i.e. data-based auralization) offers a level of authenticity not yet fully achieved with computational-based methods. On the other hand, significant challenges exist in the use of data-based auralization, since the incorporation of listener movement in the virtual space requires a dataset of room impulse responses to be measured. Furthermore, the plausibility of the reproduction of acoustic events in a VAE isalso dictated by the spatial audio reproduction method. This thesis hypothesizes that practical data-based auralization can be achieved for both individual and multi-listener scenarios using the same recording/measurement and reproduction paradigm. Of particular interest is the potential to utilize current commercially available technology to achieve this. In supporting this hypothesis, two main strands of research are defined, termed `virtual acoustic recording' and `virtual acoustic reproduction'. `Virtual acoustic recording' outlines the direct-field capture of the musical event, as well as its subsequent convolution with Spatial Room Impulse Responses (SRIRs). The process considers the perceptual differences between convolution-based recordings and real recordings, in particular the removal of coloration effects in the convolution chain, as well as the synthesis of the directional properties of the source usingcommercially available loudspeakers. A novel interpolation algorithm, as well as a perceptual analysis of source localization in reverberant environments is then used to reduce the number of measurements required in the SRIR dataset.`Virtual Acoustic Reproduction' then refers to the spatial audio presentation of an acoustic event in both single and multi-listener scenarios in a VAE. To this end, an in-depth study of spatial audio reproduction strategies is presented, where the localization accuracy of systems such as Vector Based Amplitude Panning, Ambisonics, and Wave Field Synthesis is assessed through both objective and subjective measures. The computational challenges involved inheadphone reproduction of spatial audio presentations are simplified using a novel factorization algorithm with reduces the length of head-related impulse response filters significantly, without loss of spatial information. Finally, conclusions from this thesis work are drawn pertaining to the research hypothesis, and future directions are presented, which include the incorporation of the signal processing techniques presented for computational-based auralization.

KW - Ambisonics

KW - Virtual Acoustics

KW - Dynamic Time Warping

KW - HRTFs

KW - Factorisation

KW - Phase Transform

KW - Kernel density estimation

KW - VBAP

KW - Spatial room impulse response

KW - Wave Field Synthesis

KW - Impulse Responses

M3 - Doctoral Thesis

PB - Trinity College

CY - Dublin, Ireland

ER -