Towards a Data Driven Rendering Algorithm for Visually Impaired Film and Television Audiences

TY - GEN

T1 - Towards a Data Driven Rendering Algorithm for Visually Impaired Film and Television Audiences

AU - McLoughlin, Michael Paul

AU - Lopez, Mariana Julieta

AU - Hofstadter, Krisztian

AU - Kearney, Gavin

PY - 2023/10/23

Y1 - 2023/10/23

N2 - In stereo sound systems, the listener's ability to localise sound in the reproduced stereo field is affected by a variety of factors such as their listening position and the degree of separation between the loudspeakers. Understanding the effect of these factors is important for creating object-based mixes, where the panning position and volume of sounds can be adjusted to suit the user at the point of media consumption. Being able to control the panning and volume of different sources in the renderer is especially useful for visually impaired audience members, as it can allow for greater immersion and narrative understanding for film and television. For this reason, it is important to conduct listening tests on how these audiences perceive sound over stereo systems. These listening tests can then be used to inform renderer panning algorithms. However, before inviting visually impaired people to take part in listening tests, a robust methodology and control group comparison must first be established. Here, we present an algorithm that is driven by the results of a Minimum Audible Angle (MAA) staircase listening test for different listening positions and loudspeaker base width combinations. This consisted of 22 trials across 21 sighted participants. We calculated the mean MAA for these 22 trials and used these results to determine our panning angle when given a listener position and stereo base width. We then carried out a smaller listening test to determine the effectiveness of a perceptually informed rendering algorithm (8 tests across 8 participants). We achieved this by calculating the F-Score of our method against a control method that does not account for these factors. The mean F-score was 0.62 for our method against a mean of 0.36 for our control. Our results indicate that our algorithm has the potential to be used for object-based mixes where spatial separation for all audience members is more important than precise object placement. Future research will carry out the same experiment with visually impaired audience members, and compare the results with this pilot study.

AB - In stereo sound systems, the listener's ability to localise sound in the reproduced stereo field is affected by a variety of factors such as their listening position and the degree of separation between the loudspeakers. Understanding the effect of these factors is important for creating object-based mixes, where the panning position and volume of sounds can be adjusted to suit the user at the point of media consumption. Being able to control the panning and volume of different sources in the renderer is especially useful for visually impaired audience members, as it can allow for greater immersion and narrative understanding for film and television. For this reason, it is important to conduct listening tests on how these audiences perceive sound over stereo systems. These listening tests can then be used to inform renderer panning algorithms. However, before inviting visually impaired people to take part in listening tests, a robust methodology and control group comparison must first be established. Here, we present an algorithm that is driven by the results of a Minimum Audible Angle (MAA) staircase listening test for different listening positions and loudspeaker base width combinations. This consisted of 22 trials across 21 sighted participants. We calculated the mean MAA for these 22 trials and used these results to determine our panning angle when given a listener position and stereo base width. We then carried out a smaller listening test to determine the effectiveness of a perceptually informed rendering algorithm (8 tests across 8 participants). We achieved this by calculating the F-Score of our method against a control method that does not account for these factors. The mean F-score was 0.62 for our method against a mean of 0.36 for our control. Our results indicate that our algorithm has the potential to be used for object-based mixes where spatial separation for all audience members is more important than precise object placement. Future research will carry out the same experiment with visually impaired audience members, and compare the results with this pilot study.

KW - Stereo

KW - VBAP

KW - Accessibility

KW - object-based audio

KW - Film

U2 - 10.1109/I3DA57090.2023.10289588

DO - 10.1109/I3DA57090.2023.10289588

M3 - Conference contribution

SN - 979-8-3503-1105-1

BT - 2023 Immersive and 3D Audio: from Architecture to Automotive (I3DA)

PB - IEEE

CY - Bologna, Italy

T2 - Immersive and 3D Audio: from Architecture to Automotive (I3DA)

Y2 - 5 September 2023 through 7 September 2023

ER -