Toward Emotionally-Congruent Dynamic Soundtrack Generation

Research output: Contribution to journalArticlepeer-review


Real-time control of the emotional content of sound has utility in video game soundtracking where the player controls the narrative trajectory, and the affective attributes of the sound should ideally match this trajectory. Perceived emotions can be represented in a 2-dimensional space composed of valence (positivity, e.g. happy, sad, fearful) and arousal (intensity, e.g. mild vs strong). This report is a speculative exploration of measuring and manipulating sound effects to achieve emotional congruence. An initial study suggests that timbral features can exert an influence on the perceived emotional response of a listener. A panel of listeners responded to stimuli in a set with varying timbres, while maintaining pitch, loudness, and other musical and acoustic features such as key, melodic contour, rhythm and meter, reverberant environment etc. The long term goal is to create an automated system that utilizes timbre morphing in real time to manipulate perceived affect in soundtrack generation.

Original languageEnglish
Pages (from-to)654-663
Number of pages10
JournalJournal of the Audio Engineering Society
Issue number9
Publication statusPublished - 19 Sept 2016

Cite this