From Acceleration to Rhythmicity: Smartphone-Assessed Movement Predicts Properties of Music

Melanie Irrgang; Jochen Steffens; Hauke Egermann

doi:10.1080/09298215.2020.1715447

From Acceleration to Rhythmicity: Smartphone-Assessed Movement Predicts Properties of Music

Melanie Irrgang, Jochen Steffens, Hauke Egermann

Music

Research output: Contribution to journal › Article › peer-review

Abstract

Music moves us. Yet, querying music is still a disembodied process in most music rec- ommender scenarios. New mediation technologies like querying music by movement would take account of the empirically well founded knowledge of embodied mu- sic cognition. Thus, the goal of the presented study was to explore how movement captured by smartphone accelerometer data can be related to musical properties. Participants (N = 23, mean age = 34.6 yrs, SD = 13.7 yrs, 13 females, 10 males) moved a smartphone to 15 musical stimuli of 20s length presented in random order. Motion features related to tempo, smoothness, size, and regularity were extracted from accelerometer data to predict the musical qualities “rhythmicity”, “pitch level + range” and “complexity” assessed by three music experts. Motion features se- lected by a stepwise AIC model predicted the musical properties to the following degrees “rhythmicity” (R2 = .45), “pitch level and range” (R2 = .06) and “com- plexity” (R2 = .15). We conclude that (rhythmic) music properties can be predicted from the movement it evoked, and that an embodied approach to Music Information Retrieval is feasible.

Original language	English
Pages (from-to)	1-15
Number of pages	15
Journal	Journal of New Music Research
Early online date	30 Jan 2020
DOIs	https://doi.org/10.1080/09298215.2020.1715447
Publication status	E-pub ahead of print - 30 Jan 2020

Bibliographical note

© 2020 Informa UK Limited, trading as Taylor & Francis Group. This is an author-produced version of the published paper. Uploaded in accordance with the publisher’s self-archiving policy. Further copying may not be permitted; contact the publisher for details.

Access to Document

10.1080/09298215.2020.1715447

FromAcc2Rhythmicity
© 2020 Informa UK Limited, trading as Taylor & Francis Group. This is an author-produced version of the published paper. Uploaded in accordance with the publisher’s self-archiving policy. Further copying may not be permitted; contact the publisher for details.
Accepted author manuscript, 1.84 MB

Cite this

@article{dff15d80df3e43e8a76f6492bdd40fd6,

title = "From Acceleration to Rhythmicity: Smartphone-Assessed Movement Predicts Properties of Music",

abstract = "Music moves us. Yet, querying music is still a disembodied process in most music rec- ommender scenarios. New mediation technologies like querying music by movement would take account of the empirically well founded knowledge of embodied mu- sic cognition. Thus, the goal of the presented study was to explore how movement captured by smartphone accelerometer data can be related to musical properties. Participants (N = 23, mean age = 34.6 yrs, SD = 13.7 yrs, 13 females, 10 males) moved a smartphone to 15 musical stimuli of 20s length presented in random order. Motion features related to tempo, smoothness, size, and regularity were extracted from accelerometer data to predict the musical qualities “rhythmicity”, “pitch level + range” and “complexity” assessed by three music experts. Motion features se- lected by a stepwise AIC model predicted the musical properties to the following degrees “rhythmicity” (R2 = .45), “pitch level and range” (R2 = .06) and “com- plexity” (R2 = .15). We conclude that (rhythmic) music properties can be predicted from the movement it evoked, and that an embodied approach to Music Information Retrieval is feasible.",

author = "Melanie Irrgang and Jochen Steffens and Hauke Egermann",

note = "{\textcopyright} 2020 Informa UK Limited, trading as Taylor & Francis Group. This is an author-produced version of the published paper. Uploaded in accordance with the publisher{\textquoteright}s self-archiving policy. Further copying may not be permitted; contact the publisher for details.",

year = "2020",

month = jan,

day = "30",

doi = "10.1080/09298215.2020.1715447",

language = "English",

pages = "1--15",

journal = "Journal of New Music Research",

issn = "0929-8215",

publisher = "Taylor & Francis",

}

TY - JOUR

T1 - From Acceleration to Rhythmicity

T2 - Smartphone-Assessed Movement Predicts Properties of Music

AU - Irrgang, Melanie

AU - Steffens, Jochen

AU - Egermann, Hauke

N1 - © 2020 Informa UK Limited, trading as Taylor & Francis Group. This is an author-produced version of the published paper. Uploaded in accordance with the publisher’s self-archiving policy. Further copying may not be permitted; contact the publisher for details.

PY - 2020/1/30

Y1 - 2020/1/30

N2 - Music moves us. Yet, querying music is still a disembodied process in most music rec- ommender scenarios. New mediation technologies like querying music by movement would take account of the empirically well founded knowledge of embodied mu- sic cognition. Thus, the goal of the presented study was to explore how movement captured by smartphone accelerometer data can be related to musical properties. Participants (N = 23, mean age = 34.6 yrs, SD = 13.7 yrs, 13 females, 10 males) moved a smartphone to 15 musical stimuli of 20s length presented in random order. Motion features related to tempo, smoothness, size, and regularity were extracted from accelerometer data to predict the musical qualities “rhythmicity”, “pitch level + range” and “complexity” assessed by three music experts. Motion features se- lected by a stepwise AIC model predicted the musical properties to the following degrees “rhythmicity” (R2 = .45), “pitch level and range” (R2 = .06) and “com- plexity” (R2 = .15). We conclude that (rhythmic) music properties can be predicted from the movement it evoked, and that an embodied approach to Music Information Retrieval is feasible.

AB - Music moves us. Yet, querying music is still a disembodied process in most music rec- ommender scenarios. New mediation technologies like querying music by movement would take account of the empirically well founded knowledge of embodied mu- sic cognition. Thus, the goal of the presented study was to explore how movement captured by smartphone accelerometer data can be related to musical properties. Participants (N = 23, mean age = 34.6 yrs, SD = 13.7 yrs, 13 females, 10 males) moved a smartphone to 15 musical stimuli of 20s length presented in random order. Motion features related to tempo, smoothness, size, and regularity were extracted from accelerometer data to predict the musical qualities “rhythmicity”, “pitch level + range” and “complexity” assessed by three music experts. Motion features se- lected by a stepwise AIC model predicted the musical properties to the following degrees “rhythmicity” (R2 = .45), “pitch level and range” (R2 = .06) and “com- plexity” (R2 = .15). We conclude that (rhythmic) music properties can be predicted from the movement it evoked, and that an embodied approach to Music Information Retrieval is feasible.

U2 - 10.1080/09298215.2020.1715447

DO - 10.1080/09298215.2020.1715447

M3 - Article

SN - 0929-8215

SP - 1

EP - 15

JO - Journal of New Music Research

JF - Journal of New Music Research

ER -