A video system for recognizing gestures by artificial neural networks for expressive musical control

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Author(s)

  • P Modler
  • T Myatt

Department/unit(s)

Publication details

Title of host publicationGESTURE-BASED COMMUNICATION IN HUMAN-COMPUTER INTERACTION
DatePublished - 2003
Pages541-548
Number of pages8
PublisherSPRINGER-VERLAG BERLIN
Place of PublicationBERLIN
EditorsA Camurri, G Volpe
Original languageEnglish
ISBN (Print)3-540-21072-5

Abstract

In this paper we describe a system to recognize gestures to control musical processes. For that we applied a Time Delay Neuronal Network to match gestures processed as variation of luminance information in video streams. This resulted in recognition rates of about 90% for 3 different types of hand gestures and it is presented here as a prototype for a gestural recognition system that is tolerant to ambient conditions and environments. The neural network can be trained to recognize gestures difficult to be described by postures or sign language. This can be used to adapt to unique gestures of a performer or video sequences of arbitrary moving objects. We will discuss the outcome of extending the system to learn successfully a set of 17 hand gestures. The application was implemented in jMax to achieve real-time conditions and easy integration into a musical environment. We will describe the design and learning procedure of the using the Stuttgart Neuronal Network Simulator. The system aims to integrate into an environment that enables expressive control of musical parameters (KANSEI).

Discover related content

Find related publications, people, projects, datasets and more using interactive charts.

View graph of relations