Expressy: Using a Wrist-worn Inertial Measurement Unit to Add Expressiveness to Touch-based Interactions

TY - GEN

T1 - Expressy

T2 - ACM Conference on Human Factors in Computing Systems 2016

AU - Wilkinson, Gerard

AU - Kharrufa, Ahmed

AU - Hook, Jonathan David

AU - Pursgrove, Bradley

AU - Wood, Gavin

AU - Haeuser, Hendrik

AU - Hammerla, Nils

AU - Hodges, Steve

AU - Olivier, Patrick

N1 - © 2016, The Authors. This is an author-produced version of the published paper. Uploaded in accordance with the publisher’s self-archiving policy. Further copying may not be permitted; contact the publisher for details.

PY - 2016/5/7

Y1 - 2016/5/7

N2 - Expressiveness, which we define as the extent to which rich and complex intent can be conveyed through action, is a vital aspect of many human interactions. For instance, paint on canvas is said to be an expressive medium, because it affords the artist the ability to convey multifaceted emotional intent through intricate manipulations of a brush. To date, touch devices have failed to offer users a level of expressiveness in their interactions that rivals that experienced by the painter and those completing other skilled physical tasks. We investigate how data about hand movement – provided by a motion sensor, similar to those found in many smart watches or fitness trackers – can be used to expand the expressiveness of touch interactions. We begin by introducing a conceptual model that formalizes a design space of possible expressive touch interactions. We then describe and evaluate Expressy, an approach that uses a wrist-worn inertial measurement unit to detect and classify qualities of touch interaction that extend beyond those offered by today’s typical sensing hardware. We conclude by describing a number of sample applications, which demonstrate the enhanced, expressive interaction capabilities made possible by Expressy.

AB - Expressiveness, which we define as the extent to which rich and complex intent can be conveyed through action, is a vital aspect of many human interactions. For instance, paint on canvas is said to be an expressive medium, because it affords the artist the ability to convey multifaceted emotional intent through intricate manipulations of a brush. To date, touch devices have failed to offer users a level of expressiveness in their interactions that rivals that experienced by the painter and those completing other skilled physical tasks. We investigate how data about hand movement – provided by a motion sensor, similar to those found in many smart watches or fitness trackers – can be used to expand the expressiveness of touch interactions. We begin by introducing a conceptual model that formalizes a design space of possible expressive touch interactions. We then describe and evaluate Expressy, an approach that uses a wrist-worn inertial measurement unit to detect and classify qualities of touch interaction that extend beyond those offered by today’s typical sensing hardware. We conclude by describing a number of sample applications, which demonstrate the enhanced, expressive interaction capabilities made possible by Expressy.

KW - Expressive interaction

KW - Expressiveness

KW - Inertial measurement unit

KW - Intentionality

KW - Smart watch

KW - Touch interaction

UR - http://www.scopus.com/inward/record.url?scp=84995765424&partnerID=8YFLogxK

U2 - 10.1145/2858036.2858223

DO - 10.1145/2858036.2858223

M3 - Conference contribution

T3 - CHI Conference Proceedings

SP - 2832

EP - 2844

BT - CHI 2016 - Proceedings, 34th Annual CHI Conference on Human Factors in Computing Systems

PB - ACM

Y2 - 7 May 2016 through 12 May 2016

ER -