Engagement Effects of Player Rating System-Based Matchmaking for Level Ordering in Human Computation Games

TY - GEN

T1 - Engagement Effects of Player Rating System-Based Matchmaking for Level Ordering in Human Computation Games

AU - Sarkar, Anurag

AU - Williams, Michael

AU - Deterding, Christoph Sebastian

AU - Cooper, Seth

N1 - This is an author-produced version of the published paper. Uploaded in accordance with the publisher’s self-archiving policy. Further copying may not be permitted; contact the publisher for details

PY - 2017/8/14

Y1 - 2017/8/14

N2 - Human computation games lack established ways of balancing the difficulty of tasks or levels served to players, potentially contributing to their low engagement rates. Traditional player rating systems have been suggested as a potential solution: using them to rate both players and tasks could estimate player skill and task difficulty and fuel player-task matchmaking. However, neither the effect of difficulty balancing on engagement in human computation games nor the use of player rating systems for this purpose has been empirically tested. We therefore examined the engagement effects of using the Glicko-2 player rating system to order tasks in the human computation game Paradox. An online experiment (n=294) found that both matchmaking-based and pure difficulty-based ordering of tasks led to significantly more attempted and completed levels than random ordering. Additionally, both matchmaking and random ordering led to significantly more di cult tasks being completed than pure difficulty-based ordering. We conclude that poor balancing contributes to poor engagement in human computation games, and that player rating system-based difficulty rating may be a viable and efficient way of improving both.

AB - Human computation games lack established ways of balancing the difficulty of tasks or levels served to players, potentially contributing to their low engagement rates. Traditional player rating systems have been suggested as a potential solution: using them to rate both players and tasks could estimate player skill and task difficulty and fuel player-task matchmaking. However, neither the effect of difficulty balancing on engagement in human computation games nor the use of player rating systems for this purpose has been empirically tested. We therefore examined the engagement effects of using the Glicko-2 player rating system to order tasks in the human computation game Paradox. An online experiment (n=294) found that both matchmaking-based and pure difficulty-based ordering of tasks led to significantly more attempted and completed levels than random ordering. Additionally, both matchmaking and random ordering led to significantly more di cult tasks being completed than pure difficulty-based ordering. We conclude that poor balancing contributes to poor engagement in human computation games, and that player rating system-based difficulty rating may be a viable and efficient way of improving both.

KW - human computation games

KW - matchmaking

KW - player rating systems

KW - difficulty balancing

KW - Glicko-2

KW - level ordering

KW - Player rating systems

KW - Level ordering

KW - Human computation games

KW - Matchmaking

KW - Difficulty balancing

UR - http://www.scopus.com/inward/record.url?scp=85030767307&partnerID=8YFLogxK

U2 - 10.1145/3102071.3102093

DO - 10.1145/3102071.3102093

M3 - Conference contribution

VL - Part F130151

T3 - ACM Proceedings

BT - Proceedings of the 12th International Conference on the Foundations of Digital Games, FDG 2017

A2 - Canossa, Alessandro

A2 - Sicart, Miguel

A2 - Harteveld, Casper

A2 - Zhu, Jichen

A2 - Deterding, Sebastian

PB - ACM

T2 - International Conference on the Foundations of Digital Games 2017

Y2 - 14 August 2017 through 17 August 2017

ER -