Engagement Effects of Player Rating System-Based Matchmaking for Level Ordering in Human Computation Games

Anurag Sarkar, Michael Williams, Christoph Sebastian Deterding, Seth Cooper

Research output: Chapter in Book/Report/Conference proceedingConference contribution


Human computation games lack established ways of balancing the difficulty of tasks or levels served to players, potentially contributing to their low engagement rates. Traditional player rating systems have been suggested as a potential solution: using them to rate both players and tasks could estimate player skill and task difficulty and fuel player-task matchmaking. However, neither the effect of difficulty balancing on engagement in human computation games nor the use of player rating systems for this purpose has been empirically tested. We therefore examined the engagement effects of using the Glicko-2 player rating system to order tasks in the human computation game Paradox. An online experiment (n=294) found that both matchmaking-based and pure difficulty-based ordering of tasks led to significantly more attempted and completed levels than random ordering. Additionally, both matchmaking and random ordering led to significantly more di cult tasks being completed than pure difficulty-based ordering. We conclude that poor balancing contributes to poor engagement in human computation games, and that player rating system-based difficulty rating may be a viable and efficient way of improving both.
Original languageEnglish
Title of host publicationProceedings of the 12th International Conference on the Foundations of Digital Games, FDG 2017
EditorsAlessandro Canossa, Miguel Sicart, Casper Harteveld, Jichen Zhu, Sebastian Deterding
Number of pages10
VolumePart F130151
ISBN (Electronic)978-1-4503-5319-9
Publication statusPublished - 14 Aug 2017
EventInternational Conference on the Foundations of Digital Games 2017 - Hyannis, MA, Hyannis, United States
Duration: 14 Aug 201717 Aug 2017

Publication series

NameACM Proceedings
ISSN (Electronic)2168-4081


ConferenceInternational Conference on the Foundations of Digital Games 2017
Abbreviated titleFDG'17
Country/TerritoryUnited States
Internet address

Bibliographical note

This is an author-produced version of the published paper. Uploaded in accordance with the publisher’s self-archiving policy. Further copying may not be permitted; contact the publisher for details


  • human computation games
  • matchmaking
  • player rating systems
  • difficulty balancing
  • Glicko-2
  • level ordering
  • Player rating systems
  • Level ordering
  • Human computation games
  • Matchmaking
  • Difficulty balancing

Cite this