Not Very Effective: Validity Issues of the Effectance in Games Scale

Nicholas Ballou; Heiko Breitsohl; Dominic Kao; Kathrin Gerling; Christoph Sebastian Deterding

doi:10.1145/3450337.3483492

Not Very Effective: Validity Issues of the Effectance in Games Scale

Nicholas Ballou, Heiko Breitsohl, Dominic Kao, Kathrin Gerling, Christoph Sebastian Deterding

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

Abstract

Effectance—the basic positive experience of causing effects—provides a promising explanation for the enjoyment derived from novel low-challenge game genres featuring ample ‘juicy’ feedback. To date, game researchers have studied effectance using a little-validated 11-item scale developed by Klimmt, Hartmann, and Frey. To test its dimensionality and discriminant validity, we conducted an online survey (n = 467) asking people to report on effectance and related experiences in a recent play session. Confirmatory and exploratory factor analyses show poor fit with a unidimensional factor structure and poor discriminant validity with common enjoyment and mastery/competence measures, likely due to reverse-coded items and a separable input lag factor. We discuss further possible validity issues like questionable content validity, advise against using the scale in its present form, and close with recommendations for future scale development and use.

Original language	English
Title of host publication	xtended Abstracts of the 2021 Annual Symposium on Computer-Human Interaction in Play (CHI PLAY ’21),
Publisher	ACM
Number of pages	6
DOIs	https://doi.org/10.1145/3450337.3483492
Publication status	Accepted/In press - 25 Aug 2021

Keywords

effectance
games
player experience
measurement validity

Access to Document

10.1145/3450337.3483492Licence: Unspecified

notveryeffective
Permission to make digital or hard copies of part or all of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for third-party components of this work must be honored. For all other uses, contact the owner/author(s).
Accepted author manuscript, 1.15 MBLicence: Other

The Digital Creativity Hub
Cowling, P. I., Austin, J., Cairns, P. A., Holliman, N. S., Hook, J. D., Marsden, E., Murphy, D. T., Petrie, H., Reed, D. J., Richards, J. D., Ursu, M., Wade, A., Baier, H., Beale, G., Block, F. O., Deterding, C. S., Devlin, S., Drachen, A., Kasprowicz, R. E., Smith, D. P. & Williams, D. A. H.
EPSRC
1/10/15 → 30/09/22
Project: Research project (funded) › Research

Cite this

@inproceedings{62147bebf7a14571a99a387843e18215,

title = "Not Very Effective: Validity Issues of the Effectance in Games Scale",

abstract = "Effectance—the basic positive experience of causing effects—provides a promising explanation for the enjoyment derived from novel low-challenge game genres featuring ample {\textquoteleft}juicy{\textquoteright} feedback. To date, game researchers have studied effectance using a little-validated 11-item scale developed by Klimmt, Hartmann, and Frey. To test its dimensionality and discriminant validity, we conducted an online survey (n = 467) asking people to report on effectance and related experiences in a recent play session. Confirmatory and exploratory factor analyses show poor fit with a unidimensional factor structure and poor discriminant validity with common enjoyment and mastery/competence measures, likely due to reverse-coded items and a separable input lag factor. We discuss further possible validity issues like questionable content validity, advise against using the scale in its present form, and close with recommendations for future scale development and use.",

keywords = "effectance, games, player experience, measurement validity",

author = "Nicholas Ballou and Heiko Breitsohl and Dominic Kao and Kathrin Gerling and Deterding, {Christoph Sebastian}",

year = "2021",

month = aug,

day = "25",

doi = "10.1145/3450337.3483492",

language = "English",

booktitle = "xtended Abstracts of the 2021 Annual Symposium on Computer-Human Interaction in Play (CHI PLAY {\textquoteright}21),",

publisher = "ACM",

address = "United States",

}

TY - GEN

T1 - Not Very Effective

T2 - Validity Issues of the Effectance in Games Scale

AU - Ballou, Nicholas

AU - Breitsohl, Heiko

AU - Kao, Dominic

AU - Gerling, Kathrin

AU - Deterding, Christoph Sebastian

PY - 2021/8/25

Y1 - 2021/8/25

N2 - Effectance—the basic positive experience of causing effects—provides a promising explanation for the enjoyment derived from novel low-challenge game genres featuring ample ‘juicy’ feedback. To date, game researchers have studied effectance using a little-validated 11-item scale developed by Klimmt, Hartmann, and Frey. To test its dimensionality and discriminant validity, we conducted an online survey (n = 467) asking people to report on effectance and related experiences in a recent play session. Confirmatory and exploratory factor analyses show poor fit with a unidimensional factor structure and poor discriminant validity with common enjoyment and mastery/competence measures, likely due to reverse-coded items and a separable input lag factor. We discuss further possible validity issues like questionable content validity, advise against using the scale in its present form, and close with recommendations for future scale development and use.

AB - Effectance—the basic positive experience of causing effects—provides a promising explanation for the enjoyment derived from novel low-challenge game genres featuring ample ‘juicy’ feedback. To date, game researchers have studied effectance using a little-validated 11-item scale developed by Klimmt, Hartmann, and Frey. To test its dimensionality and discriminant validity, we conducted an online survey (n = 467) asking people to report on effectance and related experiences in a recent play session. Confirmatory and exploratory factor analyses show poor fit with a unidimensional factor structure and poor discriminant validity with common enjoyment and mastery/competence measures, likely due to reverse-coded items and a separable input lag factor. We discuss further possible validity issues like questionable content validity, advise against using the scale in its present form, and close with recommendations for future scale development and use.

KW - effectance

KW - games

KW - player experience

KW - measurement validity

U2 - 10.1145/3450337.3483492

DO - 10.1145/3450337.3483492

M3 - Conference contribution

BT - xtended Abstracts of the 2021 Annual Symposium on Computer-Human Interaction in Play (CHI PLAY ’21),

PB - ACM

ER -

Not Very Effective: Validity Issues of the Effectance in Games Scale

Abstract

Keywords

Access to Document

Projects

The Digital Creativity Hub

Cite this