Wait, But Why? Assessing Behavior Explanation Strategies for Real-Time Strategy Games

TY - GEN

T1 - Wait, But Why? Assessing Behavior Explanation Strategies for Real-Time Strategy Games

AU - Robertson, Justin

AU - Kokkinakis, Athanasios

AU - Hook, Jonathan David

AU - Kirman, Ben

AU - Block, Florian Oliver

AU - Ursu, Marian

AU - Patra, Sagarika

AU - Demediuk, Simon Peter

AU - Drachen, Anders

AU - Olarewaju, Olu

N1 - This is an author-produced version of the published paper. Uploaded in accordance with the publisher’s self-archiving policy. Further copying may not be permitted; contact the publisher for details

PY - 2021/5/13

Y1 - 2021/5/13

N2 - Work in AI-based explanation systems has uncovered an interesting contradiction: people prefer and learn best from 'why' explanations but expert esports commentators primarily answer 'what' questions when explaining complex behavior in real-time strategy games. Three possible explanations for this contradiction are: 1.) broadcast audiences are well-informed and do not need 'why' explanations; 2.) consuming 'why' explanations in real-time is too cognitively demanding for audiences; or 3.) producing live 'why' explanations is too difficult for commentators. We answer this open question by investigating the effects of explanation types and presentation modalities on audience recall and cognitive load in the context of an esports broadcast. We recruit 131 Dota 2 players and split them into three groups: the first group views a Dota 2 broadcast, the second group has the addition of an interactive map that provides 'what' explanations, and the final group receives the interactive map with detailed 'why' explanations. We find that participants who receive short interactive text prompts that provide 'what' explanations outperform the other two groups on a multiple-choice recall task. We also find that participants who receive detailed 'why' explanations submit the highest reports of cognitive load. Our evidence supports the conclusion that informed audiences benefit from explanations but do not have the cognitive resources to process 'why' answers in real-time. It also supports the conclusion that stacked explanation interventions across different modalities, like audio, interactivity, and text, can aid real-time comprehension when attention resources are limited. Together, our results indicate that interactive multimedia interfaces can be leveraged to quickly guide attention and provide low-cost explanations to improve intelligibility when time is too scarce for cognitively demanding 'why' explanations.

AB - Work in AI-based explanation systems has uncovered an interesting contradiction: people prefer and learn best from 'why' explanations but expert esports commentators primarily answer 'what' questions when explaining complex behavior in real-time strategy games. Three possible explanations for this contradiction are: 1.) broadcast audiences are well-informed and do not need 'why' explanations; 2.) consuming 'why' explanations in real-time is too cognitively demanding for audiences; or 3.) producing live 'why' explanations is too difficult for commentators. We answer this open question by investigating the effects of explanation types and presentation modalities on audience recall and cognitive load in the context of an esports broadcast. We recruit 131 Dota 2 players and split them into three groups: the first group views a Dota 2 broadcast, the second group has the addition of an interactive map that provides 'what' explanations, and the final group receives the interactive map with detailed 'why' explanations. We find that participants who receive short interactive text prompts that provide 'what' explanations outperform the other two groups on a multiple-choice recall task. We also find that participants who receive detailed 'why' explanations submit the highest reports of cognitive load. Our evidence supports the conclusion that informed audiences benefit from explanations but do not have the cognitive resources to process 'why' answers in real-time. It also supports the conclusion that stacked explanation interventions across different modalities, like audio, interactivity, and text, can aid real-time comprehension when attention resources are limited. Together, our results indicate that interactive multimedia interfaces can be leveraged to quickly guide attention and provide low-cost explanations to improve intelligibility when time is too scarce for cognitively demanding 'why' explanations.

M3 - Conference contribution

BT - Proceedings of the 2021 IUI Annual Conference of Intelligent User Interfaces

PB - ACM

T2 - ACM IUI 2020 Annual Conference on Intelligent User Interfaces

Y2 - 13 April 2021 through 17 April 2021

ER -