Evaluating Interactive AI: Understanding and Controlling Placebo Effects in Human-AI Interaction

Steeven Villa; Robin Welsch; Alena Denisova; Thomas Kosch

doi:10.1145/3613905.3636304

Evaluating Interactive AI: Understanding and Controlling Placebo Effects in Human-AI Interaction

Steeven Villa, Robin Welsch, Alena Denisova, Thomas Kosch

Computer Science

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

Abstract

In the medical field, patients often experience tangible benefits from treatments they expect will improve their condition, even if the treatment has no mechanism of effect. This phenomenon often obscuring scientific evaluation of human treatment is termed the "placebo effect." Latest research in human-computer interaction has shown that using cutting-edge technologies similarly raises expectations of improvement, culminating in placebo effects that undermine evaluation efforts for user studies. This workshop delves into the role of placebo effects in human-computer interaction for cutting-edge technologies such as artificial intelligence, its influence as a confounding factor in user studies, and identifies methods that researchers can adopt to reduce its impact on study findings. By the end of this workshop, attendees will be equipped to incorporate placebo control measures in their experimental designs.

Original language	English
Title of host publication	CHI EA '24: Extended Abstracts of the CHI Conference on Human Factors in Computing Systems
Publisher	ACM
Number of pages	4
ISBN (Print)	979-8-4007-0331-7
DOIs	https://doi.org/10.1145/3613905.3636304
Publication status	Published - 11 May 2024

Bibliographical note

This is an author-produced version of the published paper. Uploaded in accordance with the University’s Research Publications and Open Access policy.

Access to Document

10.1145/3613905.3636304

_AHs_23__Placebo_Workshop
This is an author-produced version of the published paper. Uploaded in accordance with the University’s Research Publications and Open Access policy.
Accepted author manuscript, 547 KBLicence: CC BY

Cite this

@inproceedings{2b4f41603aed4a3ca64c6c63d80f22db,

title = "Evaluating Interactive AI: Understanding and Controlling Placebo Effects in Human-AI Interaction",

abstract = "In the medical field, patients often experience tangible benefits from treatments they expect will improve their condition, even if the treatment has no mechanism of effect. This phenomenon often obscuring scientific evaluation of human treatment is termed the {"}placebo effect.{"} Latest research in human-computer interaction has shown that using cutting-edge technologies similarly raises expectations of improvement, culminating in placebo effects that undermine evaluation efforts for user studies. This workshop delves into the role of placebo effects in human-computer interaction for cutting-edge technologies such as artificial intelligence, its influence as a confounding factor in user studies, and identifies methods that researchers can adopt to reduce its impact on study findings. By the end of this workshop, attendees will be equipped to incorporate placebo control measures in their experimental designs.",

author = "Steeven Villa and Robin Welsch and Alena Denisova and Thomas Kosch",

note = "This is an author-produced version of the published paper. Uploaded in accordance with the University{\textquoteright}s Research Publications and Open Access policy.",

year = "2024",

month = may,

day = "11",

doi = "10.1145/3613905.3636304",

language = "English",

isbn = "979-8-4007-0331-7",

booktitle = "CHI EA '24: Extended Abstracts of the CHI Conference on Human Factors in Computing Systems",

publisher = "ACM",

address = "United States",

}

TY - GEN

T1 - Evaluating Interactive AI: Understanding and Controlling Placebo Effects in Human-AI Interaction

AU - Villa, Steeven

AU - Welsch, Robin

AU - Denisova, Alena

AU - Kosch, Thomas

N1 - This is an author-produced version of the published paper. Uploaded in accordance with the University’s Research Publications and Open Access policy.

PY - 2024/5/11

Y1 - 2024/5/11

N2 - In the medical field, patients often experience tangible benefits from treatments they expect will improve their condition, even if the treatment has no mechanism of effect. This phenomenon often obscuring scientific evaluation of human treatment is termed the "placebo effect." Latest research in human-computer interaction has shown that using cutting-edge technologies similarly raises expectations of improvement, culminating in placebo effects that undermine evaluation efforts for user studies. This workshop delves into the role of placebo effects in human-computer interaction for cutting-edge technologies such as artificial intelligence, its influence as a confounding factor in user studies, and identifies methods that researchers can adopt to reduce its impact on study findings. By the end of this workshop, attendees will be equipped to incorporate placebo control measures in their experimental designs.

AB - In the medical field, patients often experience tangible benefits from treatments they expect will improve their condition, even if the treatment has no mechanism of effect. This phenomenon often obscuring scientific evaluation of human treatment is termed the "placebo effect." Latest research in human-computer interaction has shown that using cutting-edge technologies similarly raises expectations of improvement, culminating in placebo effects that undermine evaluation efforts for user studies. This workshop delves into the role of placebo effects in human-computer interaction for cutting-edge technologies such as artificial intelligence, its influence as a confounding factor in user studies, and identifies methods that researchers can adopt to reduce its impact on study findings. By the end of this workshop, attendees will be equipped to incorporate placebo control measures in their experimental designs.

U2 - 10.1145/3613905.3636304

DO - 10.1145/3613905.3636304

M3 - Conference contribution

SN - 979-8-4007-0331-7

BT - CHI EA '24: Extended Abstracts of the CHI Conference on Human Factors in Computing Systems

PB - ACM

ER -