An AI tool for scaffolding complex thinking: challenges and solutions in developing an LLM prompt protocol suite

Ana Teixeira de Melo; Leo Caves; Letícia  Renault; Philip Garnett; Paula Duarte Lopes; Raquel Ribeiro; Filipe Santos

doi:10.1007/s10111-025-00817-6

An AI tool for scaffolding complex thinking: challenges and solutions in developing an LLM prompt protocol suite

Ana Teixeira de Melo, Leo Caves, Letícia Renault, Philip Garnett, Paula Duarte Lopes, Raquel Ribeiro, Filipe Santos

Research output: Contribution to journal › Article › peer-review

Abstract

This paper reports an exploratory study examining the interaction between a theoretical framework for Complex Thinking and AI (LLMs), in terms of its potentialities and constraints. The aim was to develop and conduct a preliminary pilot evaluation of a tool comprising a prompt protocol suite for use with an LLM, to scaffold Complex Thinking. The tool is designed for use by an individual or group in relation to a given Target System of Interest (i.e., a real-world system, a problem, or a concern), supporting the development of more complex understandings of such systems that can guide more effective and positive actions and decisions. We describe the process of developing a suite of prompt protocols for scaffolding particular properties of Complex Thinking and report on the outcomes of a pilot test evaluation with a set of users across different domains.

Original language	English
Pages (from-to)	651-693
Number of pages	43
Journal	Cognition, Tehcnology & Work
Volume	27
Early online date	14 Jul 2025
DOIs	https://doi.org/10.1007/s10111-025-00817-6
Publication status	Published - 1 Sept 2025

Bibliographical note

Keywords

Complex Thinking
large language models
Prompt protocols
Scaffolding
artificial intelligence

Access to Document

10.1007/s10111-025-00817-6Licence: CC BY

An AI tool for scaffolding complex thinking
© The Author(s) 2025
Final published version, 1.88 MBLicence: CC BY

Cite this

@article{c3eb37702e1745a78f11afb995a5e4bf,

title = "An AI tool for scaffolding complex thinking: challenges and solutions in developing an LLM prompt protocol suite",

abstract = "This paper reports an exploratory study examining the interaction between a theoretical framework for Complex Thinking and AI (LLMs), in terms of its potentialities and constraints. The aim was to develop and conduct a preliminary pilot evaluation of a tool comprising a prompt protocol suite for use with an LLM, to scaffold Complex Thinking. The tool is designed for use by an individual or group in relation to a given Target System of Interest (i.e., a real-world system, a problem, or a concern), supporting the development of more complex understandings of such systems that can guide more effective and positive actions and decisions. We describe the process of developing a suite of prompt protocols for scaffolding particular properties of Complex Thinking and report on the outcomes of a pilot test evaluation with a set of users across different domains.",

keywords = "Complex Thinking, large language models, Prompt protocols, Scaffolding, artificial intelligence",

author = "{Teixeira de Melo}, Ana and Leo Caves and Let{\'i}cia Renault and Philip Garnett and Lopes, {Paula Duarte} and Raquel Ribeiro and Filipe Santos",

note = "{\textcopyright} The Author(s) 2025",

year = "2025",

month = sep,

day = "1",

doi = "10.1007/s10111-025-00817-6",

language = "English",

volume = "27",

pages = "651--693",

journal = "Cognition, Tehcnology & Work",

issn = "1435-5558",

publisher = "Springer London",

}

TY - JOUR

T1 - An AI tool for scaffolding complex thinking

T2 - challenges and solutions in developing an LLM prompt protocol suite

AU - Teixeira de Melo, Ana

AU - Caves, Leo

AU - Renault, Letícia

AU - Garnett, Philip

AU - Lopes, Paula Duarte

AU - Ribeiro, Raquel

AU - Santos, Filipe

PY - 2025/9/1

Y1 - 2025/9/1

N2 - This paper reports an exploratory study examining the interaction between a theoretical framework for Complex Thinking and AI (LLMs), in terms of its potentialities and constraints. The aim was to develop and conduct a preliminary pilot evaluation of a tool comprising a prompt protocol suite for use with an LLM, to scaffold Complex Thinking. The tool is designed for use by an individual or group in relation to a given Target System of Interest (i.e., a real-world system, a problem, or a concern), supporting the development of more complex understandings of such systems that can guide more effective and positive actions and decisions. We describe the process of developing a suite of prompt protocols for scaffolding particular properties of Complex Thinking and report on the outcomes of a pilot test evaluation with a set of users across different domains.

AB - This paper reports an exploratory study examining the interaction between a theoretical framework for Complex Thinking and AI (LLMs), in terms of its potentialities and constraints. The aim was to develop and conduct a preliminary pilot evaluation of a tool comprising a prompt protocol suite for use with an LLM, to scaffold Complex Thinking. The tool is designed for use by an individual or group in relation to a given Target System of Interest (i.e., a real-world system, a problem, or a concern), supporting the development of more complex understandings of such systems that can guide more effective and positive actions and decisions. We describe the process of developing a suite of prompt protocols for scaffolding particular properties of Complex Thinking and report on the outcomes of a pilot test evaluation with a set of users across different domains.

KW - Complex Thinking

KW - large language models

KW - Prompt protocols

KW - Scaffolding

KW - artificial intelligence

U2 - 10.1007/s10111-025-00817-6

DO - 10.1007/s10111-025-00817-6

M3 - Article

SN - 1435-5558

VL - 27

SP - 651

EP - 693

JO - Cognition, Tehcnology & Work

JF - Cognition, Tehcnology & Work

ER -