[Paper Review] Language Models as Critical Thinking Tools: A Case Study of Philosophers

Andre Ye, Jared Moore+2 more|arXiv (Cornell University)|Apr 6, 2024

Education and Critical Thinking Development4 citations

TL;DR

This paper investigates language models (LMs) as critical thinking tools through interviews with 21 professional philosophers, revealing that current LMs lack selfhood and initiative—key traits for deep, reflective thinking. The authors propose the selfhood-initiative model and three roles (Interlocutor, Monitor, Respondent) to guide future LM development for philosophical and broader critical inquiry.

ABSTRACT

Current work in language models (LMs) helps us speed up or even skip thinking by accelerating and automating cognitive work. But can LMs help us with critical thinking -- thinking in deeper, more reflective ways which challenge assumptions, clarify ideas, and engineer new concepts? We treat philosophy as a case study in critical thinking, and interview 21 professional philosophers about how they engage in critical thinking and on their experiences with LMs. We find that philosophers do not find LMs to be useful because they lack a sense of selfhood (memory, beliefs, consistency) and initiative (curiosity, proactivity). We propose the selfhood-initiative model for critical thinking tools to characterize this gap. Using the model, we formulate three roles LMs could play as critical thinking tools: the Interlocutor, the Monitor, and the Respondent. We hope that our work inspires LM researchers to further develop LMs as critical thinking tools and philosophers and other 'critical thinkers' to imagine intellectually substantive uses of LMs.

Motivation & Objective

• To investigate whether language models (LMs) can serve as tools for deep, reflective critical thinking, using philosophers as a case study.
• To understand the limitations of current LMs in supporting critical thinking, particularly in terms of selfhood and initiative.
• To identify how LMs could be redesigned to support complex intellectual inquiry in philosophy and related domains.
• To propose a conceptual framework—selfhood-initiative model—for evaluating and guiding the development of critical thinking tools.
• To explore the ethical and practical implications of integrating LMs into philosophical and intellectual workflows.

Proposed method

• Conducted semi-structured interviews with 21 professional philosophers to explore their critical thinking practices and experiences with LMs.
• Analyzed interview data to identify recurring themes around the perceived lack of selfhood (memory, beliefs, consistency) and initiative (curiosity, proactivity) in current LMs.
• Proposed the selfhood-initiative model as a conceptual framework to explain why human and textual resources are more effective than current LMs for critical thinking.
• Defined three potential roles for future LMs in critical thinking: Interlocutor (engaging in reflective dialogue), Monitor (evaluating reasoning for flaws), and Respondent (providing targeted, conceptually rich responses).
• Used philosophical and cognitive science literature to ground the model in theories of reflection, judgment, and intellectual engagement.
• Explored implications for LM design, interface development, and ethical considerations such as authorship and academic integrity.

Figure 1: Left – The selfhood-initiative model for critical thinking tools. Right – Illustrative sample interactions between humans and LMs playing different roles. Other alternatives are possible. The excerpt from “The Monitor” is taken from Singer ( 1972 ) .

Experimental results

Research questions

• RQ1 Why do professional philosophers find current language models ineffective for critical thinking despite their utility in idea generation and expression?
• RQ2 What core cognitive and experiential qualities are missing in current LMs that prevent them from supporting deep, reflective thinking?
• RQ3 How can language models be reimagined to support critical thinking in ways that mirror human philosophical dialogue and inquiry?
• RQ4 What roles could future LMs play in philosophical reasoning, and how would these roles differ from existing applications?
• RQ5 What ethical and design challenges arise when integrating LMs as critical thinking partners in intellectual disciplines?

Key findings

• Philosophers reported that current LMs lack selfhood—specifically, memory, consistent beliefs, and a sense of identity—making them feel detached and untrustworthy in critical dialogue.
• Philosophers also found LMs lacking in initiative, describing them as passive, uncurious, and unproactive, which undermines their ability to challenge assumptions or explore ideas deeply.
• The selfhood-initiative model was developed to explain why human interlocutors and philosophical texts are more effective for critical thinking than current LMs.
• Interviews revealed that philosophers value the emotional and intellectual engagement of human conversation, including shared subjectivity and the joy of intellectual struggle.
• Philosophers expressed concern that LMs may disrupt the process of thinking through language, especially when ideas are difficult to articulate or when there is a mismatch between thought and expression.
• Despite limitations, philosophers acknowledged that LMs could still play valuable roles if redesigned to support critical thinking—specifically as Interlocutor, Monitor, or Respondent.