[Paper Review] ChatGPT may excel in States Medical Licensing Examination but falters in basic Linear Algebra

Eli Bagno, Thierry Dana-Picard+1 more|arXiv (Cornell University)|Jun 23, 2023

Artificial Intelligence in Healthcare and Education4 citations

TL;DR

This paper evaluates ChatGPT's performance in basic Linear Algebra, revealing that while it delivers plausible, well-structured responses, it frequently commits fundamental mathematical errors and logical inconsistencies—such as providing incorrect proofs and failing to recognize contradictions—undermining its reliability as a teaching tool despite its strong performance on standardized exams like the USMLE.

ABSTRACT

The emergence of ChatGPT has been rapid, and although it has demonstrated positive impacts in certain domains, its influence is not universally advantageous. Our analysis focuses on ChatGPT's capabilities in Mathematics Education, particularly in teaching basic Linear Algebra. While there are instances where ChatGPT delivers accurate and well-motivated answers, it is crucial to recognize numerous cases where it makes significant mathematical errors and fails in logical inference. These occurrences raise concerns regarding the system's genuine understanding of mathematics, as it appears to rely more on visual patterns rather than true comprehension. Additionally, the suitability of ChatGPT as a teacher for students also warrants consideration.

Motivation & Objective

• To assess ChatGPT’s reliability in teaching and solving foundational problems in Linear Algebra.
• To investigate whether ChatGPT’s responses exhibit logical consistency and mathematical accuracy in core concepts.
• To evaluate the risks of relying on large language models as educational assistants in mathematics, especially for novice learners.
• To explore the implications of AI-generated misinformation in academic settings, particularly when presented with high confidence.

Proposed method

• The researchers conducted a series of structured dialogues with ChatGPT, posing progressively complex questions in Linear Algebra.
• They tested ChatGPT’s ability to define vector spaces, compute dimensions, and verify bases, including the space ℝ₄[x] of real polynomials up to degree 4.
• They introduced contradictions by asking follow-up questions that exposed inconsistencies in earlier responses.
• They analyzed the coherence and logical structure of ChatGPT’s reasoning, particularly when it failed to backtrack from incorrect conclusions.
• They evaluated the model’s response to requests for proofs, such as the dimension theorem for linear transformations.
• They examined how ChatGPT handles self-contradictory claims and whether it acknowledges errors when confronted.

Experimental results

Research questions

• RQ1 Can ChatGPT accurately and consistently solve basic problems in Linear Algebra, such as determining the dimension of a vector space?
• RQ2 Does ChatGPT recognize and correct logical inconsistencies in its own reasoning when confronted with contradictions?
• RQ3 To what extent do ChatGPT’s responses appear mathematically sound to untrained users despite containing fundamental errors?
• RQ4 How does the model’s behavior reflect a lack of true mathematical understanding, even when it produces plausible explanations?
• RQ5 What are the pedagogical risks of using large language models like ChatGPT as teaching assistants in mathematics education?

Key findings

• ChatGPT correctly identified the dimension of ℝ₄[x] as 5, demonstrating competence in basic definitions.
• It affirmed that the standard monomial basis {1, x, x², x³, x⁴} is a valid basis for ℝ₄[x], which is accurate.
• When asked whether all bases of a vector space have the same number of elements, ChatGPT gave a correct answer with a detailed explanation.
• ChatGPT failed to recognize that the set {1−x, x−x², x²−x³, x⁴−x³} cannot be a basis of ℝ₄[x], as it is linearly dependent and has only four elements, contradicting the earlier claim about basis cardinality.
• When confronted with the contradiction, ChatGPT eventually acknowledged the error, but only after a lengthy and misleading chain of reasoning.
• The model often produces convincing but incorrect proofs and fails to detect internal inconsistencies, suggesting reliance on pattern matching rather than genuine comprehension.