[論文レビュー] A Survey on Large Language Model Hallucination via a Creativity Perspective

Xuhui Jiang, Yuxing Tian+4 more|arXiv (Cornell University)|Feb 2, 2024

Online Learning and Analytics被引用数 10

ひとこと要約

論文は LLM の幻覚を純粋に否定的な現象として捉えるのではなく、潜在的な創造性として再定義し、分類法、創造性の評価、および発散的思考と収束的思考の段階を通じて幻覚を活用する方法をレビューする。

ABSTRACT

Hallucinations in large language models (LLMs) are always seen as limitations. However, could they also be a source of creativity? This survey explores this possibility, suggesting that hallucinations may contribute to LLM application by fostering creativity. This survey begins with a review of the taxonomy of hallucinations and their negative impact on LLM reliability in critical applications. Then, through historical examples and recent relevant theories, the survey explores the potential creative benefits of hallucinations in LLMs. To elucidate the value and evaluation criteria of this connection, we delve into the definitions and assessment methods of creativity. Following the framework of divergent and convergent thinking phases, the survey systematically reviews the literature on transforming and harnessing hallucinations for creativity in LLMs. Finally, the survey discusses future research directions, emphasizing the need to further explore and refine the application of hallucinations in creative processes within LLMs.

研究の動機と目的

• Define and categorize LLM hallucinations and their negative impact in critical tasks.
• Review creativity conceptions from cognitive science and how they apply to LLMs.
• Explore how hallucinations can be transformed into valuable creative outputs.
• Propose divergent and convergent phases for harnessing hallucinations in creative tasks.
• Identify future directions for datasets, metrics, and frameworks to balance creativity with reliability.

提案手法

• Survey existing LLM hallucination taxonomies and detection/reduction strategies.
• Analyze historical and cognitive science literature to relate hallucination to creativity.
• Discuss definitions and measures of creativity relevant to LLMs.
• Present a bifurcated framework (divergent and convergent phases) for generating and refining creative hallucinations.
• Review empirical approaches to measure LLM creativity and the role of prompts, interaction, and evaluation.
• Outline future research directions and methodological needs for harnessing hallucinations creatively.

実験結果

リサーチクエスチョン

• RQ1 Is hallucination in LLMs always harmful, or can it foster creativity?
• RQ2 How can we effectively harness hallucinations for creativity while mitigating risks?
• RQ3 What definitions and metrics of creativity are suitable for evaluating LLM outputs?
• RQ4 How can divergent and convergent thinking frameworks be operationalized in LLMs for creativity?
• RQ5 What future datasets, benchmarks, and training approaches are needed to study LLM creativity and hallucination interplay?

主な発見

• Hallucinations may serve as catalysts for creativity rather than solely being detrimental.
• A two-phase framework (divergent phase for creative hallucination generation and convergent phase for evaluation and refinement) is proposed.
• Creativity in LLMs can be assessed using cognitive-science-inspired metrics and adapted benchmarks, though evaluation remains challenging.
• Current literature covers detection and reduction of hallucinations but there is a need for systematic approaches to identify and leverage beneficial hallucinations.
• Human-in-the-loop and multi-agent interactions, prompts, and training strategies can enhance the diversity and quality of creative outputs.
• There is a demand for richer datasets, benchmarks, and theoretical grounding to synthesize creativity with reliability in LLMs.