[Paper Review] DynHD: Hallucination Detection for Diffusion Large Language Models via Denoising Dynamics Deviation Learning

Yanyu Qian, Yue Tan+3 more|arXiv (Cornell University)|Mar 17, 2026

Mental Health via Writing0 citations

TL;DR

DynHD detects hallucinations in diffusion LLMs by constructing semantic-aware evidence from token entropy and modeling denoising dynamics with a reference trajectory and deviation-based detector, achieving state-of-the-art AUROC across benchmarks.

ABSTRACT

Diffusion large language models (D-LLMs) have emerged as a promising alternative to auto-regressive models due to their iterative refinement capabilities. However, hallucinations remain a critical issue that hinders their reliability. To detect hallucination responses from model outputs, token-level uncertainty (e.g., entropy) has been widely used as an effective signal to indicate potential factual errors. Nevertheless, the fixed-length generation paradigm of D-LLMs implies that tokens contribute unevenly to hallucination detection, with only a small subset providing meaningful signals. Moreover, the evolution trend of uncertainty throughout the diffusion process can also provide important signals, highlighting the necessity of modeling its denoising dynamics for hallucination detection. In this paper, we propose DynHD that bridge these gaps from both spatial (token sequence) and temporal (denoising dynamics) perspectives. To address the information density imbalance across tokens, we propose a semantic-aware evidence construction module that extracts hallucination-indicative signals by filtering out non-informative tokens and emphasizing semantically meaningful ones. To model denoising dynamics for hallucination detection, we introduce a reference evidence generator that learns the expected evolution trajectory of uncertainty evidence, along with a deviation-based hallucination detector that makes predictions by measuring the discrepancy between the observed and reference trajectories. Extensive experiments demonstrate that DynHD consistently outperforms state-of-the-art baselines while achieving higher efficiency across multiple benchmarks and backbone models.

Motivation & Objective

• Motivate reliable hallucination detection for diffusion LLMs (D-LLMs) which use fixed-length sequences and iterative denoising.
• Address information density imbalance across tokens that dilutes detection signals.
• Model the temporal evolution of uncertainty (denoising dynamics) to capture process-level signals of hallucinations.
• Develop a two-stage framework that constructs semantic-aware evidence and learns deviations from a reference trajectory.
• Demonstrate robustness and efficiency across multiple datasets and backbone D-LLMs.

Proposed method

• Semantic-aware evidence construction: filter out non-informative structural tokens and summarize entropy of semantic tokens via three statistics per step (mean across semantic tokens, max entropy, and mean of top-k entropy).
• Build an evidence trajectory E = (a_T, a_{T-1}, ..., a_0) from step-wise statistics.
• Dynamical deviation learning: train a reference evidence dynamics generator g_theta conditioned on the query to model normal evidence evolution.
• Use a deviation-based detector that combines observed evidence a_t, reference a_hat_t, and their difference Delta a_t with learnable temporal weights to predict hallucination.
• Introduce regularizers that emphasize late-stage stagnation and potential rebound in uncertainty, guided by an EMA-based adaptive boundary.
• End-to-end objective combines classification loss with path and rebound regularizers (L_cls + lambda1*L_path + lambda2*L_reb).

Figure 1: Visualization of spatial uncertainty distribution during decoding. Tokens exhibiting the highest entropy spikes serve as the primary indicators of factual instability. On the contrary, intermediate and structural tokens provide limited cues for hallucination detection.

Experimental results

Research questions

• RQ1 How can semantic token filtering and multivariate entropy statistics improve hallucination signals in D-LLMs?
• RQ2 Can modeling the denoising dynamics with a reference trajectory improve detection of hallucinations over state-of-the-art trajectory-based methods?
• RQ3 Do late-stage dynamics (stagnation/rebound) provide stronger cues for factuality in D-LLMs across datasets?
• RQ4 Is the DynHD framework robust and efficient across different D-LLM backbones and QA tasks?
• RQ5 What is the impact of ablations on evidence construction and deviation modeling components on detection performance?

Key findings

• DynHD achieves state-of-the-art AUROC on TriviaQA, HotpotQA, and CSQA across LLaDA-8B-Instruct and Dream-7B-Instruct backbones, with average gains over baselines.
• DynHD outperforms TraceDet by an average AUROC margin of 12.2% on reported settings.
• Ablation studies show token filtering and the three entropy statistics are essential for strong performance; removing components degrades AUROC.
• Uniform temporal pooling and attention-based weighting improve the temporal aggregation of evidence, with late-step emphasis aligning with stagnation/rebound signals.
• DynHD demonstrates favorable speed-accuracy trade-offs, being more efficient than multi-sample methods while delivering higher accuracy.

(a)