[Paper Review] DuCCAE: A Hybrid Engine for Immersive Conversation via Collaboration, Augmentation, and Evolution

Xin Shen, Zhishu Jiang+14 more|arXiv (Cornell University)|Feb 25, 2026

Social Robot Interaction and HRI0 citations

TL;DR

DuCCAE decouples real-time response from asynchronous agentic execution to enable immersive conversations with long-horizon tasks, achieving improved reliability and production-scale effectiveness in Baidu Search.

ABSTRACT

Immersive conversational systems in production face a persistent trade-off between responsiveness and long-horizon task capability. Real-time interaction is achievable for lightweight turns, but requests involving planning and tool invocation (e.g., search and media generation) produce heavy-tail execution latency that degrades turn-taking, persona consistency, and user trust. To address this challenge, we propose DuCCAE (Conversation while Collaboration with Augmentation and Evolution), a hybrid engine for immersive conversation deployed within Baidu Search, serving millions of users. DuCCAE decouples real-time response generation from asynchronous agentic execution and synchronizes them via a shared state that maintains session context and execution traces, enabling asynchronous results to be integrated back into the ongoing dialogue. The system orchestrates five subsystems-Info, Conversation, Collaboration, Augmentation, and Evolution-to support multi-agent collaboration and continuous improvement. We evaluate DuCCAE through a comprehensive framework that combines offline benchmarking on the Du-Interact dataset and large-scale production evaluation within Baidu Search. Experimental results demonstrate that DuCCAE outperforms strong baselines in agentic execution reliability and dialogue quality while reducing latency to fit strict real-time budgets. Crucially, deployment metrics since June 2025 confirm substantial real-world effectiveness, evidenced by a tripling of Day-7 user retention to 34.2% and a surge in the complex task completion rate to 65.2%. Our hybrid architecture successfully preserves conversational continuity while enabling reliable agentic execution, offering practical guidelines for deploying scalable agentic systems in industrial settings.

Motivation & Objective

• Address the tension between real-time responsiveness and long-horizon task capability in immersive conversational systems.
• Propose a latency-decoupled architecture that synchronizes fast real-time responses with asynchronous agentic reasoning.
• Develop an evolving, data-driven pipeline to continually improve system performance via automated and human-in-the-loop evaluation.
• Deploy and evaluate the system in a large-scale industrial setting (Baidu Search) to demonstrate production viability and business impact.

Proposed method

• Introduce a dual-track dataflow with Fast Track for real-time responses and Slow Track for asynchronous reasoning.
• Implement a Unified Shared State to synchronize outputs from both tracks without breaking immersion.
• Orchestrate five subsystems (Info, Conversation, Collaboration, Augmentation, Evolution) for perception, routing, multi-agent planning, tool use, and learning.
• Use Retrieval-Augmented Generation (RAG) and a Model Context Protocol (MCP) to ground outputs with external knowledge and tools.
• Employ an Evolution System with Data Flywheel for automated assessment, SFT, and RL to continuously improve the agents.

Experimental results

Research questions

• RQ1 Can a latency-decoupled, dual-track architecture preserve conversational continuity while enabling reliable long-horizon tool use?
• RQ2 Does evolutionary, domain-specific training enable mid-sized models to surpass larger general-purpose models in complex task execution?
• RQ3 What are the benefits of a multi-agent Collaboration System for asynchronous planning and execution in production settings?
• RQ4 How does memory separation (User vs. Agent Memory) affect persona consistency and user trust over time?
• RQ5 What production-level metrics (retention, complex task completion) improve with such a system in a large-scale deployment?

Key findings

• DuCCAE achieves sub-second responses on the Fast Track while enabling asynchronous collaboration on the Slow Track.
• In production, Day-7 user retention tripled to 34.2% and Complex Task Completion Rate rose to 65.2%.
• DuCCAE-V3 attains 82.5% dispatch precision, 72.4% success rate, and 71.1% fidelity with Avg latency 1,880 ms.
• Evolutionary data (Du-Interact-Evo) with SFT and RL substantially improves performance beyond larger vanilla models (e.g., 70B) in SR.
• Visual perception ablation shows a 77% latency reduction when using a caption-based visual-to-text approach (480 ms vs 2100 ms).
• The system sustains longer conversations (Avg. session turns from 4.2 to 12.5) and higher task success without sacrificing immersion.