[论文解读] Motion-to-Response Content Generation via Multi-Agent AI System with Real-Time Safety Verification

HyeYoung Lee|arXiv (Cornell University)|Jan 20, 2026

Emotion and Mood Recognition被引用 0

一句话总结

本文提出一个四智能体系统，将基于音频的情感转化为安全、实时、适龄的响应内容，并包含安全性验证循环与在设备端部署。

ABSTRACT

This paper proposes a multi-agent artificial intelligence system that generates response-oriented media content in real time based on audio-derived emotional signals. Unlike conventional speech emotion recognition studies that focus primarily on classification accuracy, our approach emphasizes the transformation of inferred emotional states into safe, age-appropriate, and controllable response content through a structured pipeline of specialized AI agents. The proposed system comprises four cooperative agents: (1) an Emotion Recognition Agent with CNN-based acoustic feature extraction, (2) a Response Policy Decision Agent for mapping emotions to response modes, (3) a Content Parameter Generation Agent for producing media control parameters, and (4) a Safety Verification Agent enforcing age-appropriateness and stimulation constraints. We introduce an explicit safety verification loop that filters generated content before output, ensuring compliance with predefined rules. Experimental results on public datasets demonstrate that the system achieves 73.2% emotion recognition accuracy, 89.4% response mode consistency, and 100% safety compliance while maintaining sub-100ms inference latency suitable for on-device deployment. The modular architecture enables interpretability and extensibility, making it applicable to child-adjacent media, therapeutic applications, and emotionally responsive smart devices.

研究动机与目标

• bridging emotion recognition and content generation with explicit policy and safety layers.
• Enable interpretable, modular, and on-device emotion-to-response content generation.
• Ensure age-appropriateness and controlled stimulation via rule-based safety verification.
• Demonstrate real-time performance and privacy-preserving, edge-focused deployment.

提出的方法

• Four cooperative agents process input: Emotion Recognition, Response Policy Decision, Content Parameter Generation, and Safety Verification.
• CNN-based acoustic feature extraction and softmax-based emotion classification for e* in C emotion categories.
• Policy mapping from emotion and arousal to discrete response modes via a decision tree.
• Content parameter generation predicting multi-modal media controls (audio, visual, text) from the chosen mode.
• Explicit safety verification using rule-based constraints with a regeneration loop if rules are violated.

Figure 1: Overall architecture of the proposed multi-agent system for emotion-to-response content generation. The system comprises four specialized agents operating sequentially: Emotion Recognition Agent, Response Policy Decision Agent, Content Parameter Generation Agent, and Safety Verification Ag

实验结果

研究问题

• RQ1 How accurately can emotions be recognized from audio in a lightweight on-device setting?
• RQ2 Can recognized emotions be reliably mapped to safe, age-appropriate response modes?
• RQ3 Do generated content parameters satisfy safety constraints with real-time verification?
• RQ4 What is the end-to-end latency of the system across different hardware, and is it suitable for on-device deployment?
• RQ5 What is the impact of the safety verification loop on output quality and reliability?

主要发现

• Emotion recognition accuracy ranges across datasets, with 73.2% on IEMOCAP for 4-class, 78.5% on RAVDESS for 4-class, and 89.3% on synthetic data.
• Response mode accuracy achieved 89.4%, with high precision and recall around 86–87% across macro metrics.
• Safety verification achieved 100% pass rate across rules, with only 1.6% regenerations needed.
• End-to-end inference latency stayed under 100 ms on tested hardware, including edge devices.
• Ablation shows multi-agent structure improves consistency and safety compared to end-to-end setups.

Figure 2: Processing flowchart of the emotion-to-response content generation method. The system processes audio input through sequential stages (S10-S60) with a safety verification loop that triggers content regeneration upon failure.