[Paper Review] From Eliza to XiaoIce: Challenges and Opportunities with Social Chatbots

Heung‐Yeung Shum, Xiaodong He+1 more|arXiv (Cornell University)|Jan 6, 2018

AI in Service Interactions66 citations

TL;DR

The paper surveys the evolution of social chatbots, argues for engagement-driven design using IQ and EQ, and highlights XiaoIce as a case study for long, empathetic conversations.

ABSTRACT

Conversational systems have come a long way since their inception in the 1960s. After decades of research and development, we've seen progress from Eliza and Parry in the 60's and 70's, to task-completion systems as in the DARPA Communicator program in the 2000s, to intelligent personal assistants such as Siri in the 2010s, to today's social chatbots like XiaoIce. Social chatbots' appeal lies not only in their ability to respond to users' diverse requests, but also in being able to establish an emotional connection with users. The latter is done by satisfying users' need for communication, affection, as well as social belonging. To further the advancement and adoption of social chatbots, their design must focus on user engagement and take both intellectual quotient (IQ) and emotional quotient (EQ) into account. Users should want to engage with a social chatbot; as such, we define the success metric for social chatbots as conversation-turns per session (CPS). Using XiaoIce as an illustrative example, we discuss key technologies in building social chatbots from core chat to visual awareness to skills. We also show how XiaoIce can dynamically recognize emotion and engage the user throughout long conversations with appropriate interpersonal responses. As we become the first generation of humans ever living with AI, we have a responsibility to design social chatbots to be both useful and empathetic, so they will become ubiquitous and help society as a whole.

Motivation & Objective

• Trace the historical progression of conversational systems from Eliza/Parry to modern social chatbots like XiaoIce.
• Argue that success for social chatbots should balance intellectual capabilities (IQ) with emotional intelligence (EQ).
• Propose conversation-turns per session (CPS) as a metric for user engagement.
• Illustrate XiaoIce’s capabilities across core chat, visual awareness, and skills to enable sustained, meaningful interactions.
• Emphasize ethical and societal responsibilities in designing empathetic, useful social chatbots.

Proposed method

• Review and synthesize the evolution of conversational systems.
• Define and apply the IQ/EQ framework to social chatbot design.
• Use XiaoIce as a concrete example to discuss architecture and engagement strategies.
• Describe how emotion recognition and interpersonal responses support long, coherent conversations.
• Outline core components and technologies involved in modern social chatbots (core chat, visual awareness, skills).

Experimental results

Research questions

• RQ1 What are the key challenges and opportunities in building social chatbots that engage users emotionally and intellectually?
• RQ2 How should success be measured for social chatbots beyond task completion?
• RQ3 How can social chatbots sustain long-term, meaningful conversations through emotion recognition and interpersonal responses?
• RQ4 What architectural components enable a chatbot like XiaoIce to maintain user engagement over extended sessions?

Key findings

• Engagement is the central objective for social chatbots, not just task completion.
• A combined IQ and EQ approach better supports long, empathetic interactions.
• CPS (conversation-turns per session) is proposed as a practical success metric for social chatbots.
• XiaoIce exemplifies integration of core chat, visual awareness, and skills to sustain ongoing dialogue.
• Emotion recognition and appropriate interpersonal responses are essential for empathetic user experiences.
• Designing social chatbots carries responsibilities to be useful and empathetic, with societal impact considerations.