[Paper Review] The Lumiere Project: Bayesian User Modeling for Inferring the Goals and Needs of Software Users

Eric Horvitz, Jack Breese+3 more|arXiv (Cornell University)|Jan 30, 2013

Data Management and AlgorithmsComputer Science17 references659 citations

TL;DR

The Lumiere Project introduces a Bayesian user modeling framework that infers software users' goals and needs by analyzing their actions, queries, and contextual background. By employing probabilistic reasoning over event streams and persistent user profiles, the system enables intelligent assistance, forming the foundation for Microsoft Office '97's Office Assistant feature.

ABSTRACT

The Lumiere Project centers on harnessing probability and utility to provide assistance to computer software users. We review work on Bayesian user models that can be employed to infer a users needs by considering a user's background, actions, and queries. Several problems were tackled in Lumiere research, including (1) the construction of Bayesian models for reasoning about the time-varying goals of computer users from their observed actions and queries, (2) gaining access to a stream of events from software applications, (3) developing a language for transforming system events into observational variables represented in Bayesian user models, (4) developing persistent profiles to capture changes in a user expertise, and (5) the development of an overall architecture for an intelligent user interface. Lumiere prototypes served as the basis for the Office Assistant in the Microsoft Office '97 suite of productivity applications.

Motivation & Objective

• To develop a probabilistic framework that infers users' evolving goals and needs from observed behavior in software applications.
• To address the challenge of modeling time-varying user goals using Bayesian inference on event streams from software interactions.
• To create a persistent user profile that captures changes in user expertise and preferences over time.
• To design an extensible architecture for integrating user modeling into intelligent user interfaces.
• To enable proactive, context-aware assistance in productivity software through real-time inference of user intent.

Proposed method

• Constructs Bayesian networks to represent the joint probability distribution over user goals, actions, and contextual variables.
• Transforms low-level system events (e.g., menu selections, command inputs) into observational variables using a domain-specific language for event interpretation.
• Applies dynamic Bayesian networks to model goal transitions and update beliefs about user intentions over time.
• Integrates utility theory to prioritize assistance actions based on predicted user needs and expected benefit.
• Maintains persistent user profiles that adapt to changes in task performance, frequency of actions, and query patterns.
• Employs a modular architecture that decouples event collection, user model inference, and action selection for extensibility.

Experimental results

Research questions

• RQ1 How can a Bayesian user model infer a user’s current and evolving goals from observed actions and queries in real time?
• RQ2 What is an effective way to map low-level software events into meaningful observations for user modeling?
• RQ3 How can user expertise and task proficiency be modeled and updated dynamically over time?
• RQ4 What architectural principles support the integration of probabilistic user modeling into real-world user interfaces?
• RQ5 How can utility-based reasoning guide the selection of timely and relevant assistance actions?

Key findings

• The Bayesian user model successfully inferred user goals with high accuracy by combining observed actions, queries, and contextual cues.
• The system demonstrated that persistent user profiles could adapt to changes in user expertise, improving long-term assistance relevance.
• The integration of utility theory enabled the system to prioritize assistance actions based on expected value, reducing user interruption.
• The Lumiere prototypes formed the technical foundation for the Office Assistant in Microsoft Office '97, validating the approach in a production environment.
• The event-to-observation transformation language enabled scalable and maintainable modeling of diverse software interaction patterns.