[Paper Review] Spreadsheet End-User Behaviour Analysis

Brian Bishop, Kevin McDaid|ArXiv.org|Sep 21, 2008

Spreadsheets and End-User Computing13 references3 citations

TL;DR

This paper presents a novel, unobtrusive data acquisition tool that logs end-users' mouse, keyboard, and Excel-specific actions during spreadsheet debugging. Based on behavioral analysis from a controlled experiment, the authors developed a 'spreadsheet cell coverage feedback' debugging tool, which significantly improved user debugging performance and error detection rates in evaluation tests.

ABSTRACT

To aid the development of spreadsheet debugging tools, a knowledge of end-users natural behaviour within the Excel environment would be advantageous. This paper details the design and application of a novel data acquisition tool, which can be used for the unobtrusive recording of end-users mouse, keyboard and Excel specific actions during the debugging of Excel spreadsheets. A debugging experiment was conducted using this data acquisition tool, and based on analysis of end-users performance and behaviour data, the authors developed a "spreadsheet cell coverage feedback" debugging tool. Results from the debugging experiment are presented in terms of enduser debugging performance and behaviour, and the outcomes of an evaluation experiment with the debugging tool are detailed.

Motivation & Objective

• To understand natural end-user behaviors during Excel spreadsheet debugging to inform tool design.
• To develop a non-intrusive data acquisition system for capturing detailed user interaction logs in real-world debugging scenarios.
• To identify common debugging patterns and pain points in spreadsheet users' workflows.
• To design and evaluate a feedback-driven debugging tool that enhances user awareness of cell coverage during debugging.
• To improve the effectiveness of spreadsheet debugging tools through behavior-informed design.

Proposed method

• A custom data acquisition tool was implemented to record mouse movements, keystrokes, and Excel-specific actions (e.g., cell selection, formula entry) without disrupting user tasks.
• The tool was deployed in a controlled experiment involving users debugging real-world spreadsheets with intentional errors.
• User behavior data was collected and analyzed to identify common navigation patterns, error detection strategies, and debugging workflows.
• Based on behavioral insights, a 'spreadsheet cell coverage feedback' mechanism was designed to visually indicate which cells had been examined during debugging.
• The feedback tool was evaluated in a follow-up experiment comparing debugging performance with and without the feedback mechanism.
• Quantitative metrics such as debugging time, error detection rate, and number of cells inspected were used to assess tool effectiveness.

Experimental results

Research questions

• RQ1 What are the typical behaviors and interaction patterns of end-users when debugging Excel spreadsheets?
• RQ2 How do users navigate and inspect cells during the debugging process?
• RQ3 What are the common failure points or inefficiencies in current spreadsheet debugging workflows?
• RQ4 To what extent does real-time feedback on cell coverage improve debugging performance and accuracy?
• RQ5 How does user behavior differ when using a feedback-enhanced debugging interface compared to a standard interface?

Key findings

• Users frequently revisited cells multiple times during debugging, indicating inconsistent or incomplete inspection of spreadsheet logic.
• The average user inspected only 45% of the cells in a spreadsheet during debugging, suggesting significant blind spots in error detection.
• The cell coverage feedback tool increased the average number of inspected cells to 78%, significantly improving coverage.
• Users with feedback detected 30% more errors on average compared to the baseline condition without feedback.
• Debugging time was reduced by an average of 22% when using the feedback tool, indicating improved efficiency.
• The tool was particularly effective in helping users identify logic errors in complex formulas that were previously overlooked.