[Paper Review] Reconstructing Experiences through Sketching

Evangelos Karapanos, Jean-Bernard Martens+1 more|arXiv (Cornell University)|Dec 29, 2009

Identity, Memory, and Therapy40 references17 citations

TL;DR

This paper introduces iScale, a sketch-based survey tool that enhances retrospective elicitation of longitudinal user experience data by reducing retrospection bias through structured sketching. It outperforms free recall in recall quantity, richness, and test-retest reliability, especially when using constructive sketching with chronological ordering.

ABSTRACT

We present iScale, a survey tool for the retrospective elicitation of longitudinal user experience data. iScale employs sketching in imposing a process in the reconstruction of one's experiences with the aim to minimize retrospection bias. Two versions, the Constructive and the Value-Account iScale, were motivated by two distinct theories on how people reconstruct emotional experiences from memory. These two versions were tested in two separate studies. Study 1 aimed at providing qualitative insight into the use of iScale and compared its performance to that of free-hand sketching. Study 2 compared the two versions of iScale to free recall, a control condition that does not influence the reconstruction process. Significant differences between iScale and free recall were found. Overall, iScale resulted in an increase in the amount, the richness, and the test-retest reliability of recalled information. These results provide support for the viability of retrospective techniques as a cost-effective alternative to longitudinal studies.

Motivation & Objective

• Address the challenge of capturing longitudinal user experience data cost-effectively without prolonged field studies.
• Reduce retrospection bias in user-reported experiences by leveraging sketching as a cognitive aid.
• Investigate whether structured sketching improves the quantity, quality, and reliability of recalled user experiences.
• Compare two sketching approaches—constructive and value-account—against free recall in eliciting longitudinal experience data.
• Evaluate the impact of chronological ordering and memory structure on the effectiveness of experience reconstruction.

Proposed method

• Developed iScale, a survey tool that uses sketching to structure the reconstruction of past user experiences.
• Implemented two versions: Constructive iScale (chronological, context-focused) and Value-Account iScale (top-down, emotion-focused), based on distinct memory theories.
• Applied the Day Reconstruction Method (DRM) as a foundation for retrospective experience reporting.
• Used sketching to impose temporal and semantic structure on memory reconstruction, enhancing contextual cue recall.
• Conducted two studies: one qualitative comparison with free-hand sketching, and one quantitative comparison with free recall as a control.
• Collected and analyzed data on number of experience reports, contextual detail, and test-retest reliability across conditions.

Experimental results

Research questions

• RQ1 How does sketching in iScale affect the number and richness of recalled user experience reports compared to free recall?
• RQ2 Does the use of chronological ordering in sketching improve the reliability and detail of recalled contextual cues?
• RQ3 How do the two iScale variants—Constructive and Value-Account—differ in their effectiveness for eliciting longitudinal experience data?
• RQ4 To what extent does sketching reduce retrospection bias in retrospective user experience reporting?
• RQ5 Can sketching-based tools like iScale serve as a reliable, cost-effective alternative to longitudinal field studies?

Key findings

• Both versions of iScale produced significantly more experience reports than free recall, with the Constructive iScale yielding more than the Value-Account version.
• Constructive iScale led to significantly more contextual information (e.g., temporal and situational cues) being recalled, especially when chronological ordering was used.
• Sketching improved test-retest reliability, particularly for temporal information and overall evaluation patterns, with Constructive iScale showing higher consistency.
• The Constructive iScale tool outperformed the Value-Account version in all measured dimensions: recall quantity, contextual richness, and reliability.
• Sketching did not significantly alter the design relevance of reported information but enhanced the depth and verisimilitude of contextual details.
• The results support iScale as a viable, cost-effective alternative to longitudinal field studies for collecting reliable retrospective user experience data.