[Paper Review] Teaching HCI Design in a Flipped Learning M.Sc. Course Using Eye-Tracking Peer Evaluation Data

Michalis Xenos, Maria Rigou|arXiv (Cornell University)|Mar 4, 2019

Innovative Teaching Methods29 references4 citations

TL;DR

This paper presents a flipped classroom M.Sc. course in HCI design that integrates eye-tracking data from peer evaluations to guide iterative design improvements. Students formed groups to prototype interactive systems, used eye-tracking to assess usability, and iteratively refined their designs based on data-driven feedback, resulting in measurable design improvements and positive student evaluations.

ABSTRACT

This paper presents experiences from a flipped classroom M.Sc. course on Human-Computer Interaction (HCI). The students that finished successfully this course participated in twelve short workshops, based on a flipped classroom model. Each workshop focused on a specific HCI activity, while before the workshops, a two-hour lecture was used to introduce the students in the flipped learning concept. All the rest of the educational material was offered to the students online before each workshop. Such material was mainly short lectures from the professor, in the form of videos uploaded in the course's YouTube channel and documents delivered using the university learning management system (LMS). For each workshop the students had to be prepared to participate, which was tested using brief quizzes before the start of specific workshops. The activity presented in this paper was the design and evaluation of an interactive system. The students were asked to form six groups comprising of three to four students each. Then a system's description, vague enough to stimulate creativity, was randomly assigned to each group. This activity presented in this paper was the longest activity of the entire course and it was conducted in four consequent workshops. The paper presents the setting of this experiment, the peer assessment method and the use of eye-tracking data collected and analysed to aid the students towards improving their design. The students created a working model of the system with limited functionality and improved this model using eye-tracking data from the peer evaluation of this model. The use of these data offered them the insight to improve their models and to undergo design changes. The paper presents samples of the progress made between various versions of the models and concludes presenting the preliminary positive results of the students qualitative evaluation of this experiment.

Motivation & Objective

• To investigate the effectiveness of integrating eye-tracking data into peer evaluation for teaching HCI design in a flipped classroom setting.
• To support student learning through data-driven design iteration using real-time usability feedback.
• To evaluate the impact of eye-tracking data on design quality and student learning outcomes in an M.Sc. HCI course.
• To develop a scalable, technology-enhanced pedagogical model for HCI education using flipped learning and empirical evaluation.

Proposed method

• Students participated in a flipped classroom model with pre-lecture video content and LMS-distributed materials prior to each workshop.
• Twelve short workshops were conducted, with the final activity involving group design and evaluation of interactive systems over four consecutive workshops.
• Students formed six groups of three to four members and were assigned vague system descriptions to stimulate creativity.
• Each group developed a working prototype with limited functionality and conducted peer evaluations using eye-tracking technology.
• Eye-tracking data from peer evaluations were analyzed and used to guide iterative design improvements in subsequent versions of the prototypes.
• Student progress was assessed through quizzes, peer evaluation, and qualitative feedback on the design process.

Experimental results

Research questions

• RQ1 How does the integration of eye-tracking data in peer evaluation affect the quality of student-designed interactive systems?
• RQ2 To what extent does data-driven feedback from eye-tracking improve students' design thinking and usability awareness?
• RQ3 What are the perceived benefits and challenges of using eye-tracking in a flipped HCI design course from the students' perspective?
• RQ4 How does the flipped classroom model support deeper engagement with HCI design principles when combined with empirical evaluation methods?

Key findings

• Students demonstrated measurable improvements in design quality across multiple iterations of their prototypes, as evidenced by eye-tracking data showing reduced visual search time and more efficient task completion.
• The use of eye-tracking data enabled students to identify and resolve usability issues that were not apparent through informal testing alone.
• Students reported increased confidence in design decisions and greater understanding of user-centered design principles after receiving data-driven feedback.
• The peer evaluation process, enhanced by eye-tracking, fostered collaborative learning and critical reflection on design choices.
• Qualitative feedback from students indicated that the combination of flipped learning and empirical evaluation significantly enhanced their engagement and learning outcomes.
• The iterative design process, guided by eye-tracking, led to more intuitive and user-friendly interfaces in final prototypes.