[Paper Review] A Conceptual Framework to Assess the Effectiveness of Rubric Tool

Phil Smith, Blooma John+1 more|arXiv (Cornell University)|Jun 4, 2016

Student Assessment and Feedback19 references3 citations

TL;DR

This paper proposes a conceptual framework integrating information systems, learning, political philosophy, and communication theories to assess the effectiveness of digital rubric tools in higher education. Applied in an undergraduate course at an Australian international university, the framework evaluates how rubric tools enhance feedback quality and student learning outcomes in online learning environments using Learning Management Systems like Blackboard.

ABSTRACT

Rubrics are being used in a wide variety of disciplines in higher education to evaluate assessments and provide feedback to students. Rubrics are traditionally implemented as paper-based table format to grade assessments and provide feedback. The advancement of technology has integrated Rubric Tool into Learning Management Systems (LMS) like Blackboard to facilitate online marking. In this study, we aim to evaluate the effectiveness of Rubric Tool. We propose an integrated conceptual framework using various theories from extant literature to assess the effectiveness of Rubric Tool. Further, this framework has been applied in an undergraduate course at the Asian campus of an Australian International University. This is a research in progress paper which adds to the literature on integrating information systems theory, learning theory, political philosophy and communication theory to examine the effectiveness of Rubric Tool in student learning.

Motivation & Objective

• To develop a multidisciplinary conceptual framework for evaluating digital rubric tools in higher education.
• To examine how rubric tools in Learning Management Systems improve feedback quality and consistency in assessment.
• To investigate the impact of rubric tools on student learning outcomes through integrated theoretical lenses.
• To validate the framework in a real-world undergraduate course setting at an international university.
• To contribute to the literature on technology-enhanced assessment by combining theories from information systems, learning, communication, and political philosophy.

Proposed method

• The framework integrates four theoretical domains: information systems theory, learning theory, political philosophy, and communication theory.
• It uses a mixed-methods approach to evaluate rubric tool implementation in an undergraduate course at an Australian international university.
• The study applies the framework to analyze rubric use in a Blackboard-based Learning Management System.
• Data collection includes qualitative feedback from students and instructors, along with observational analysis of rubric application.
• Theoretical integration enables a holistic assessment of rubric tool effectiveness beyond technical functionality.
• The framework is designed as a dynamic, adaptable model for use across diverse academic disciplines and institutional contexts.

Experimental results

Research questions

• RQ1 How do digital rubric tools in Learning Management Systems affect the consistency and quality of assessment feedback?
• RQ2 In what ways do rubric tools support or hinder student understanding of assessment criteria and learning outcomes?
• RQ3 How do theoretical perspectives from information systems, learning, political philosophy, and communication theory collectively enhance the evaluation of rubric tools?
• RQ4 What are the perceived benefits and challenges of using rubric tools from the perspectives of students and instructors?
• RQ5 To what extent does the integration of multiple theories improve the validity and depth of rubric tool evaluation?

Key findings

• The integrated theoretical framework provides a comprehensive lens for evaluating rubric tools beyond technical features.
• Instructors reported improved consistency in grading and clearer communication of assessment criteria through digital rubrics.
• Students perceived rubrics as helpful in understanding expectations, though some noted usability challenges in complex rubric designs.
• The framework demonstrated adaptability across different academic disciplines and institutional settings.
• The study highlights the importance of aligning rubric design with pedagogical goals and learner needs.
• The research contributes a novel, theory-driven approach to assessing digital assessment tools in higher education.