[Paper Review] The Mastery Rubric for Statistics and Data Science: promoting coherence and consistency in data science education and training

Rochelle E. Tractenberg, Donna LaLonde+1 more|arXiv (Cornell University)|Aug 15, 2023

Statistics Education and MethodologiesMathematics3 citations

TL;DR

The paper introduces the Mastery Rubric for Statistics and Data Science (MR-SDS), a framework that standardizes curriculum development and assessment by mapping knowledge, skills, and abilities (KSAs) across developmental stages. It enables consistent, evidence-based evaluation of student independence and proficiency across statistics, computing, and domain-specific applications, supporting coherence in data science education from undergraduate to doctoral levels.

ABSTRACT

Consensus based publications of both competencies and undergraduate curriculum guidance documents targeting data science instruction for higher education have recently been published. Recommendations for curriculum features from diverse sources may not result in consistent training across programs. A Mastery Rubric was developed that prioritizes the promotion and documentation of formal growth as well as the development of independence needed for the 13 requisite knowledge, skills, and abilities for professional practice in statistics and data science, SDS. The Mastery Rubric, MR, driven curriculum can emphasize computation, statistics, or a third discipline in which the other would be deployed or, all three can be featured. The MR SDS supports each of these program structures while promoting consistency with international, consensus based, curricular recommendations for statistics and data science, and allows 'statistics', 'data science', and 'statistics and data science' curricula to consistently educate students with a focus on increasing learners independence. The Mastery Rubric construct integrates findings from the learning sciences, cognitive and educational psychology, to support teachers and students through the learning enterprise. The MR SDS will support higher education as well as the interests of business, government, and academic work force development, bringing a consistent framework to address challenges that exist for a domain that is claimed to be both an independent discipline and part of other disciplines, including computer science, engineering, and statistics. The MR-SDS can be used for development or revision of an evaluable curriculum that will reliably support the preparation of early e.g., undergraduate degree programs, middle e.g., upskilling and training programs, and late e.g., doctoral level training practitioners.

Motivation & Objective

• To address inconsistencies in data science education across higher education programs.
• To provide a standardized framework for curriculum design that emphasizes developmental progression and independence.
• To align data science programs with international consensus-based recommendations in statistics and data science education.
• To support the integration of scientific reasoning, ethics, and reproducibility into curricula through explicit, observable KSAs.
• To enable reliable internal and external evaluation of programs based on concrete, performance-based evidence of student proficiency.

Proposed method

• The Mastery Rubric (MR) is structured with KSAs as rows and developmental stages (e.g., emerging, progressing, independent) as columns.
• Each KSA is described in terms of observable, performance-based criteria at each developmental level.
• The rubric integrates findings from learning sciences, cognitive psychology, and educational theory to support meaningful developmental progression.
• It supports curricula emphasizing statistics, computation, or interdisciplinary integration, ensuring coherence across program types.
• The rubric enables both formative and summative assessment by linking student work products to specific stages of mastery.
• It allows for alignment with existing curriculum guidelines (e.g., NAS 2018, ASA 2016) and facilitates curriculum evaluation and revision.

Experimental results

Research questions

• RQ1 How can a standardized rubric improve consistency and coherence in data science education across diverse higher education programs?
• RQ2 What developmental stages of knowledge, skills, and abilities (KSAs) are necessary for students to achieve independence in statistics and data science?
• RQ3 How can the Mastery Rubric support the integration of scientific reasoning, ethics, and reproducibility into data science curricula?
• RQ4 In what ways can the MR-SDS enable reliable, evidence-based assessment of student proficiency across different program types (e.g., statistics-focused, computation-focused, interdisciplinary)?
• RQ5 How does the MR-SDS facilitate curriculum evaluation and alignment with international consensus-based recommendations?

Key findings

• The Mastery Rubric for Statistics and Data Science (MR-SDS) provides a structured, developmental framework that makes curriculum goals and student progress explicit across knowledge, skills, and abilities.
• The rubric enables consistent assessment of student independence by describing observable performance at distinct developmental stages, moving beyond subjective or perception-based evaluations.
• Programs using the MR-SDS can reliably demonstrate that students achieve comparable levels of proficiency in core KSAs, regardless of curriculum emphasis (e.g., statistics, computation, or interdisciplinary application).
• The MR-SDS supports the integration of scientific reasoning, ethics, and reproducibility as observable, assessable components of training, aligning with consensus recommendations.
• Curricula built on the MR-SDS allow for both internal and external evaluation based on concrete work products, reducing reliance on variable or sample-dependent metrics.
• The framework supports curriculum development and revision at all levels—undergraduate, upskilling, and doctoral—by providing a common language for assessing mastery and independence.