[Paper Review] Machine Teaching: A New Paradigm for Building Machine Learning Systems

Patrice Y. Simard, Saleema Amershi+9 more|arXiv (Cornell University)|Jul 21, 2017

Machine Learning and Data ClassificationComputer Science2 references76 citations

TL;DR

The paper argues for machine teaching as a discipline focused on teacher productivity, proposing a decoupled, interface-driven paradigm that broadens who can build ML systems. It draws parallels to programming and software engineering to accelerate ML deployment and collaboration.

ABSTRACT

The current processes for building machine learning systems require practitioners with deep knowledge of machine learning. This significantly limits the number of machine learning systems that can be created and has led to a mismatch between the demand for machine learning systems and the ability for organizations to build them. We believe that in order to meet this growing demand for machine learning systems we must significantly increase the number of individuals that can teach machines. We postulate that we can achieve this goal by making the process of teaching machines easy, fast and above all, universally accessible. While machine learning focuses on creating new algorithms and improving the accuracy of "learners", the machine teaching discipline focuses on the efficacy of the "teachers". Machine teaching as a discipline is a paradigm shift that follows and extends principles of software engineering and programming languages. We put a strong emphasis on the teacher and the teacher's interaction with data, as well as crucial components such as techniques and design principles of interaction and visualization. In this paper, we present our position regarding the discipline of machine teaching and articulate fundamental machine teaching principles. We also describe how, by decoupling knowledge about machine learning algorithms from the process of teaching, we can accelerate innovation and empower millions of new uses for machine learning models.

Motivation & Objective

• Highlight the limitations of the current ML development process due to overreliance on ML experts.
• Propose machine teaching as a discipline focused on teacher productivity and interaction with data.
• Advocate decoupling teaching from runtime ML algorithms through standardized interfaces.
• Draw parallels between machine teaching and software engineering to leverage tooling and collaboration.
• Outline principles, definitions, and a roadmap for the machine teaching discipline.

Proposed method

• Define machine teaching and machine teaching research and distinguish them from traditional ML research.
• Argue for decoupling teaching from ML algorithms to enable runtime-agnostic teaching via interfaces.
• Map teaching concepts to programming analogies (concepts, features, schemas, decomposition).
• Propose system requirements for the teaching language to be expressive, scalable, and consistent with learning theory.
• Discuss lessons from programming (decomposition, version control, APIs) as guiding principles for MT tooling and processes.

Experimental results

Research questions

• RQ1 What is machine teaching and how does it differ from traditional machine learning research?
• RQ2 How can teaching be decoupled from runtime ML algorithms to improve productivity and collaboration?
• RQ3 What interfaces and abstractions are needed to support scalable, reusable teaching across runtimes?
• RQ4 What organizational and technical parallels from programming can inform the development of MT tools and processes?

Key findings

• Machine teaching reframes ML model building as a teacher-centric activity with metrics tied to teacher cost, interpretability, and scalability.
• Teaching can be decoupled from ML algorithms using interfaces that express inputs, outputs, and schemas, enabling runtime-agnostic teaching.
• Sub-concepts, features, and schemas facilitate decomposing concepts for easier manipulation and documentation by teachers.
• Analogies with programming show MT can benefit from version control, APIs, high-level teaching languages, and modular toolchains.
• The MT paradigm aims to democratize ML by expanding the set of people who can build ML systems without requiring deep ML expertise.