[Paper Review] Scikit-Multiflow: A Multi-output Streaming Framework

Jacob Montiel, Jesse Read+2 more|arXiv (Cornell University)|Jul 12, 2018

Data Stream Mining TechniquesComputer Science7 references277 citations

TL;DR

scikit-multiflow is a Python framework for multi-output/multi-label and streaming data mining, integrating state-of-the-art methods, stream generators, and evaluators to enable stream learning research and experimentation.

ABSTRACT

Scikit-multiflow is a multi-output/multi-label and stream data mining framework for the Python programming language. Conceived to serve as a platform to encourage democratization of stream learning research, it provides multiple state of the art methods for stream learning, stream generators and evaluators. scikit-multiflow builds upon popular open source frameworks including scikit-learn, MOA and MEKA. Development follows the FOSS principles and quality is enforced by complying with PEP8 guidelines and using continuous integration and automatic testing. The source code is publicly available at https://github.com/scikit-multiflow/scikit-multiflow.

Motivation & Objective

• Fill a void for a Python-based stream learning framework that interacts with scikit-learn and extends state-of-the-art methods for streams.
• Provide tools to facilitate the development and evaluation of stream learning research.
• Foster democratization of stream learning by making tools accessible within the Python data science ecosystem.

Proposed method

• Base StreamModel class with fit, partial_fit, predict, and predict_proba methods.
• Stream object provides continuous data flow; StreamEvaluator handles data querying, training/testing, and performance tracking.
• Prequential (interleaved test-then-train) evaluation as a core evaluation method for streams.
• Support for stream generators, learners, change detectors, and evaluation methods (prequential and hold-out).
• Integration with existing frameworks such as scikit-learn, MOA, and MEKA inspired by their interfaces and methodologies.
• BSD-licensed, open-source, with documentation, continuous integration, and public repository.

Experimental results

Research questions

• RQ1 How can a Python framework support multi-output/multi-label streaming learning with incremental training?
• RQ2 Can scikit-multiflow provide a common ground bridging scikit-learn, MOA, and MEKA for stream learning research?
• RQ3 What tools (generators, learners, detectors, evaluators) are needed to facilitate development and evaluation of stream learning methods?
• RQ4 How does prequential evaluation operate within this framework to assess model performance over time?

Key findings

• The framework offers stream generators, learners, change detectors, and evaluators suitable for multi-output learning.
• It supports both binary/multi-class and multi-label/multi-output scenarios with incremental training.
• It enables prequential evaluation and hold-out evaluation for performance tracking in streaming contexts.
• The project emphasizes democratization of stream learning by making tools accessible in Python and interoperable with scikit-learn.
• Code is open source under the BSD License with continuous integration and public hosting.