[Paper Review] A brief introduction to the Grey Machine Learning

Xin Ma|arXiv (Cornell University)|May 4, 2018

Grey System Theory ApplicationsDecision Sciences43 citations

TL;DR

The paper introduces Grey Machine Learning (GML), a kernel-based extension of grey system theory that estimates unknown nonlinear functions in small-sample dynamical systems by kernel methods, linking grey models with LSSVM.

ABSTRACT

This paper presents a brief introduction to the key points of the Grey Machine Learning (GML) based on the kernels. The general formulation of the grey system models have been firstly summarized, and then the nonlinear extension of the grey models have been developed also with general formulations. The kernel implicit mapping is used to estimate the nonlinear function of the GML model, by extending the nonparametric formulation of the LSSVM, the estimation of the nonlinear function of the GML model can also be expressed by the kernels. A short discussion on the priority of this new framework to the existing grey models and LSSVM have also been discussed in this paper. And the perspectives and future orientations of this framework have also been presented.

Motivation & Objective

• Motivate the integration of grey system theory with machine learning to handle unknown nonlinear relationships in small-sample dynamical systems.
• Generalize grey models by replacing the unknown nonlinear function with a kernel-based estimator.
• Show that kernel-based, regularized estimation yields a flexible, semi-parametric framework for grey models.
• Provide practical implementations and open issues to guide future GML development.

Proposed method

• Present the general continuous and discrete formulations of GML with an unknown function phi(t).
• Establish a linear representation in a high-dimensional feature space to express phi(t) as omega^T phi(x(t)).
• Use kernel methods to compute inner products via K(x(k), x(t)) and solve a regularized least squares problem (ridge) to obtain lambda and b.
• Derive a linear system (kernelized representer solution) to obtain the nonlinear estimate hat{y}(t).
• Extend to semiparametric estimation by combining linear and kernel components.
• Discuss kernel selection, hyperparameter tuning, and training algorithms in the GML context.

Experimental results

Research questions

• RQ1 How can grey models incorporate unknown nonlinear relationships using kernel-based estimators?
• RQ2 What is the computational and theoretical foundation that enables kernel methods to estimate the nonlinear part in GML?
• RQ3 Do kernel-based, regularized formulations improve performance over classical grey models and static ML models like LSSVM in small-sample settings?
• RQ4 What are the practical considerations and open issues when applying GML to real-world dynamical systems?
• RQ5 How do semiparametric extensions compare to nonparametric kernel estimations within the GML framework?

Key findings

• The nonlinear function in GML can be represented as a kernel sum hat{phi}(t)=sum lambda_k K(x(k), x(t)).
• GML reduces to a linear system involving the kernel matrix Omega and solves for lambda and b.
• The framework situates GML as a combination of known linear grey structure with a kernel-based nonlinear component.
• GML aligns with the Representer Theorem, supporting kernel-based estimation in regularized learning.
• Empirical results in prior works suggest GML can outperform classical grey models and static LSSVM for nonlinear dynamics in small samples.