[论文解读] The Tsetlin Machine -- A Game Theoretic Bandit Driven Approach to Optimal Pattern Recognition with Propositional Logic

Ole‐Christoffer Granmo|arXiv (Cornell University)|Apr 4, 2018

Optimization and Search Problems参考文献 48被引用 99

一句话总结

引入 Tsetlin Machine，一种由 Tsetlin Automata 组成、可扩展且可解释的模式识别器，组织成子句并通过博弈理论学习方案协同以在嘈杂环境中实现全局最优。它在与标准机器学习方法的对比中展现出有竞争力的准确性，并提供可解释的命题逻辑模式。

ABSTRACT

Although simple individually, artificial neurons provide state-of-the-art performance when interconnected in deep networks. Arguably, the Tsetlin Automaton is an even simpler and more versatile learning mechanism, capable of solving the multi-armed bandit problem. Merely by means of a single integer as memory, it learns the optimal action in stochastic environments through increment and decrement operations. In this paper, we introduce the Tsetlin Machine, which solves complex pattern recognition problems with propositional formulas, composed by a collective of Tsetlin Automata. To eliminate the longstanding problem of vanishing signal-to-noise ratio, the Tsetlin Machine orchestrates the automata using a novel game. Further, both inputs, patterns, and outputs are expressed as bits, while recognition and learning rely on bit manipulation, simplifying computation. Our theoretical analysis establishes that the Nash equilibria of the game align with the propositional formulas that provide optimal pattern recognition accuracy. This translates to learning without local optima, only global ones. In five benchmarks, the Tsetlin Machine provides competitive accuracy compared with SVMs, Decision Trees, Random Forests, Naive Bayes Classifier, Logistic Regression, and Neural Networks. We further demonstrate how the propositional formulas facilitate interpretation. In conclusion, we believe the combination of high accuracy, interpretability, and computational simplicity makes the Tsetlin Machine a promising tool for a wide range of domains.

研究动机与目标

• Address the limitations of traditional Learning Automata in pattern recognition, including poor pattern representation and vanishing signal-to-noise ratio.
• Propose a scalable, interpretable learning framework based on Tsetlin Automata that builds propositional clauses.
• Coordinate millions of automata via a game with Type I and Type II feedback to guarantee global, not local, optima.
• Leverage bit-pattern clauses for efficient computation and interpretability.
• Demonstrate competitive performance on diverse datasets and show potential as a building block for larger architectures.

提出的方法

• Represent patterns as conjunctive clauses over a literals set derived from input bits and their negations.
• Use a team of 2o Tsetlin Automata to decide inclusion or exclusion of each literal in each clause.
• Employ a novel game-based learning with Type I and Type II feedback to guide clause construction toward accurate patterns.
• Coordinate clause learning with a resource allocation mechanism to ensure diverse coverage of sub-patterns via a summation target.
• Infer final outputs by voting across clauses with a polarity scheme (positive/negative) and a threshold.
• Provide interpretation-friendly bit patterns that correspond to human-readable propositional formulas.

实验结果

研究问题

• RQ1 How can a large collection of Tsetlin Automata be coordinated to form optimal propositional formulas for pattern recognition?
• RQ2 Can Type I and Type II feedback in a game-theoretic setting overcome the vanishing signal-to-noise ratio in decentralized learning?
• RQ3 Do propositional-logic-based clauses learned online achieve competitive accuracy compared with standard ML methods on benchmark datasets?
• RQ4 To what extent are the resulting patterns interpretable and usable as building blocks for larger architectures?

主要发现

• The Tsetlin Machine learns propositional formulas composed of clauses that enable accurate pattern recognition.
• A game-theoretic coordination scheme with Type I and Type II feedback mitigates noise and supports learning with millions of automata.
• The approach yields competitive accuracy against SVMs, Decision Trees, Random Forests, Naive Bayes, Logistic Regression, and Neural Networks across multiple benchmarks.
• Clauses are bit-pattern representations that are easy to interpret and inspect by humans.
• The method supports online learning and can be integrated as a building block to construct more advanced architectures.