[Paper Review] Descriptions of Objectives and Processes of Mechanical Learning

Chuyu Xiong|arXiv (Cornell University)|May 31, 2017

Robotic Mechanisms and Dynamics4 references3 citations

TL;DR

This paper proposes a formal framework for mechanical learning centered on objective and subjective patterns, introducing X-forms as algebraic representations for internal learning processes. It establishes that with sufficient data and specific learning capabilities, a learning machine can universally learn any pattern through strategies like embedding into parameter space or abstracting via 'squeezing to higher'—offering a theoretical foundation for universal learning and reinterpreting deep learning through this lens.

ABSTRACT

In [1], we introduced mechanical learning and proposed 2 approaches to mechanical learning. Here, we follow one such approach to well describe the objects and the processes of learning. We discuss 2 kinds of patterns: objective and subjective pattern. Subjective pattern is crucial for learning machine. We prove that for any objective pattern we can find a proper subjective pattern based upon least base patterns to express the objective pattern well. X-form is algebraic expression for subjective pattern. Collection of X-forms form internal representation space, which is center of learning machine. We discuss learning by teaching and without teaching. We define data sufficiency by X-form. We then discussed some learning strategies. We show, in each strategy, with sufficient data, and with certain capabilities, learning machine indeed can learn any pattern (universal learning machine). In appendix, with knowledge of learning machine, we try to view deep learning from a different angle, i.e. its internal representation space and its learning dynamics.

Motivation & Objective

• To formalize the objects and processes of mechanical learning, distinguishing between objective and subjective patterns.
• To establish a theoretical foundation for universal learning in mechanical systems using X-forms and data sufficiency.
• To demonstrate that learning machines with certain capabilities can learn any pattern, making them universal learners.
• To reframe deep learning through the lens of internal representation space and learning dynamics.
• To guide the design of effective learning machines with self-awareness, extensible strategies, and prior knowledge integration.

Proposed method

• Introduces X-forms as algebraic expressions for subjective patterns, forming the internal representation space of a learning machine.
• Defines data sufficiency via X-forms: sufficient to support or bound a given X-form.
• Proposes three learning strategies: embedding into parameter space, squeezing to higher abstraction from within, and from both inside and outside.
• Demonstrates that with sufficient data and specific capabilities, a learning machine becomes a universal learner.
• Uses level 1 learning machines as a base, with potential for evolution to higher-level learning via self-modification.
• Reinterprets deep learning (e.g., RBM stacks) as a specific instance of the embedding strategy in the proposed framework.

Experimental results

Research questions

• RQ1 How can objective patterns be systematically represented and learned through subjective patterns in a mechanical learning system?
• RQ2 What conditions on data and capabilities ensure that a learning machine can universally learn any pattern?
• RQ3 How does the internal representation space—defined by X-forms—govern learning dynamics and abstraction?
• RQ4 In what way does the learning-by-teaching strategy enable universal learning, and how can it inform unsupervised learning?
• RQ5 How does deep learning correspond to the proposed learning strategies, particularly embedding into parameter space?

Key findings

• For any objective pattern, a corresponding subjective pattern expressed via X-forms can be constructed using least base patterns, ensuring expressibility.
• Data sufficiency is formally defined in terms of supporting and bounding X-forms, providing a criterion for adequate training data.
• Three learning strategies—embedding, internal squeezing, and combined squeezing—are shown to enable universal learning under sufficient data and capabilities.
• The first strategy, embedding into parameter space, is identified as the core mechanism underlying deep learning in restricted Boltzmann machine stacks.
• A learning machine with the capabilities to learn by teaching and to learn without teaching is proven to be universal, suggesting these as minimal effective capabilities.
• The framework provides a theoretical basis for Mathematical Learning Theory and suggests that X-forms form a natural, navigable internal representation space for learning machines.