[Paper Review] Detection, Recognition and Tracking of Moving Objects from Real-time Video via SP Theory of Intelligence and Species Inspired PSO

Kumar S. Ray, Sayandip Dutta+1 more|arXiv (Cornell University)|Apr 12, 2017

Video Surveillance and Tracking Methods14 references7 citations

TL;DR

This paper proposes a novel real-time framework for detecting, recognizing, and tracking moving objects in video using the SP Theory of Intelligence for object representation and species-inspired Particle Swarm Optimization (PSO) for tracking. By leveraging multiple alignments and polythetic categories to model hierarchical parts and subparts, the method achieves robust recognition under occlusion and scene variability, with competitive performance on standard video benchmarks like David and Jogging.

ABSTRACT

In this paper, we address the basic problem of recognizing moving objects in video images using SP Theory of Intelligence. The concept of SP Theory of Intelligence which is a framework of artificial intelligence, was first introduced by Gerard J Wolff, where S stands for Simplicity and P stands for Power. Using the concept of multiple alignment, we detect and recognize object of our interest in video frames with multilevel hierarchical parts and subparts, based on polythetic categories. We track the recognized objects using the species based Particle Swarm Optimization (PSO). First, we extract the multiple alignment of our object of interest from training images. In order to recognize accurately and handle occlusion, we use the polythetic concepts on raw data line to omit the redundant noise via searching for best alignment representing the features from the extracted alignments. We recognize the domain of interest from the video scenes in form of wide variety of multiple alignments to handle scene variability. Unsupervised learning is done in the SP model following the DONSVIC principle and natural structures are discovered via information compression and pattern analysis. After successful recognition of objects, we use species based PSO algorithm as the alignments of our object of interest is analogues to observation likelihood and fitness ability of species. Subsequently, we analyze the competition and repulsion among species with annealed Gaussian based PSO. We have tested our algorithms on David, Walking2, FaceOcc1, Jogging and Dudek, obtaining very satisfactory and competitive results.

Motivation & Objective

• To address the challenge of recognizing moving objects in real-time video under varying lighting, pose, and occlusion conditions.
• To develop a unified framework that integrates object detection, recognition, and tracking using principles from the SP Theory of Intelligence.
• To improve tracking robustness by modeling object alignments as fitness functions in a species-based PSO optimization framework.
• To enable unsupervised learning through information compression and pattern discovery via the DONSVIC principle.
• To handle scene variability by representing objects through multiple hierarchical alignments using polythetic categories.

Proposed method

• Utilizes the SP Theory of Intelligence, where 'S' stands for Simplicity and 'P' for Power, to represent objects via multiple hierarchical alignments of parts and subparts.
• Applies polythetic categories to model object features by identifying commonalities across multiple instances, reducing noise and redundancy in raw image data.
• Employs unsupervised learning based on the DONSVIC principle to discover natural structures through information compression and pattern analysis.
• Maps object alignments to observation likelihood and fitness in a species-based PSO algorithm, where each particle represents a potential object state.
• Implements annealed Gaussian-based PSO to model competition and repulsion among species (particles), enhancing convergence and robustness during tracking.
• Uses training images to extract multiple alignments, which are then used as reference templates for real-time recognition and tracking in video sequences.

Experimental results

Research questions

• RQ1 How can object recognition in real-time video be improved under occlusion and scene variability using a unified cognitive framework?
• RQ2 To what extent can the SP Theory of Intelligence enable robust representation of hierarchical object parts through multiple alignments?
• RQ3 How does species-inspired PSO enhance tracking performance compared to standard PSO in dynamic video environments?
• RQ4 Can unsupervised learning via the DONSVIC principle effectively discover natural object structures without labeled data?
• RQ5 How do polythetic categories contribute to noise reduction and improved recognition accuracy in raw video data?

Key findings

• The proposed method achieved highly competitive recognition and tracking results on benchmark video sequences including David, Walking2, FaceOcc1, Jogging, and Dudek.
• The use of multiple alignments and polythetic categories significantly improved recognition robustness under partial occlusion and scene variations.
• Species-inspired PSO demonstrated enhanced convergence and stability in tracking by modeling inter-particle competition and repulsion using annealed Gaussian distributions.
• Unsupervised learning based on the DONSVIC principle successfully discovered natural object structures through information compression, enabling effective feature extraction without manual annotation.
• The integration of SP-based representation with PSO-based tracking led to a cohesive system that outperformed baseline methods in handling complex video dynamics.
• The framework showed strong generalization across diverse video domains, indicating scalability and adaptability to real-world surveillance and monitoring applications.