[Paper Review] Spatio-activity based object detection

Jarrad Springett, J. Vendrig|ArXiv.org|Mar 11, 2008

Video Surveillance and Tracking Methods5 references3 citations

TL;DR

This paper proposes SAMMI, a lightweight spatio-activity based object detection method that operates directly on DCT coefficients from JPEG-encoded video streams, avoiding full pixel-domain reconstruction. By leveraging temporal coherence across neighboring blocks and using an approximated median method for scene modeling, SAMMI achieves high accuracy and robustness—especially in detecting small or stationary objects—while reducing processing time and memory usage by up to 97% compared to Mixture of Gaussians.

ABSTRACT

We present the SAMMI lightweight object detection method which has a high level of accuracy and robustness, and which is able to operate in an environment with a large number of cameras. Background modeling is based on DCT coefficients provided by cameras. Foreground detection uses similarity in temporal characteristics of adjacent blocks of pixels, which is a computationally inexpensive way to make use of object coherence. Scene model updating uses the approximated median method for improved performance. Evaluation at pixel level and application level shows that SAMMI object detection performs better and faster than the conventional Mixture of Gaussians method.

Motivation & Objective

• Address the scalability challenge of real-time video analysis in large camera networks by reducing computational overhead in foreground detection.
• Improve detection accuracy for small and stationary objects that are often missed by conventional background modeling techniques.
• Minimize resource usage (processing time and memory) without sacrificing detection performance, enabling deployment in large-scale surveillance systems.
• Reduce object fragmentation caused by noise and aliasing through iterative, neighbor-adapted classification based on temporal coherence.
• Enable efficient, real-time operation by processing video data directly in the DCT domain, avoiding costly pixel-domain decompression.

Proposed method

• Processes Motion JPEG streams directly in the DCT domain, using quantized DCT coefficients as input instead of reconstructing pixel values.
• Applies a spatio-temporal coherence model where neighboring blocks are classified together based on similarity in their temporal activity patterns.
• Employs an iterative, neighbor-adapted classification process that refines foreground/background decisions using temporal consistency across blocks.
• Uses an approximated median method for scene model updating, which improves robustness to lighting changes and noise while reducing computational cost.
• Applies a mode-reduction strategy that retains only components visible for a minimum percentage of time, filtering out transient noise.
• Introduces a configurable active mode bonus to balance sensitivity to new moving objects versus stability for stationary ones.

Experimental results

Research questions

• RQ1 Can object detection accuracy be maintained while significantly reducing computational cost in large-scale video surveillance systems?
• RQ2 How effective is DCT-domain processing in preserving detection performance compared to traditional pixel-domain methods like Mixture of Gaussians?
• RQ3 To what extent does exploiting temporal coherence between neighboring blocks reduce object fragmentation and improve tracking suitability?
• RQ4 Can an approximated median method for scene modeling outperform exponential moving averages in terms of robustness and efficiency?
• RQ5 What is the impact of iterative neighbor-adapted classification on detection of small or stationary objects in complex scenes?

Key findings

• SAMMI achieved an F1-measure of 0.80 at pixel level, significantly outperforming Mixture of Gaussians (0.55), primarily due to higher recall (0.95 vs. 0.42).
• Tracking suitability improved from 0.38 (SAMMI with 3 iterations) to 0.27 (0 iterations), indicating reduced object fragmentation with more iterations.
• Processing speed increased from 5 fps (Mixture of Gaussians) to 25 fps (SAMMI with 3 iterations), demonstrating a 5x speedup despite minimal optimization.
• Scene model memory usage was reduced from 36,288 KB (Mixture of Gaussians) to just 1,080 KB (SAMMI), a 97% reduction.
• The method maintained high accuracy even for small objects and stationary foregrounds, which are typically missed by conventional approaches.
• The approximated median method improved robustness to lighting changes and noise, especially in bright or dark regions, without increasing computational cost.