[Paper Review] Using SVDD in SimpleMKL for 3D-Shapes Filtering

Gaëlle Loosli, Hattoibe Aboubacar|arXiv (Cornell University)|Dec 7, 2017

Anomaly Detection Techniques and Applications10 references3 citations

TL;DR

This paper proposes MK-SVDD and Slim-MK-SVDD, novel extensions of Support Vector Data Description (SVDD) within the SimpleMKL framework for 3D-shape filtering and outlier detection. By incorporating a novel penalization term to favor tighter decision boundaries, the Slim-MK-SVDD variant improves performance over standard SVDD and One-Class SVM in both supervised and unsupervised settings, particularly in distinguishing similar 3D shape classes with few positive examples.

ABSTRACT

This paper proposes the adaptation of Support Vector Data Description (SVDD) to the multiple kernel case (MK-SVDD), based on SimpleMKL. It also introduces a variant called Slim-MK-SVDD that is able to produce a tighter frontier around the data. For the sake of comparison, the equivalent methods are also developed for One-Class SVM, known to be very similar to SVDD for certain shapes of kernels. Those algorithms are illustrated in the context of 3D-shapes filtering and outliers detection. For the 3D-shapes problem, the objective is to be able to select a sub-category of 3D-shapes, each sub-category being learned with our algorithm in order to create a filter. For outliers detection, we apply the proposed algorithms for unsupervised outliers detection as well as for the supervised case.

Motivation & Objective

• To address the challenge of filtering 3D-shapes using only a few positive examples of a target shape class.
• To adapt SVDD to the multiple kernel learning (MKL) framework using the SimpleMKL algorithm for improved generalization and flexibility.
• To develop a tighter decision boundary method via a novel penalization term that reduces overfitting and improves separation from similar classes.
• To evaluate the proposed methods in both supervised and unsupervised outlier detection scenarios on 3D-shape datasets.

Proposed method

• Adapts SVDD to the multiple kernel setting by formulating a primal optimization problem that combines kernels via convex weights, ensuring the smallest sphere encloses all positive training examples.
• Derives the dual problem for MK-SVDD using Lagrangian relaxation and KKT conditions, enabling optimization via the SimpleMKL framework with gradient-based solvers.
• Introduces Slim-MK-SVDD by adding a penalization term to the SimpleMKL objective that discourages solutions relying on few support vectors, promoting tighter decision boundaries.
• Uses Gaussian and polynomial kernels in the multiple kernel combination, with kernel weights learned via gradient descent under simplex constraints.
• Employs Extended Reeb Graphs as 3D-shape descriptors to represent shape data in a high-dimensional feature space suitable for kernel methods.
• Compares MK-SVDD and Slim-MK-SVDD against standard SVDD, One-Class SVM, and MK-OCSVM across multiple 3D-shape filtering and outlier detection tasks.

Experimental results

Research questions

• RQ1 Can SVDD be effectively extended to the multiple kernel learning (MKL) framework using the SimpleMKL algorithm for 3D-shape filtering?
• RQ2 Does the proposed Slim-MK-SVDD variant produce tighter decision boundaries than standard MK-SVDD and MK-OCSVM in 3D-shape classification tasks?
• RQ3 How do MK-SVDD and Slim-MK-SVDD perform in distinguishing a target 3D-shape class from similar classes when only a few positive examples are available?
• RQ4 Can the proposed methods detect outliers effectively in both supervised and unsupervised settings using only positive training examples?

Key findings

• Slim-MK-SVDD achieves an average AUC of 0.945 in differentiating a target 3D shape class from a similar class, significantly outperforming MK-OCSVM (AUC 0.64) and MK-SVDD (AUC 0.94) in the same task.
• In the task of filtering a target class from all others, Slim-MK-SVDD achieves an average AUC of 0.90, with the first positive example appearing on average at rank 7.5, indicating strong early detection capability.
• For the most challenging case (class 11), Slim-MK-SVDD achieves an AUC of 0.93 with the first target example ranked at 8.7, outperforming MK-OCSVM (AUC 0.93 but first target at rank 128.1).
• The Slim-MK-SVDD variant reduces the average rank of the first target example from 13.5 (MK-SVDD) to 2.6 in class 1, demonstrating superior separation from similar classes.
• In outlier detection, Slim-MK-SVDD achieves a 20% win ratio in the supervised setting, significantly better than random (33%) and outperforming MK-OCSVM (0% win ratio).
• The proposed penalization term in Slim-MK-SVDD leads to a 15% improvement in AUC on average compared to standard MK-SVDD in distinguishing similar shape classes.