[Paper Review] Characterizing Classes of Potential Outliers through Traffic Data Set Data Signature 2D nMDS Projection

Erlo Robert F. Oquendo, Jhoirene Clemente+2 more|arXiv (Cornell University)|Feb 24, 2017

Traffic and Road Safety1 references3 citations

TL;DR

This paper proposes a formal, statistically grounded method to classify potential outliers in traffic data using 2D non-metric multidimensional scaling (nMDS) projections combined with confidence bands and ellipses. It distinguishes three classes—absolute, valid, and ambiguous outliers—achieving higher precision than prior visual inspection methods, identifying 11 new outliers beyond the 10 previously reported in literature using the 2006 NLEX Balintawak Northbound dataset.

ABSTRACT

This paper presents a formal method for characterizing the potential outliers from the data signature projection of traffic data set using Non-Metric Multidimensional Scaling (nMDS) visualization. Previous work had only relied on visual inspection and the subjective nature of this technique may derive false and invalid potential outliers. The identification of correct potential outliers had already been an open problem proposed in literature. This is due to the fact that they pinpoint areas and time frames where traffic incidents/accidents occur along the North Luzon Expressway (NLEX) in Luzon. In this paper, potential outliers are classified into (1) absolute potential outliers; (2) valid potential outliers; and (3) ambiguous potential outliers through the use of confidence bands and confidence ellipse. A method is also described to validate cluster membership of identified ambiguous potential outliers. Using the 2006 NLEX Balintawak Northbound (BLK-NB) data set, we were able to identify two absolute potential outliers, nine valid potential outliers, and five ambiguous potential outliers. In a literature where Vector Fusion was used, 10 potential outliers were identified. Given the results for the nMDS visualization using the confidence bands and confidence ellipses, all of these 10 potential outliers were also found and 8 new potential outliers were also found.

Motivation & Objective

• To address the open problem of unreliable outlier identification in traffic data due to subjective visual inspection.
• To formalize the detection of potential outliers in traffic datasets using data signature visualization.
• To classify identified outliers into three distinct categories: absolute, valid, and ambiguous, based on statistical confidence regions.
• To validate cluster membership of ambiguous outliers through a defined method to improve reliability.

Proposed method

• Applying non-metric multidimensional scaling (nMDS) to project high-dimensional traffic data signatures into a 2D visualization space.
• Constructing 95% confidence bands and confidence ellipses around the nMDS-embedded data points to define statistical boundaries for outlier detection.
• Classifying outliers based on their position relative to the confidence bands and ellipses: absolute if outside both, valid if outside one but inside the other, ambiguous if near the boundary.
• Using a cluster validation technique to assess the membership of ambiguous outliers, ensuring they are not misclassified due to proximity to cluster centers.
• Analyzing the 2006 NLEX Balintawak Northbound (BLK-NB) traffic dataset as the primary empirical test case.
• Comparing results with prior work using Vector Fusion, which identified 10 outliers, to evaluate sensitivity and discovery power.

Experimental results

Research questions

• RQ1 How can statistical confidence regions improve the reliability of outlier detection in traffic data compared to visual inspection alone?
• RQ2 What are the distinct classes of potential outliers identifiable through nMDS projection with confidence bands and ellipses?
• RQ3 Can the proposed method detect new outliers beyond those identified in previous studies using different techniques?
• RQ4 How can cluster membership of ambiguous outliers be validated to reduce false positives?
• RQ5 To what extent does the nMDS-based approach outperform or align with prior methods like Vector Fusion in identifying meaningful traffic outliers?

Key findings

• The method identified two absolute potential outliers, nine valid potential outliers, and five ambiguous potential outliers in the 2006 NLEX Balintawak Northbound dataset.
• All 10 potential outliers previously reported using Vector Fusion were successfully recovered using the nMDS-based approach with confidence regions.
• The method discovered eight new potential outliers not identified in the prior Vector Fusion study, indicating higher sensitivity.
• The use of confidence bands and ellipses significantly reduced subjectivity in outlier classification, enabling a formal, repeatable process.
• The cluster validation method successfully assessed the membership of ambiguous outliers, enhancing the reliability of the classification.
• The results demonstrate that nMDS with statistical confidence regions provides a more robust and objective framework for identifying traffic-related anomalies.