[논문 리뷰] Tracklets Predicting Based Adaptive Graph Tracking

Chaobing Shan, Chunbo Wei외 5명|arXiv (Cornell University)|2020. 10. 18.

Video Surveillance and Tracking Methods참고 문헌 69인용 수 35

한 줄 요약

TPAGT는 트랙렛 모션 기반 특징 재추출 및 위치, 외관, 역사를 융합하는 적응형 그래프 신경망을 통해 강건한 다중 객체 추적을 달성하며 MOT16/17에서 최첨단 MOT 점수에 도달.

ABSTRACT

Most of the existing tracking methods link the detected boxes to the tracklets using a linear combination of feature cosine distances and box overlap. But the problem of inconsistent features of an object in two different frames still exists. In addition, when extracting features, only appearance information is utilized, neither the location relationship nor the information of the tracklets is considered. We present an accurate and end-to-end learning framework for multi-object tracking, namely extbf{TPAGT}. It re-extracts the features of the tracklets in the current frame based on motion predicting, which is the key to solve the problem of features inconsistent. The adaptive graph neural network in TPAGT is adopted to fuse locations, appearance, and historical information, and plays an important role in distinguishing different objects. In the training phase, we propose the balanced MSE LOSS to successfully overcome the unbalanced samples. Experiments show that our method reaches state-of-the-art performance. It achieves 76.5\% MOTA on the MOT16 challenge and 76.2\% MOTA on the MOT17 challenge.

연구 동기 및 목표

• 프레임 간 특징 불일치를 해결하여 강건한 다중 객체 추적의 동기를 부여.
• 모션 예측에 기반하여 현재 프레임에서 트랙렛 특징을 재추출하는 방법을 개발.
• 적응형 그래프 신경망을 통해 위치, 외관 및 역사 정보를 융합.
• 데이터 불균형 학습 샘플 문제를 균형 MSE 손실로 완화.
• MOT16 및 MOT17 벤치마크에서 최첨단 성능을 입증.

제안 방법

• Tracklets predicting-based feature re-extracting: predict tracklet motion and re-extract features in the current frame using pyramid LK to align features.
• Adaptive Graph Neural Network: treat detections and tracklets as bipartite graph nodes; adaptively weight edge info using IOU and feature similarity to update node embeddings.
• Compute similarity with normalized embeddings and form an output similarity matrix for matching.
• Balanced MSE Loss (BMSE): balance positive/negative sample contributions to address data imbalance in training.
• Inference uses an augmented similarity matrix with a margin and Hungarian algorithm for data association.
• Ablation utilities: backbone comparison, motion estimation methods, and AGNN variants to assess contribution of each component.

실험 결과

연구 질문

• RQ1 How to align tracklet features with current-frame detections to reduce cross-frame feature inconsistency?
• RQ2 Can adaptive graph neural networks effectively fuse location, appearance, and historical tracklet information to improve association?
• RQ3 Does balancing the loss help mitigate unbalanced sample distribution in MOT data associations?
• RQ4 What is the impact of tracklet motion-based feature re-extraction on overall MOT performance?
• RQ5 How does TPAGT compare to state-of-the-art trackers on MOT16 and MOT17 under public and private detections?

주요 결과

방법	MOTA ↑	IDF1 ↑	MT ↑	ML ↓	FP ↓	FN ↓	IDSw ↓
Ours (Public MOT16)	62.7	60.3	28.5	26.9	5077	61952	978
Ours (Public MOT17)	62.0	59.5	27.8	31.5	15114	196672	2621

• TPAGT achieves 62.7% MOTA and 60.3% IDF1 on MOT16 public detections with the proposed approach.
• TPAGT achieves 62.0% MOTA and 59.5% IDF1 on MOT17 public detections with the proposed approach.
• On MOT16 MOT17 private detections, TPAGT attains 76.5% MOTA and 68.6% IDF1 (MOTA) and 68.0% IDF1 (IDF1) respectively, surpassing several prior methods.
• Re-extracting tracklet features in the current frame (motion-based alignment) significantly improves MOTA and IDF1.
• Adaptive graph in TPAGT (AGNN) substantially improves performance over non-adaptive GNN variants.
• Balanced MSE Loss yields better results than Triplet Loss in their ablation.