[Paper Review] Multi-Resolution Weak Supervision for Sequential Data

Frédéric Sala, Paroma Varma+9 more|arXiv (Cornell University)|Oct 21, 2019

AI in cancer detection56 references4 citations

TL;DR

Dugong is the first framework for multi-resolution weak supervision in sequential data, modeling per-frame, per-window, and per-scene labeling sources with complex correlations. It uses linear system solving and parameter sharing to estimate source accuracies and correlations without ground truth, achieving 24.2 F1 points higher than existing weak supervision methods and 16.0 F1 points over traditional supervision on video and sensor tasks.

ABSTRACT

Since manually labeling training data is slow and expensive, recent industrial and scientific research efforts have turned to weaker or noisier forms of supervision sources. However, existing weak supervision approaches fail to model multi-resolution sources for sequential data, like video, that can assign labels to individual elements or collections of elements in a sequence. A key challenge in weak supervision is estimating the unknown accuracies and correlations of these sources without using labeled data. Multi-resolution sources exacerbate this challenge due to complex correlations and sample complexity that scales in the length of the sequence. We propose Dugong, the first framework to model multi-resolution weak supervision sources with complex correlations to assign probabilistic labels to training data. Theoretically, we prove that Dugong, under mild conditions, can uniquely recover the unobserved accuracy and correlation parameters and use parameter sharing to improve sample complexity. Our method assigns clinician-validated labels to population-scale biomedical video repositories, helping outperform traditional supervision by 36.8 F1 points and addressing a key use case where machine learning has been severely limited by the lack of expert labeled data. On average, Dugong improves over traditional supervision by 16.0 F1 points and existing weak supervision approaches by 24.2 F1 points across several video and sensor classification tasks.

Motivation & Objective

• Address the lack of weak supervision frameworks that handle multi-resolution labeling sources in sequential data such as video and sensor streams.
• Model complex correlations and unknown accuracies across per-frame, per-window, and per-scene supervision sources without access to ground truth labels.
• Reduce sample complexity in long sequences by leveraging parameter sharing and structural assumptions on source correlations.
• Incorporate distribution priors (e.g., label frequency) to improve label estimation fidelity in low-data regimes.
• Enable scalable, end-to-end training of deep learning models using weakly supervised data from diverse, noisy sources in real-world applications.

Proposed method

• Formalize multi-resolution weak supervision as a latent variable model with sequential dependencies across resolutions (frame, window, scene).
• Estimate source accuracy and correlation parameters by solving two linear systems derived from source agreements and disagreements.
• Enforce model identifiability under mild conditions on source correlation structure, ensuring unique recovery of parameters.
• Apply parameter sharing across resolutions and time steps to reduce the number of parameters and improve sample efficiency.
• Integrate user-defined or data-derived class priors to regularize label estimation and improve performance on rare events.
• Develop a stochastic gradient descent (SGD) variant for efficient training in deep learning frameworks like PyTorch, achieving 90× faster inference.

Experimental results

Research questions

• RQ1 Can a weak supervision framework effectively model multi-resolution labeling sources (e.g., frame, window, scene) in sequential data while accounting for complex correlations?
• RQ2 Under what conditions can the underlying model parameters (accuracies and correlations) be uniquely recovered from unlabeled data?
• RQ3 How does parameter sharing across resolutions affect sample complexity and model generalization in long sequential sequences?
• RQ4 To what extent can distribution priors improve label estimation accuracy, especially for rare event detection in sequential data?
• RQ5 How does Dugong compare in performance to traditional supervision and existing weak supervision baselines across diverse sequential classification tasks?

Key findings

• Dugong improves over traditional supervision by 16.0 F1 points on average across multiple video and sensor classification tasks.
• Dugong outperforms existing weak supervision approaches by 24.2 F1 points on average, demonstrating significant gains in label quality.
• The ablation study shows that removing parameter tying reduces performance by 9.2 F1 points on average, highlighting the importance of parameter sharing.
• Removing temporal dependencies leads to a 10.2 F1 point drop, indicating that modeling sequential correlations prevents overestimation of source reliability.
• User-defined class priors improve performance by 14.8 F1 points on the Shot dataset, where rare transitions are critical, showing the value of domain-informed priors.
• The SGD variant of Dugong achieves 90× faster runtime compared to standard implementations, enabling deployment in modern deep learning pipelines.