[Paper Review] ALOHA: Auxiliary Loss Optimization for Hypothesis Augmentation

Ethan M. Rudd, Felipe N. Ducau+3 more|arXiv (Cornell University)|Mar 13, 2019

Advanced Malware Detection Techniques29 references8 citations

TL;DR

This paper proposes ALOHA, a multi-task learning framework that improves malware detection by training deep neural networks on multiple auxiliary loss functions derived from threat intelligence metadata—such as multi-source labels, detection counts, and semantic tags—resulting in a 53.8% reduction in false negative rate at 10^{-5} FPR and a 42.6% reduction at 10^{-3} FPR compared to single-task baselines.

ABSTRACT

Malware detection is a popular application of Machine Learning for Information Security (ML-Sec), in which an ML classifier is trained to predict whether a given file is malware or benignware. Parameters of this classifier are typically optimized such that outputs from the model over a set of input samples most closely match the samples' true malicious/benign (1/0) target labels. However, there are often a number of other sources of contextual metadata for each malware sample, beyond an aggregate malicious/benign label, including multiple labeling sources and malware type information (e.g., ransomware, trojan, etc.), which we can feed to the classifier as auxiliary prediction targets. In this work, we fit deep neural networks to multiple additional targets derived from metadata in a threat intelligence feed for Portable Executable (PE) malware and benignware, including a multi-source malicious/benign loss, a count loss on multi-source detections, and a semantic malware attribute tag loss. We find that incorporating multiple auxiliary loss terms yields a marked improvement in performance on the main detection task. We also demonstrate that these gains likely stem from a more informed neural network representation and are not due to a regularization artifact of multi-target learning. Our auxiliary loss architecture yields a significant reduction in detection error rate (false negatives) of 42.6% at a false positive rate (FPR) of $10^{-3}$ when compared to a similar model with only one target, and a decrease of 53.8% at $10^{-5}$ FPR.

Motivation & Objective

• To improve malware detection performance by leveraging rich, multi-source metadata from threat intelligence feeds as auxiliary training targets.
• To investigate whether performance gains from multi-target learning stem from improved representations or merely regularization effects.
• To develop a robust, per-sample loss weighting scheme that accommodates missing or incomplete metadata during training.
• To evaluate the impact of different auxiliary loss types (binary, Poisson, multi-label) on detection performance across varying false positive rate thresholds.
• To demonstrate that auxiliary loss optimization enhances model generalization without altering the input feature space or requiring additional data collection.

Proposed method

• Training deep neural networks with multiple auxiliary loss functions: binary cross-entropy for multi-source malicious/benign labels, Poisson loss for detection count predictions, and multi-label cross-entropy for semantic malware attribute tags.
• Implementing a per-sample loss weighting scheme that dynamically adjusts loss contributions based on label availability, enabling effective training even with incomplete metadata.
• Using a shared feature encoder across all auxiliary heads to ensure representation consistency and joint optimization of all objectives.
• Evaluating model performance via ROC curves and detection error rates across varying false positive rate thresholds (e.g., 10^{-3}, 10^{-5}).
• Conducting ablation studies to isolate the contribution of each auxiliary loss type and assess their individual and combined effects.
• Performing ablation and analysis to distinguish between representation improvement and regularization as the source of performance gains.

Experimental results

Research questions

• RQ1 Does incorporating multiple auxiliary losses derived from threat intelligence metadata significantly improve main-task malware detection performance?
• RQ2 Are performance gains due to improved neural network representations or simply a regularization effect from multi-objective training?
• RQ3 How do different types of auxiliary losses (e.g., binary, count, multi-label) affect detection performance across different false positive rate regimes?
• RQ4 Can auxiliary loss optimization improve detection in low-FPR regimes where false negatives are most critical?
• RQ5 Can the proposed framework handle missing or incomplete metadata without degrading performance?

Key findings

• Incorporating all auxiliary losses reduced the detection error rate (false negative rate) by 53.8% at a false positive rate of 10^{-5} compared to a single-task baseline.
• At a false positive rate of 10^{-3}, the detection error rate was reduced by 42.6% when using all auxiliary losses.
• The multi-label vendor and semantic attribute tag losses had the strongest impact on performance at low false positive rates (≤ 10^{-3}), improving detection in critical operational zones.
• The Poisson loss on detection counts provided the most significant boost to AUC and improved detection performance at higher false positive rates (≥ 10^{-3}).
• Performance gains were attributed to a more informative and generalized neural network representation, not merely regularization from multi-task learning.
• The model achieved consistently lower-variance ROC curves across all FPR levels, indicating more stable and reliable detection performance.