[Paper Review] Automatic Detection Of Noise Events at Shooting Range Using Machine Learning

Jon Nordby, Fabian Nemazi+1 more|arXiv (Cornell University)|Jul 23, 2021

Music and Audio Processing19 references4 citations

TL;DR

This paper presents a machine learning system that automatically detects and counts gunshots and explosive events at outdoor shooting ranges using acoustic sensors and time-series audio features. A convolutional neural network achieved the best performance, with 80.5% F1 score and 4.5% insertion error, enabling reliable logging of training activities for regulatory compliance.

ABSTRACT

Outdoor shooting ranges are subject to noise regulations from local and national authorities. Restrictions found in these regulations may include limits on times of activities, the overall number of noise events, as well as limits on number of events depending on the class of noise or activity. A noise monitoring system may be used to track overall sound levels, but rarely provide the ability to detect activity or count the number of events, required to compare directly with such regulations. This work investigates the feasibility and performance of an automatic detection system to count noise events. An empirical evaluation was done by collecting data at a newly constructed shooting range and training facility. The data includes tests of multiple weapon configurations from small firearms to high caliber rifles and explosives, at multiple source positions, and collected on multiple different days. Several alternative machine learning models are tested, using as inputs time-series of standard acoustic indicators such as A-weighted sound levels and 1/3 octave spectrogram, and classifiers such as Logistic Regression and Convolutional Neural Networks. Performance for the various alternatives are reported in terms of the False Positive Rate and False Negative Rate. The detection performance was found to be satisfactory for use in automatic logging of time-periods with training activity.

Motivation & Objective

• Develop an automated system to detect and count noise events at shooting ranges for regulatory compliance.
• Address the lack of event-level detection in standard noise monitoring systems that only log continuous sound levels.
• Evaluate machine learning models on real-world acoustic data from a national police training facility.
• Enable accurate logging of training activities to meet strict annual limits on gunshots and explosives.
• Improve upon existing noise monitoring by detecting impulsive events rather than just overall sound levels.

Proposed method

• Collected time-series acoustic data from multiple weapon types and positions at a national police training facility.
• Extracted standard acoustic indicators including A-weighted sound levels and 1/3 octave spectrograms as model inputs.
• Trained and compared multiple classifiers: Logistic Regression, Random Forest, and Convolutional Neural Networks (CNN).
• Used per-window evaluation with detection error tradeoff (DET) curves and per-event metrics via sed-eval.
• Applied a threshold to convert probability outputs into binary event detections for logging purposes.
• Analyzed false positives and false negatives to identify label inaccuracies and potential improvements in event boundary definition.

Experimental results

Research questions

• RQ1 Can machine learning models reliably detect and count impulsive noise events such as gunshots and explosions in real-world outdoor shooting range environments?
• RQ2 How do different machine learning models (e.g., CNN, Logistic Regression, Random Forest) compare in terms of false positive and false negative rates for noise event detection?
• RQ3 To what extent can the system support regulatory compliance by accurately logging the number of training events within annual limits?
• RQ4 What impact do labeling inconsistencies—especially around event onsets—have on model evaluation and performance?
• RQ5 Can the system be extended to distinguish between noise from training activities and other environmental sources like traffic or helicopters?

Key findings

• The Convolutional Neural Network (CNN) model achieved the highest performance with an F1 score of 80.5%, precision of 95.5%, and recall of 69.6%.
• The CNN model had a significantly lower False Positive Rate than Logistic Regression and Random Forest, which is critical given the low proportion of noisy time windows.
• The system achieved an insertion error rate of 4.5%, meeting the requirement to estimate event counts within ±10%.
• The deletion error rate was 30.4%, exceeding the target, indicating missed events remain a challenge.
• An analysis revealed that many incorrectly labeled windows were actually correctly classified, suggesting that refining event boundary definitions (e.g., focusing on impulsive onsets) could improve performance.
• The system demonstrated feasibility for automated logbook generation of training activities, supporting compliance with time-of-day and annual event limits.