[Paper Review] DeepDrum: An Adaptive Conditional Neural Network

Dimos Makris, Μάξιμος Καλιακάτσος-Παπακώστας+1 more|arXiv (Cornell University)|Sep 17, 2018

Music and Audio Processing9 references3 citations

TL;DR

DeepDrum proposes an adaptive conditional neural network combining LSTM layers for learning drum rhythm sequences with feed-forward (conditional) layers that incorporate external musical constraints—such as bass and guitar notes, tempo, and time signatures. The model generates stylistically coherent drum patterns that adapt to unseen musical conditions, including unfamiliar time signatures, demonstrating robust generalization beyond training data.

ABSTRACT

Considering music as a sequence of events with multiple complex dependencies, the Long Short-Term Memory (LSTM) architecture has proven very efficient in learning and reproducing musical styles. However, the generation of rhythms requires additional information regarding musical structure and accompanying instruments. In this paper we present DeepDrum, an adaptive Neural Network capable of generating drum rhythms under constraints imposed by Feed-Forward (Conditional) Layers which contain musical parameters along with given instrumentation information (e.g. bass and guitar notes). Results on generated drum sequences are presented indicating that DeepDrum is effective in producing rhythms that resemble the learned style, while at the same time conforming to given constraints that were unknown during the training process.

Motivation & Objective

• To develop a deep learning model capable of generating drum rhythms that reflect learned musical styles while adapting to external musical constraints.
• To address the limitation of existing models that generate rhythms in isolation, without considering interactions with other instruments or structural elements.
• To enable generation of drum patterns under conditions not present in the training data, such as novel time signatures (e.g., 3/8, 9/8).
• To simulate human-like drumming behavior by allowing the model to respond to both past and future musical context.

Proposed method

• The architecture uses three separate LSTM blocks, each responsible for predicting one of three drum component types (e.g., snare, kick, hi-hat), learning sequential dependencies in drum events.
• Two feed-forward (FF) modules—Pre-FF and Post-FF—process conditional inputs, including one-hot encoded representations of bass, guitar, tempo, metrical structure, and grouping over a moving temporal window.
• The Pre-FF processes past musical context and is concatenated with each drum input before the LSTM block; the Post-FF processes current and future information and is concatenated with the LSTM output.
• The model employs two stacked LSTM layers with 256 hidden units and dropout regularization (0.2) on all connections, using ReLU and linear activations in the FF layers.
• Conditional inputs are encoded in one-hot format and processed through a moving window to capture temporal context across time-steps.
• The final output is generated via independent softmax layers for each drum component, enabling end-to-end training with cross-entropy loss.

Experimental results

Research questions

• RQ1 Can a deep neural network generate drum rhythms that imitate a learned musical style while adapting to external musical constraints not seen during training?
• RQ2 How effectively can the model generalize to novel musical conditions such as unfamiliar time signatures (e.g., 3/8, 9/8) that were absent in the training data?
• RQ3 To what extent do conditional feed-forward layers improve the model’s ability to generate rhythmically coherent and contextually appropriate drum patterns?
• RQ4 Can the model simulate human drummer behavior by responding to both past and future musical events (e.g., tempo changes, phrase boundaries)?

Key findings

• DeepDrum successfully generated drum rhythms that conform to musical constraints such as bass and guitar notes, tempo, and time signatures, even when these were not present in the training data.
• The model produced sequences that resemble the learned musical style, as confirmed by t-SNE visualization of global rhythm features clustering near ground-truth examples of the training style.
• Late-generation outputs (after 100 training epochs) for pieces in the training style (PT–PF) showed closer alignment with ground-truth feature distributions, indicating improved learning and generalization.
• The model generated plausible drum patterns for disco-style pieces (AB), with feature distributions shifting toward the target style over training, demonstrating cross-genre adaptability.
• The inclusion of conditional feed-forward layers significantly enhanced the model’s ability to maintain rhythmic coherence and respond to structural changes such as phrase boundaries and tempo shifts.
• The model demonstrated robustness in handling previously unseen time signatures (e.g., 3/8, 9/8), indicating that conditional inputs enable generalization beyond the training distribution.