[Paper Review] The Importance of Song Context and Song Order in Automated Music Playlist Generation

Andreu Vall, Massimo Quadrana+2 more|arXiv (Cornell University)|Jul 12, 2018

Music and Audio Processing10 references4 citations

TL;DR

This paper investigates the impact of song context and order in automated music playlist generation using three models—popularity-based, collaborative filtering, and RNN—on two curated playlist datasets. Results show song context significantly improves recommendation quality, though popularity bias can mask this effect; song order, however, does not notably enhance predictions beyond context.

ABSTRACT

The automated generation of music playlists can be naturally regarded as a sequential task, where a recommender system suggests a stream of songs that constitute a listening session. In order to predict the next song in a playlist, some of the playlist models proposed so far consider the current and previous songs in the playlist (i.e., the song context) and possibly the order of the songs in the playlist. We investigate the impact of the song context and the song order on next-song recommendations by conducting dedicated off-line experiments on two datasets of hand-curated music playlists. Firstly, we compare three playlist models, each able to consider a different song context length: a popularity-based model, a song-based Collaborative Filtering (CF) model and a Recurrent-Neural-Network-based model (RNN). We also consider a model that predicts next songs at random as a reference. Secondly, we challenge the RNN model (the only model from the first experiment able to consider the song order) by manipulating the order of songs within playlists. Our results indicate that the song context has a positive impact on the quality of next-song recommendations, even though this effect can be masked by the bias towards very popular songs. Furthermore, in our experiments the song order does not appear as a crucial variable to predict better next-song recommendations.

Motivation & Objective

• To evaluate how song context influences next-song recommendation quality in automated playlist generation.
• To assess whether modeling song order improves recommendation performance compared to context-only approaches.
• To investigate the extent to which popularity bias affects model performance in playlist recommendation tasks.
• To compare the effectiveness of different modeling approaches—popularity-based, collaborative filtering, and RNN—on curated playlist data.
• To test the robustness of RNN models by manipulating song order within playlists to isolate its impact.

Proposed method

• Evaluated three models: popularity-based, song-based collaborative filtering (CF), and a recurrent neural network (RNN) model.
• Used two datasets of hand-curated music playlists to conduct off-line experiments.
• Trained the RNN model to capture sequential dependencies by considering the order of songs in a playlist.
• Conducted ablation studies by shuffling song order in playlists to isolate the effect of sequence order on RNN predictions.
• Used standard evaluation metrics (e.g., precision, recall) to compare model performance across different context lengths.
• Treated random next-song prediction as a baseline to measure improvement over chance.

Experimental results

Research questions

• RQ1 How does increasing the length of song context affect the quality of next-song recommendations?
• RQ2 To what extent does modeling song order improve recommendation performance compared to context-only models?
• RQ3 How does popularity bias influence the performance of playlist recommendation models?
• RQ4 Can RNN-based models effectively leverage song order to improve recommendations on curated playlists?
• RQ5 How robust are RNN models to perturbations in song order, such as random shuffling?

Key findings

• Song context significantly improves recommendation quality, with longer context lengths yielding better performance across all models.
• The popularity-based model performs poorly compared to context-aware models, indicating that popularity alone is insufficient for high-quality recommendations.
• The RNN model outperforms both the popularity-based and collaborative filtering models, demonstrating the value of sequential modeling.
• Despite the RNN's ability to model order, manipulating song order through shuffling had minimal impact on prediction quality, suggesting order is not a crucial factor in this setting.
• Popularity bias masks the positive effect of song context, particularly in models that do not explicitly account for sequential dependencies.
• The lack of improvement from order manipulation implies that the sequential patterns learned by RNNs may be more influenced by content and context than by strict order.