[Paper Review] Deep Learning Models in Software Requirements Engineering

Maria Naumcheva|arXiv (Cornell University)|May 17, 2021

Software Engineering Research27 references4 citations

TL;DR

This paper investigates the application of Variational Autoencoders (VAEs) for automated software requirements generation, using a fine-tuned sentence autoencoder on a 3,000-sample requirements dataset. Despite generating mostly repetitive, non-semantic phrases like "the shall shall to to," the study establishes a foundational framework for deep generative modeling in requirements engineering and identifies dataset size and model architecture as key levers for future improvement.

ABSTRACT

Requirements elicitation is an important phase of any software project: the errors in requirements are more expensive to fix than the errors introduced at later stages of software life cycle. Nevertheless, many projects do not devote sufficient time to requirements. Automated requirements generation can improve the quality of software projects. In this article we have accomplished the first step of the research on this topic: we have applied the vanilla sentence autoencoder to the sentence generation task and evaluated its performance. The generated sentences are not plausible English and contain only a few meaningful words. We believe that applying the model to a larger dataset may produce significantly better results. Further research is needed to improve the quality of generated data.

Motivation & Objective

• To explore the feasibility of using deep generative models, particularly Variational Autoencoders (VAEs), for automated software requirements generation.
• To assess whether VAEs can learn meaningful semantic representations from software requirements texts and generate plausible, coherent requirements.
• To identify limitations in current VAE-based approaches when applied to small, domain-specific datasets in requirements engineering.
• To lay the groundwork for future research by establishing a baseline model and dataset for automated requirements generation.

Proposed method

• Trained a Variational Autoencoder (VAE) with a bidirectional LSTM encoder, a variational inference module, and an LSTM decoder for sequential word generation.
• Used pre-trained GloVe word embeddings to represent input words in a dense vector space.
• Preprocessed the dataset by tokenizing sentences, removing punctuation, and limiting vocabulary to the 1,000 most frequent words.
• Performed latent space interpolation between two real requirements to evaluate generation quality.
• Trained the model for 100 epochs using the custom requirements dataset of ~3,000 entries.
• Evaluated generated outputs by comparing them to real requirements using qualitative inspection and semantic coherence analysis.

Experimental results

Research questions

• RQ1 Can a VAE-based autoencoder generate semantically meaningful and grammatically correct software requirements from a small, domain-specific dataset?
• RQ2 How do the generated requirements compare in quality and coherence to real requirements in terms of lexical diversity and semantic content?
• RQ3 What impact does dataset size have on the performance of VAEs in software requirements generation?
• RQ4 Which architectural modifications (e.g., conditional VAEs, dilated CNN decoders) could improve generation quality for this task?

Key findings

• The model generated sentences dominated by repeated stop words such as "the" and "to," with only occasional meaningful terms like "system" and "shall.".
• The most frequent meaningful words in generated outputs were "system" and "shall," which are highly prevalent in real requirements, indicating partial learning of key semantic features.
• Despite 100 epochs of training, the generated sentences lacked grammatical structure and semantic coherence, indicating poor generation quality on the current dataset.
• The model’s performance was limited by the small size of the training dataset (~3,000 entries), suggesting that larger datasets are essential for meaningful improvement.
• Latent space interpolation produced outputs with no clear semantic progression, reinforcing the model’s failure to learn meaningful disentangled representations.
• The study concludes that while VAEs are a viable starting point for requirements generation, significant improvements are needed through larger datasets and advanced architectures.