[Paper Review] Code Generation as a Dual Task of Code Summarization

Bolin Wei, Ge Li+3 more|arXiv (Cornell University)|Oct 14, 2019

Software Engineering ResearchComputer Science64 citations

TL;DR

The paper proposes a dual learning framework that jointly trains code summarization (CS) and code generation (CG) models, enforcing probabilistic duality and attention weight symmetry to improve both tasks across Java and Python datasets.

ABSTRACT

Code summarization (CS) and code generation (CG) are two crucial tasks in the field of automatic software development. Various neural network-based approaches are proposed to solve these two tasks separately. However, there exists a specific intuitive correlation between CS and CG, which have not been exploited in previous work. In this paper, we apply the relations between two tasks to improve the performance of both tasks. In other words, exploiting the duality between the two tasks, we propose a dual training framework to train the two tasks simultaneously. In this framework, we consider the dualities on probability and attention weights, and design corresponding regularization terms to constrain the duality. We evaluate our approach on two datasets collected from GitHub, and experimental results show that our dual framework can improve the performance of CS and CG tasks over baselines.

Motivation & Objective

• Motivate leveraging the intrinsic duality between CS and CG to improve both tasks.
• Develop a joint training framework that couples CS and CG via probabilistic and attention-based constraints.
• Demonstrate that dual training yields improvements over state-of-the-art baselines on multiple datasets.

Proposed method

• Use Seq2Seq with encoder–decoder and attention for CS (code to comment) and CG (comment to code).
• Introduce a dual loss term based on probabilistic duality: l_dual based on P(x|y) and P(y|x) and their marginals.
• Impose an attention-duality constraint by aligning CS and CG attention via Jensen–Shannon divergence between per-token attention distributions.
• Apply joint optimization with separate parameters for CS and CG models but shared dual constraints, trained with warm-started models.
• Evaluate on Java and Python datasets with BLEU, METEOR, ROUGE-L for CS and BLEU (plus valid-code percentage) for CG.

Experimental results

Research questions

• RQ1 Can a dual-training framework leveraging CS↔CG duality improve the quality of generated comments and code?
• RQ2 Do probabilistic duality and attention-based constraints contribute to better CS and CG performance?
• RQ3 How does joint training affect code validity and attention distributions compared to independent training?
• RQ4 What are the empirical gains on Java and Python datasets for CS and CG tasks?

Key findings

• The Dual Model consistently outperforms all baselines on CS metrics (BLEU, METEOR, ROUGE-L) for both Java and Python.
• For CS on Java and Python, the Dual Model achieves the highest BLEU, METEOR, and ROUGE-L scores among compared methods.
• For CG, the Dual Model improves BLEU and the percentage of valid code over the Basic Model.
• Joint training increases the percentage of valid code on both Java and Python datasets.
• Ablation shows both probabilistic and attention dualities contribute to CS gains, with attention duality providing larger gains when used alone or with probabilistic duality.
• Qualitative examples show smoother attention distributions and high semantic similarity to human-written outputs.