[Paper Review] Non-autoregressive Model for Full-line Code Completion

Fang Liu, Zhiyi Fu+4 more|arXiv (Cornell University)|Apr 21, 2022

Software Engineering Research4 citations

TL;DR

This paper proposes SANAR, a syntax-aware non-autoregressive model for full-line code completion that enables parallel token generation, achieving up to 9× faster inference than autoregressive models while outperforming both autoregressive and non-autoregressive baselines on Python and Java datasets. The model uses an adaptive, syntax-guided sampling strategy during training to improve performance without relying on left-to-right autoregressive generation.

ABSTRACT

Code completion tools are frequently used by software developers to accelerate software development by suggesting the following code elements. Completing a sequence of code tokens (e.g., a full line of code) has been proved more efficient than predicting a single token at a time. To complete the code sequence, researchers are employing AutoRegressive (AR) decoders to generate tokens in a left-to-right, token-by-token fashion. Consequently, the prediction of the next token depends on all previously generated tokens, which leads to high latency in inference. To improve the efficiency and accuracy of full-line code completion, in this paper, we propose a Non-AutoRegressive (NAR) model for code completion boosted by a syntax-aware sampling strategy. Our experimental results on two widely used datasets suggest that our model outperforms both AR and NAR baselines on full-line code completion, and it is faster than the AR model with up to 9 times speed-up.

Motivation & Objective

• To address the high inference latency of autoregressive models in full-line code completion by enabling parallel token generation.
• To investigate whether code token dependencies are weak enough to support non-autoregressive generation, unlike in natural language generation.
• To design a training strategy that improves non-autoregressive code completion performance by dynamically sampling tokens based on syntax type and difficulty.
• To demonstrate that non-autoregressive models can be effectively applied to code completion, achieving both higher speed and better accuracy than existing approaches.

Proposed method

• Proposes a non-autoregressive Transformer-based model (SANAR) that generates all target tokens in parallel, bypassing autoregressive dependency.
• Introduces a syntax-aware sampling strategy that dynamically selects tokens for masking and regeneration during training based on token type and prediction confidence.
• Uses a differentiable, adaptive sampling mechanism that reduces reliance on left-to-right autoregressive inductive bias, improving training stability and performance.
• Employs a single-pass inference process where the entire code line is generated in one forward pass, significantly reducing latency.
• Applies a conditional independence factorization during training: $\mathcal{L}_{NAR} = \sum_{t=1}^{T} \log p(y_t|X;\theta)$, enabling parallel decoding.
• Adapts iterative refinement techniques from NMT, such as CMLM, to iteratively improve predictions on low-confidence tokens during training.

Experimental results

Research questions

• RQ1 Is the dependency among code tokens weak enough to allow non-autoregressive generation, unlike in natural language generation?
• RQ2 Can a non-autoregressive model achieve competitive or better performance than autoregressive models in full-line code completion?
• RQ3 Does a syntax-aware, adaptive sampling strategy during training improve the performance of non-autoregressive code completion models?
• RQ4 How does the token repetition ratio in non-autoregressive code completion compare to that in neural machine translation, and does it affect model quality?
• RQ5 Can non-autoregressive models achieve significant speedups without sacrificing accuracy in code completion tasks?

Key findings

• SANAR outperforms both autoregressive and non-autoregressive baselines on full-line code completion for Python and Java, demonstrating superior accuracy.
• The model achieves up to 9× faster inference than autoregressive models, with an average speed-up of 5–6×, especially benefiting long code lines.
• The syntax-aware sampling strategy improves performance over random masking (GLAT) and reduces performance degradation when increasing sampling ratio, indicating effective token selection.
• The token repetition ratio in SANAR is not significantly higher than in autoregressive models, suggesting weaker inter-token dependencies in code than in NMT.
• Empirical analysis confirms that code token dependencies are weaker than in natural language, supporting the feasibility of non-autoregressive code generation.
• The model achieves comparable performance when generating from right-to-left or left-to-right, indicating that left-to-right order is not essential for code completion.