[Paper Review] Stable Style Transformer: Delete and Generate Approach with Encoder-Decoder for Text Style Transfer

Joosung Lee|arXiv (Cornell University)|May 25, 2020

Natural Language Processing Techniques8 citations

TL;DR

This paper proposes Stable Style Transformer (SST), a two-stage text style transfer method that first deletes style-dependent attribute markers using a classifier-based approach and then generates style-transferred sentences via a Transformer encoder-decoder. The method achieves stable performance across all automatic evaluation metrics—content, fluency, style accuracy, and semantic similarity—outperforming unstable baselines in human evaluation, making it suitable for real-world deployment.

ABSTRACT

Text style transfer is the task that generates a sentence by preserving the content of the input sentence and transferring the style. Most existing studies are progressing on non-parallel datasets because parallel datasets are limited and hard to construct. In this work, we introduce a method that follows two stages in non-parallel datasets. The first stage is to delete attribute markers of a sentence directly through a classifier. The second stage is to generate a transferred sentence by combining the content tokens and the target style. We experiment on two benchmark datasets and evaluate context, style, fluency, and semantic. It is difficult to select the best system using only these automatic metrics, but it is possible to select stable systems. We consider only robust systems in all automatic evaluation metrics to be the minimum conditions that can be used in real applications. Many previous systems are difficult to use in certain situations because performance is significantly lower in several evaluation metrics. However, our system is stable in all automatic evaluation metrics and has results comparable to other models. Also, we compare the performance results of our system and the unstable system through human evaluation. Our code and data are available at the link (https://github.com/rungjoo/Stable-Style-Transformer).

Motivation & Objective

• To address the instability of existing text style transfer models across automatic evaluation metrics.
• To develop a method that decouples style and content without relying on adversarial training or disentangled latent representations.
• To enable intuitive, model-agnostic deletion of attribute markers using classifier probability shifts.
• To ensure stable performance across multiple evaluation metrics, making the system suitable for real-world applications.
• To explore latent space walking for improved style control in generated sentences.

Proposed method

• A two-stage framework: first, a pre-trained classifier identifies and deletes style attribute markers based on significant changes in prediction probability when tokens are masked.
• The method is model-agnostic and does not require attention weights or frequency-based heuristics for marker detection.
• Content tokens from the deleted sentence are combined with a target style embedding to generate the transferred sentence using a Transformer-based encoder-decoder.
• The model is trained with a style loss to improve style transfer accuracy without degrading content preservation.
• Hyperparameters α and β control the trade-off between content preservation and style transfer, enabling adjustable performance.
• Latent space walking is explored by interpolating style vectors to generate sentences with intermediate style attributes.

Experimental results

Research questions

• RQ1 Can a non-adversarial, non-disentangled method achieve stable performance across multiple automatic evaluation metrics in text style transfer?
• RQ2 How does classifier-based attribute marker deletion compare to frequency-based or attention-weighted methods in terms of stability and control?
• RQ3 To what extent can style control be improved through latent space interpolation in the style vector space?
• RQ4 How do hyperparameters α and β affect the trade-off between content preservation and style transfer?
• RQ5 Can stable automatic evaluation results reliably predict human preference in text style transfer?

Key findings

• SST achieves stable performance across all automatic evaluation metrics—content (G-BLEU: 19.11), fluency (t-PPL: 306.65), style accuracy (82.2%), and semantic similarity (BERTscore: 89.96)—on the Yelp dataset.
• The inclusion of style loss improves style accuracy by 4 points and enhances data-fluency, though it slightly reduces G-BLEU, indicating a trade-off in optimization.
• Human evaluation confirms that SST is more reliable than unstable baselines, with consistent preference across multiple metrics.
• The model’s performance is robust across both Yelp and Amazon datasets, demonstrating generalization to different domains and style transfer directions.
• Latent space walking shows that interpolating style vectors can generate intermediate-style sentences, though not always neutral, suggesting room for improvement in vector space alignment.
• The classifier-based deletion method enables intuitive, model-agnostic attribute marker identification without requiring predefined dictionaries or attention mechanisms.