[论文解读] Scene Text Synthesis for Efficient and Effective Deep Network Training

Changgong Zhang, Fangneng Zhan+2 more|arXiv (Cornell University)|Jan 26, 2019

Handwritten Text Recognition Techniques参考文献 51被引用 33

一句话总结

本论文提出一种前景–背景嵌入技术，用于合成带注释的场景文本训练图像，包含两个组件——上下文感知的语义一致性与和谐外观自适应——在场景文本检测与识别上评估，性能与真实图像相当或更好。

ABSTRACT

A large amount of annotated training images is critical for training accurate and robust deep network models but the collection of a large amount of annotated training images is often time-consuming and costly. Image synthesis alleviates this constraint by generating annotated training images automatically by machines which has attracted increasing interest in the recent deep learning research. We develop an innovative image synthesis technique that composes annotated training images by realistically embedding foreground objects of interest (OOI) into background images. The proposed technique consists of two key components that in principle boost the usefulness of the synthesized images in deep network training. The first is context-aware semantic coherence which ensures that the OOI are placed around semantically coherent regions within the background image. The second is harmonious appearance adaptation which ensures that the embedded OOI are agreeable to the surrounding background from both geometry alignment and appearance realism. The proposed technique has been evaluated over two related but very different computer vision challenges, namely, scene text detection and scene text recognition. Experiments over a number of public datasets demonstrate the effectiveness of our proposed image synthesis technique - the use of our synthesized images in deep network training is capable of achieving similar or even better scene text detection and scene text recognition performance as compared with using real images.

研究动机与目标

• 通过生成带注释的合成图像降低训练深度网络的标注成本。
• 开发一个合成流水线，在语义连贯的上下文中放置前景对象。
• 确保几何与外观的真实感，以提高合成数据对训练的有效性。
• 在场景文本检测与识别基准上评估该技术，以与真实图像训练进行比较。

提出的方法

• 在保持语义连贯性的前提下，将感兴趣的前景对象嵌入背景图像。
• 强制上下文感知放置，使前景对象与背景中有语义意义的区域对齐。
• 应用和谐的外观自适应，实现前景对象与背景之间的几何对齐和外观真实感。
• 生成适合深度网络训练的带注释的合成训练图像。
• 评估合成技术对下游场景文本检测与识别任务的影响。

实验结果

研究问题

• RQ1 使用所提出方法训练的合成图像能否在文本检测与识别上达到与真实图像相当的性能？
• RQ2 上下文感知的语义一致性是否提升场景文本任务的训练效果？
• RQ3 和谐的外观自适应是否提升嵌入对象的真实感和深度学习的有用性？
• RQ4 合成图像在训练鲁棒场景文本模型方面与真实图像相比如何？

主要发现

• 使用所提出技术合成的图像对训练深度网络有效。
• 在场景文本检测与识别方面，与使用真实图像相比，该方法达到相似或更好的性能。
• 上下文感知的一致性和外观自适应有助于提升合成数据的训练有效性。
• 实验验证了现实前景嵌入对提升模型鲁棒性的价值。