[论文解读] Age Progression/Regression by Conditional Adversarial Autoencoder

Zhifei Zhang, Yang Song+1 more|arXiv (Cornell University)|Feb 27, 2017

Face recognition and analysis参考文献 28被引用 98

一句话总结

本論文提出了一種條件對抗自編碼器（CAAE），在不需要成對樣本的情況下，通過學習個體特定的人臉流形並沿著年齡遍歷以保持身份，實現年齡的推進與回退。

ABSTRACT

"If I provide you a face image of mine (without telling you the actual age when I took the picture) and a large amount of face images that I crawled (containing labeled faces of different ages but not necessarily paired), can you show me what I would look like when I am 80 or what I was like when I was 5?" The answer is probably a "No." Most existing face aging works attempt to learn the transformation between age groups and thus would require the paired samples as well as the labeled query image. In this paper, we look at the problem from a generative modeling perspective such that no paired samples is required. In addition, given an unlabeled image, the generative model can directly produce the image with desired age attribute. We propose a conditional adversarial autoencoder (CAAE) that learns a face manifold, traversing on which smooth age progression and regression can be realized simultaneously. In CAAE, the face is first mapped to a latent vector through a convolutional encoder, and then the vector is projected to the face manifold conditional on age through a deconvolutional generator. The latent vector preserves personalized face features (i.e., personality) and the age condition controls progression vs. regression. Two adversarial networks are imposed on the encoder and generator, respectively, forcing to generate more photo-realistic faces. Experimental results demonstrate the appealing performance and flexibility of the proposed framework by comparing with the state-of-the-art and ground truth.

研究动机与目标

• 在查詢圖像上不需要成對數據或年齡標籤的情況下，推動/實現年齡推進與回退。
• 學習一個臉部流形，使老化過程可以平滑遍歷，同時保留個體身份。
• 將個性（潛在變量 z）與年齡（標籤 l）解耦，以實現靈活的雙向老化。
• 確保生成的臉部是照片級現實感，並且對姿態、表情和遮擋具有魯棒性。

提出的方法

• 使用編碼器 E 將臉部編碼為保留個性的潛在向量 z。
• 以 G(z,l) 產生
  06x 從 z 和 l 作為 G(z,l)。
• 設置 z 判別器 Dz，以強制 z 的均勻先驗分佈，實現平滑的老化轉換。
• 設置圖像判別器 Dimg，確保生成的人臉具有照片級真實感並符合年齡特徵。
• 在輸入 x 與生成的 dx（帶有老化標籤 l）之間最小化重建損失，以及用於降低伪影的 TV 損失。
• 使用對抗目標以及 L2/TV 損失，交替更新 E、G、Dz 和 Dimg 進行訓練。

Figure 1: We assume the face images lie on a manifold ( $\mathcal{M}$ ) , and images are clustered according to their ages and personality by a different direction. Given a query image, it will first projected to the manifold, and then after the smooth transformation on the manifold, the correspondi

实验结果

研究问题

• RQ1 是否可以學習一個臉部流形，使得在沒有成對數據的情況下進行年齡推進/回退？
• RQ2 將個性（z）與年齡（l）解耦是否能在保留身份的同時實現現實且雙向的老化？
• RQ3 雙判別器（對 z 和對生成圖像）是否提高真實性和年齡條件保真度？
• RQ4 在老化過程中，該方法對姿態、表情和遮擋的變化是否具有魯棒性？
• RQ5 模型是否能生成超出訓練分佈的合理年齡並保持身份？

主要发现

• CAAE 能在不同年齡生成照片級現實感的人臉，而不依賴成對樣本。
• 潛在向量 z 被鼓勵為均勻分佈，從而實現平滑的年齡遍歷。
• 對圖像的判別器有助於產生年齡適宜的高質量紋理，提升老化的真實感。
• 該框架在老化過程中保留個性，並對姿態、表情和遮擋具有魯棒性。
• 質性與用戶基礎評估表明，相較於前作具有更好的真實性和身份保留。

Figure 2: Illustration of traversing on the face manifold $\mathcal{M}$ . The input faces $x_{1}$ and $x_{2}$ are encoded to $z_{1}$ and $z_{2}$ by an encoder $E$ , which represents the personality. Concatenated by random age labels $l_{1}$ and $l_{2}$ , the latent vectors $[z_{1},l_{1}]$ and $[z_{2