[論文レビュー] Deep Predictive Coding Network for Object Recognition

Haiguang Wen, Kuan Han+4 more|arXiv (Cornell University)|Feb 13, 2018

Cell Image Analysis Techniques参考文献 53被引用数 41

ひとこと要約

双方向の再帰ニューラルネットワークに基づく予測符号化が、再帰的なボトムアップおよびトップダウン処理を実行してオブジェクト認識を向上させ、計算サイクルが増加するにつれて standards データセット上で feedforward のみのベースラインを上回る。

ABSTRACT

Based on the predictive coding theory in neuroscience, we designed a bi-directional and recurrent neural net, namely deep predictive coding networks (PCN). It has feedforward, feedback, and recurrent connections. Feedback connections from a higher layer carry the prediction of its lower-layer representation; feedforward connections carry the prediction errors to its higher-layer. Given image input, PCN runs recursive cycles of bottom-up and top-down computation to update its internal representations and reduce the difference between bottom-up input and top-down prediction at every layer. After multiple cycles of recursive updating, the representation is used for image classification. With benchmark data (CIFAR-10/100, SVHN, and MNIST), PCN was found to always outperform its feedforward-only counterpart: a model without any mechanism for recurrent dynamics. Its performance tended to improve given more cycles of computation over time. In short, PCN reuses a single architecture to recursively run bottom-up and top-down processes. As a dynamical system, PCN can be unfolded to a feedforward model that becomes deeper and deeper over time, while refining it representation towards more accurate and definitive object recognition.

研究の動機と目的

• Predictive coding 理論を、フィードフォワード、フィードバック、および再帰的接続を備えた深層ネットワークに翻訳してオブジェクト認識を行う。
• 内部表現を recursive なボトムアップおよびトップダウンサイクルを通じて更新し、各層の予測誤差を最小化するモデルを開発する。
• 標準ベンチマーク（CIFAR-10/100、SVHN、MNIST）で評価し、feedforward のみのベースラインと比較する。
• 計算サイクル数を増やすと認識性能にどう影響するかを分析する。

提案手法

• higher-layer feedback が lower-layer 表現の予測を伝え、bottom-up 信号が上位層へ予測誤差を伝える、deep predictive coding network (PCN) と呼ばれる bi-directional、再帰的ニューラルネットワークを設計する。
• Iteratively run bottom-up and top-down processing cycles to reduce the difference between input and top-down predictions across layers.
• Unfold the dynamical system to a progressively deeper feedforward model over time as representations refine for object recognition.
• Empirically evaluate PCN on CIFAR-10/100, SVHN, and MNIST and compare against a non-recurrent feedforward counterpart.

実験結果

リサーチクエスチョン

• RQ1 Predictive-coding–based networks can outperform traditional feedforward networks on standard object recognition benchmarks?
• RQ2 How does the number of recursive computation cycles influence recognition accuracy on these benchmarks?
• RQ3 Do higher-layer feedback and recurrent connections contribute to more accurate internal representations for classification?

主な発見

• PCN outperforms its feedforward-only counterpart across benchmark datasets.
• Performance improves as more computation cycles are executed over time.
• The model reuses a single architecture to perform recursive bottom-up and top-down processing, refining representations for more accurate recognition.