[论文解读] The History Began from AlexNet: A Comprehensive Survey on Deep Learning Approaches

Md Zahangir Alom, Tarek M. Taha+7 more|arXiv (Cornell University)|Mar 3, 2018

Generative Adversarial Networks and Image Synthesis参考文献 76被引用 435

一句话总结

一份全面的综述，总结深度学习方法、架构、变体、应用、框架和基准测试，追踪自 AlexNet 以来在多个领域的发展。

ABSTRACT

Deep learning has demonstrated tremendous success in variety of application domains in the past few years. This new field of machine learning has been growing rapidly and applied in most of the application domains with some new modalities of applications, which helps to open new opportunity. There are different methods have been proposed on different category of learning approaches, which includes supervised, semi-supervised and un-supervised learning. The experimental results show state-of-the-art performance of deep learning over traditional machine learning approaches in the field of Image Processing, Computer Vision, Speech Recognition, Machine Translation, Art, Medical imaging, Medical information processing, Robotics and control, Bio-informatics, Natural Language Processing (NLP), Cyber security, and many more. This report presents a brief survey on development of DL approaches, including Deep Neural Network (DNN), Convolutional Neural Network (CNN), Recurrent Neural Network (RNN) including Long Short Term Memory (LSTM) and Gated Recurrent Units (GRU), Auto-Encoder (AE), Deep Belief Network (DBN), Generative Adversarial Network (GAN), and Deep Reinforcement Learning (DRL). In addition, we have included recent development of proposed advanced variant DL techniques based on the mentioned DL approaches. Furthermore, DL approaches have explored and evaluated in different application domains are also included in this survey. We have also comprised recently developed frameworks, SDKs, and benchmark datasets that are used for implementing and evaluating deep learning approaches. There are some surveys have published on Deep Learning in Neural Networks [1, 38] and a survey on RL [234]. However, those papers have not discussed the individual advanced techniques for training large scale deep learning models and the recently developed method of generative models [1].

研究动机与目标

• 综述包括 DNN、CNN、RNN (LSTM, GRU)、Auto-Encoder、Deep Belief Network、GAN 和 DRL 在内的深度学习方法的发展。
• 总结这些技术的最新进展及其变体及训练方法。
• 评审在图像处理、计算机视觉、NLP、医学、机器人、网络安全等领域的应用等。
• 提供用于实现和评估深度学习方法的框架、SDK 和基准数据集的概述。

提出的方法

• 按架构和学习范式对深度学习方法进行审查和分类。
• 总结大规模模型的显著变体和训练方法。
• 将 DL 方法映射到应用领域并评估其影响。
• 讨论在 DL 研究中使用的最近的框架、SDK 和基准数据集。

实验结果

研究问题

• RQ1 自 AlexNet 出现以来，哪些主要的深度学习架构及其变体被开发出来？
• RQ2 这些 DL 方法如何在不同领域和任务中应用？
• RQ3 哪些框架、SDK 和基准数据集促进了 DL 方法的实现和评估？
• RQ4 在训练大规模深度学习模型和生成式方法方面有哪些显著的趋势和进展？

主要发现

• 包括 CNNs、RNNs、AE、DBN、GANs 和 DRL 的 DL 方法在多个领域取得了比传统机器学习更先进的性能。
• 该综述整合了所述 DL 方法的最近发展和高级变体。
• 应用涵盖图像处理、计算机视觉、语音识别、NLP、医学成像、机器人学、生物信息学、网络安全等领域。
• 本文汇总了用于实现和评估 DL 方法的框架、SDK 和基准数据集。