[Paper Review] Structure Learning of Deep Networks via DNA Computing Algorithm

Guoqiang Zhong, Tao Li+2 more|arXiv (Cornell University)|Oct 25, 2018

DNA and Biological Computing17 references4 citations

TL;DR

This paper proposes a DNA computing-inspired algorithm to automatically learn high-performance deep neural network architectures with skip-connections. By encoding layers as short DNA strands and networks as long DNA strands via base pairing, the method explores a flexible architecture space, achieves 99.73% accuracy on MNIST and 95.10% on CIFAR-10, and demonstrates that early stopping effectively eliminates poor-performing models after few training epochs.

ABSTRACT

Convolutional Neural Network (CNN) has gained state-of-the-art results in many pattern recognition and computer vision tasks. However, most of the CNN structures are manually designed by experienced researchers. Therefore, auto- matically building high performance networks becomes an important problem. In this paper, we introduce the idea of using DNA computing algorithm to automatically learn high-performance architectures. In DNA computing algorithm, we use short DNA strands to represent layers and long DNA strands to represent overall networks. We found that most of the learned models perform similarly, and only those performing worse during the first runs of training will perform worse finally than others. The indicates that: 1) Using DNA computing algorithm to learn deep architectures is feasible; 2) Local minima should not be a problem of deep networks; 3) We can use early stop to kill the models with the bad performance just after several runs of training. In our experiments, an accuracy 99.73% was obtained on the MNIST data set and an accuracy 95.10% was obtained on the CIFAR-10 data set.

Motivation & Objective

• To address the challenge of manually designing high-performance convolutional neural network (CNN) architectures, which is time-consuming and requires expert knowledge.
• To explore a more flexible architecture search space that supports arbitrary skip-connections between any layers, overcoming limitations of fixed-stage or plain-stack encoding systems.
• To develop a novel encoding system using DNA strands to represent layers and networks, enabling parallel exploration of diverse architectures.
• To reduce training time and computational cost by applying early stopping based on early performance, eliminating poorly performing models quickly.

Proposed method

• Layers are encoded as short DNA strands containing layer type and skip-connection information, with each strand representing a single layer in a neural network.
• Complete network architectures are formed by combining short DNA strands into long strands via base pairing, simulating the assembly of neural network components.
• The DNA computing process generates a large pool of candidate architectures (long DNA strands) in parallel, mimicking biochemical reactions in a 'DNA soup'.
• Each candidate architecture is trained on a dataset, and performance on a validation set determines its fitness, with early stopping used to discard underperforming models after a few epochs.
• The search space is defined by architectures that include skip-connections, allowing flexible depth, number of pooling layers, and arbitrary inter-layer connections.
• The method is simulated computationally, with performance evaluated on MNIST and CIFAR-10 using data augmentation and standard training protocols.

Experimental results

Research questions

• RQ1 Can DNA computing be effectively used to explore and generate high-performing deep neural network architectures with skip-connections?
• RQ2 Does the proposed DNA-based encoding system allow for a more flexible and less constrained architecture search space compared to fixed-stage or plain-stack methods?
• RQ3 Can early stopping based on early training performance reliably identify and eliminate low-performing models with minimal impact on final accuracy?
• RQ4 To what extent do models in the DNA-computed search space exhibit similar performance, and does this suggest that local minima are not a major obstacle in training deep networks?
• RQ5 Can non-experts achieve competitive performance on benchmark datasets using this automated architecture search method?

Key findings

• The proposed DNA computing algorithm successfully generated high-performing models, achieving 99.73% test accuracy on the MNIST dataset.
• On the CIFAR-10 dataset, the best model reached 95.10% test accuracy, demonstrating competitive performance against state-of-the-art methods.
• Most models in the search space performed similarly, with only a small fraction showing consistently poor performance from the start.
• Early stopping after 10, 20, or 45 epochs—based on predefined accuracy thresholds—effectively eliminated underperforming models with minimal impact on overall accuracy.
• The results suggest that local minima are not a significant issue in training deep networks when using this search strategy.
• The method enables high accuracy with fewer training iterations, significantly reducing computational cost while maintaining model performance.