[Paper Review] SwitchX: Gmin-Gmax Switching for Energy-Efficient and Robust Implementation of Binary Neural Networks on ReRAM Xbars

Abhiroop Bhattacharjee, Priyadarshini Panda|arXiv (Cornell University)|Nov 30, 2020

Advanced Memory and Neural Computing4 citations

TL;DR

SwitchX proposes a novel weight-mapping strategy for binarized neural networks (BNNs) on ReRAM crossbars, favoring high-resistance state (HRS) synapses over low-resistance state (LRS) ones to reduce the impact of crossbar non-idealities. This approach enhances adversarial robustness by up to 16% and improves energy efficiency by up to 22%, especially when combined with state-aware training.

ABSTRACT

Memristive crossbars can efficiently implement Binarized Neural Networks (BNNs) wherein the weights are stored in high-resistance states (HRS) and low-resistance states (LRS) of the synapses. We propose SwitchX mapping of BNN weights onto ReRAM crossbars such that the impact of crossbar non-idealities, that lead to degradation in computational accuracy, are minimized. Essentially, SwitchX maps the binary weights in such manner that a crossbar instance comprises of more HRS than LRS synapses. We find BNNs mapped onto crossbars with SwitchX to exhibit better robustness against adversarial attacks than the standard crossbar-mapped BNNs, the baseline. Finally, we combine SwitchX with state-aware training (that further increases the feasibility of HRS states during weight mapping) to boost the robustness of a BNN on hardware. We find that this approach yields stronger defense against adversarial attacks than adversarial training, a state-of-the-art software defense. We perform experiments on a VGG16 BNN with benchmark datasets (CIFAR-10, CIFAR-100 & TinyImagenet) and use Fast Gradient Sign Method and Projected Gradient Descent adversarial attacks. We show that SwitchX combined with state-aware training can yield upto ~35% improvements in clean accuracy and ~6-16% in adversarial accuracies against conventional BNNs. Furthermore, an important by-product of SwitchX mapping is increased crossbar power savings, owing to an increased proportion of HRS synapses, that is furthered with state-aware training. We obtain upto ~21-22% savings in crossbar power consumption for state-aware trained BNN mapped via SwitchX on 16x16 & 32x32 crossbars using the CIFAR-10 & CIFAR-100 datasets.

Motivation & Objective

• Address the vulnerability of BNNs to adversarial attacks when implemented on non-ideal ReRAM crossbars.
• Mitigate accuracy degradation caused by crossbar non-idealities such as interconnect parasitics and device variations.
• Improve energy efficiency by increasing the proportion of HRS synapses, which consume less power.
• Enhance adversarial robustness through hardware-aware weight mapping without relying solely on software-based defenses.
• Integrate state-aware training with SwitchX to further increase HRS feasibility and improve both accuracy and robustness.

Proposed method

• Propose SwitchX mapping, which reorganizes BNN weights so that crossbars contain more HRS than LRS synapses.
• Leverage the fact that HRS synapses are less sensitive to resistive non-idealities, reducing error propagation during MAC operations.
• Use a modified weight permutation strategy to maximize HRS utilization during crossbar mapping.
• Combine SwitchX with state-aware training, which adjusts the training process to favor HRS states during weight quantization.
• Model crossbar non-idealities including interconnect parasitics, device variations, and sensing/driver resistances using a circuit-level simulation framework.
• Evaluate performance using standard BNNs (VGG16) on CIFAR-10, CIFAR-100, and TinyImageNet with FGSM and PGD adversarial attacks.

Experimental results

Research questions

• RQ1 Can a hardware-aware weight-mapping strategy reduce the impact of ReRAM crossbar non-idealities on BNN inference accuracy?
• RQ2 Does increasing the proportion of HRS synapses in ReRAM crossbars improve adversarial robustness of BNNs?
• RQ3 Can SwitchX mapping, combined with state-aware training, enhance both energy efficiency and robustness simultaneously?
• RQ4 How does SwitchX compare to state-of-the-art software-based adversarial training in terms of robustness under hardware non-idealities?
• RQ5 To what extent do crossbar power savings scale with HRS-LRS imbalance in the weight distribution?

Key findings

• SwitchX mapping reduces accuracy degradation from crossbar non-idealities by favoring HRS synapses, which are less affected by resistive variations.
• SwitchX combined with state-aware training improves clean accuracy by up to ~35% and adversarial accuracy by ~6–16% over conventional BNNs on CIFAR-10 and CIFAR-100.
• The approach achieves up to ~22% crossbar power savings on 32×32 crossbars and ~21% on 16×16 crossbars when using state-aware training.
• Layer-wise power analysis shows significant savings, especially between conv7 and conv8, when HRS-LRS distribution becomes more non-uniform.
• SwitchX outperforms adversarial training in robustness against white-box attacks, even under resistive non-idealities that degrade performance.
• The method provides a dual benefit: improved energy efficiency and enhanced adversarial robustness, without compromising model accuracy.