Abstract
Memristor-based deep learning accelerators provide a promising solution to improve the energy efficiency of neuromorphic computing systems. However, the electrical properties and crossbar structure of memristors make these accelerators error-prone. In addition, due to the hardware constraints, the way to deploy neural network models on memristor crossbar arrays affects the computation parallelism and communication overheads. To enable reliable and energy-efficient memristor-based accelerators, a simulation platform is needed to precisely analyze the impact of non-ideal circuit/device properties on the inference accuracy and the influence of different deployment strategies on performance and energy consumption. In this paper, we propose a flexible simulation framework, DL-RSIM, to tackle this challenge. A rich set of reliability impact factors and deployment strategies are explored by DL-RSIM, and it can be incorporated with any deep learning neural networks implemented by TensorFlow. Using several representative convolutional neural networks as case studies, we show that DL-RSIM can guide chip designers to choose a reliability-friendly design option and energy-efficient deployment strategies and develop optimization techniques accordingly.
- [1] . 2012. Learning multiple layers of features from tiny images. University of Toronto (2012).Google Scholar
- [2] . 2017. CACTI 7: New tools for interconnect exploration in innovative off-chip memories. ACM Trans. Archit. Code Optim. 14, 2, Article
14 (2017), 25 pages.Google ScholarDigital Library
- [3] . 2016. Memristive Boltzmann machine: A hardware accelerator for combinatorial optimization and deep learning. In IEEE International Symposium on High Performance Computer Architecture (HPCA). 1–13.Google Scholar
- [4] . 2020. GENIEx: A Generalized Approach to Emulating Non-Ideality in Memristive Xbars using Neural Networks. 6 pages.Google Scholar
- [5] . 2018. NeuroSim: A circuit-level macro model for benchmarking neuro-inspired architectures in online learning. IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems (TCAD) 37, 12 (2018), 3067–3080.Google Scholar
Digital Library
- [6] . 2018. A 65nm 1Mb nonvolatile computing-in-memory ReRAM macro with sub-16ns multiply-and-accumulate for binary DNN AI edge processors. In IEEE International Solid - State Circuits Conference (ISSCC). 494–496.Google Scholar
Cross Ref
- [7] . 2016. PRIME: A novel processing-in-memory architecture for neural network computation in ReRAM-based main memory. In ACM/IEEE International Symposium on Computer Architecture (ISCA). 27–39.Google Scholar
- [8] 2009. ImageNet: A large-scale hierarchical image database. In IEEE Conference on Computer Vision and Pattern Recognition (CVPR). 248–255.Google Scholar
Cross Ref
- [9] . 2012. Neural acceleration for general-purpose approximate programs. In IEEE/ACM International Symposium on Microarchitecture (MICRO). 449–460.Google Scholar
- [10] . 2018. Making memristive neural network accelerators reliable. In IEEE International Symposium on High Performance Computer Architecture (HPCA). 52–65.Google Scholar
- [11] . 2015. A study of array resistance distribution and a novel operation algorithm for WOx ReRAM memory. In International Conference on Solid State Devices and Materials (SSDM). 1168–1169.Google Scholar
- [12] . 2016. Dot-product engine for neuromorphic computing: Programming 1T1M crossbar to accelerate matrix-vector multiplication. In ACM/IEEE Design Automation Conference (DAC). 1–6.Google Scholar
Digital Library
- [13] . 2019. FPSA: A full system stack solution for reconfigurable ReRAM-Based NN accelerator architecture. In International Conference on Architectural Support for Programming Languages and Operating Systems (ASPLOS). 733–747.Google Scholar
Digital Library
- [14] . 2011. GPUs and the future of parallel computing. IEEE Micro 31, 5 (2011), 7–17.Google Scholar
Digital Library
- [15] . 2012. ImageNet classification with deep convolutional neural networks. In Advances in Neural Information Processing Systems (NIPS).Google Scholar
Digital Library
- [16] . 2020. MemTorch: An open-source simulation framework for memristive deep learning systems. In IEEE International Symposium on Circuits and Systems (ISCAS). 1–5.Google Scholar
- [17] . 1998. Gradient-based learning applied to document recognition. Proc. IEEE 86, 11 (1998), 2278–2324.Google Scholar
- [18] . 2019. A system-level simulator for RRAM-based neuromorphic computing chips. ACM Trans. Archit. Code Optim. 15, 4, Article
64 (2019), 24 pages.Google Scholar - [19] . 2020. HitM: High-throughput ReRAM-based PIM for multi-modal neural networks. In IEEE/ACM International Conference On Computer Aided Design (ICCAD). 1–7.Google Scholar
Digital Library
- [20] . 2011. Analysis of power consumption and linearity in capacitive digital-to-analog converters used in successive approximation ADCs. IEEE Transactions on Circuits and Systems I: Regular Papers 58, 8 (2011), 1736–1748.Google Scholar
Cross Ref
- [21] . 2014. OverFeat: Integrated Recognition, Localization and Detection using Convolutional Networks.
arxiv:cs.CV/1312.6229 .Google Scholar - [22] . 2016. ISAAC: A convolutional neural network accelerator with in-situ analog arithmetic in crossbars. In ACM/IEEE International Symposium on Computer Architecture (ISCA). 14–26.Google Scholar
- [23] . 2017. A 462GOPs/J RRAM-based nonvolatile intelligent processor for energy harvesting IoE system featuring nonvolatile logics and processing-in-memory. In Symposium on VLSI Technology (VLSIT). T260–T261.Google Scholar
- [24] . 2014. Deep learning face representation from predicting 10,000 classes. In IEEE Conference on Computer Vision and Pattern Recognition (CVPR). 1891–1898.Google Scholar
Digital Library
- [25] . 2012. Metal-oxide RRAM. Proc. IEEE 100, 6 (2012), 1951–1970.Google Scholar
Cross Ref
- [26] . 2020. An architecture-level energy and area estimator for processing-in-memory accelerator designs. In IEEE International Symposium on Performance Analysis of Systems and Software (ISPASS). 116–118.Google Scholar
- [27] . 2018. MNSIM: Simulation platform for memristor-based neuromorphic computing system. IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems (TCAD) 37, 5 (2018), 1009–1022.Google Scholar
Index Terms
DL-RSIM: A Reliability and Deployment Strategy Simulation Framework for ReRAM-based CNN Accelerators
Recommendations
PRIME: a novel processing-in-memory architecture for neural network computation in ReRAM-based main memory
ISCA'16Processing-in-memory (PIM) is a promising solution to address the "memory wall" challenges for future computer systems. Prior proposed PIM architectures put additional computation logic in or near memory. The emerging metal-oxide resistive random access ...
Trained Biased Number Representation for ReRAM-Based Neural Network Accelerators
Special Issue on HALO for Energy-Constrained On-Chip Machine LearningRecent works have demonstrated the promise of using resistive random access memory (ReRAM) to perform neural network computations in memory. In particular, ReRAM-based crossbar structures can perform matrix-vector multiplication directly in the analog ...
Int-Monitor: a model triggered hardware trojan in deep learning accelerators
AbstractDeep learning accelerators have domain-specific architectures, this special memory hierarchy and working mode could bring about new crucial security vulnerabilities. Neural network reuse PE resources layer by layer, after a layer finished, ...






Comments