Guest Editors
Daniele Ielmini Politecnico di Milano, Italy
Tuo-Hung Hou National Chiao Tung University, Taiwan
Jeehwan Kim MIT, USA
Ming Liu IMECAS, China
Manan Suri IIT Delhi, India
Scope
Neuromorphic engineering aims at developing computing systems that mimic the same structure and the same information processing as the human brain. The brain structure is totally different from the traditional von Neumann architecture of the conventional computers, in that neurons and synapses share the same location in the biological network. Also, the information is exchanged via asynchronous spikes, instead of synchronous bit strings in digital computers. Reproducing the brain structure and processes would give rise to cognitive computers with outstanding energy efficiency and the ability to learn and adapt to a changing environment. Achieving this goal, however, will be possible only by developing a novel class of neuromorphic devices that can reproduce the biological brain processes by physical processes. This include electron devices relying of a broad range of materials, such as metal oxides, chalcogenides, ferroelectric and ferromagnetic materials. Various device structures have been explored, including thin films, nanowires/nanodots and van der Waals heterostuctures. Various brain-inspired architectures and algorithms have been implemented in hardware. Still, there is no consensus about the materials and devices that will constitute tomorrow's neuromorphic computing technology.
The scope of this Special Issue of Semiconductor Science and Technology is to provide an overview of the status and outlook of neuromorphic devices and applications. The most recent advances in materials, devices, circuits and algorithms will be collected and the major challenges toward the development of efficient neuromorphic hardware will be addressed. The Special Issue aims at becoming a top reference for the community of neuromorphic students, researchers and engineers.
How to submit
Either go to mc04.manuscriptcentral.com/sst-iop or click on "Submit an article" on the right-hand side of this page, and select "Special Issue Article" as the article type, then "Special issue on Neuromorphic Devices and Applications".
This special issue will include papers submitted to Neuromorphic Computing and Engineering (NCE) and you are welcome to contribute to this journal also. If you wish to contribute to this journal instead, please submit your manuscript here, identifying your contribution to this special issue in the cover letter. The journal is fully Open Access, and all article fees will be waived for 2021 submissions.
IOP Publishing has just launched NCE, the world's first journal dedicated exclusively to all aspects of neuromorphic computing and engineering. At the interface of physics, electrical engineering, materials science, bioscience, chemistry, mathematics and computer science, it will be truly inter- and multidisciplinary.
Important dates and deadlines
Please note the submission deadline is now 31st July 2021, which has been extended to allow authors more time to write their high quality papers.
Papers
Open access
Memristive devices based on single ZnO nanowires—from material synthesis to neuromorphic functionalities
G Milano et al 2022 Semicond. Sci. Technol. 37 034002
Memristive and resistive switching devices are considered promising building blocks for the realization of artificial neural networks and neuromorphic systems. Besides conventional top-down memristive devices based on thin films, resistive switching devices based on nanowires (NWs) have attracted great attention, not only for the possibility of going beyond current scaling limitations of the top-down approach, but also as model systems for the localization and investigation of the physical mechanism of switching. This work reports on the fabrication of memristive devices based on ZnO NWs, from NW synthesis to single NW-based memristive cell fabrication and characterization. The bottom-up synthesis of ZnO NWs was performed by low-pressure chemical vapor deposition according to a self-seeding vapor-solid (VS) mechanism on a Pt substrate over large scale (∼cm2), without the requirement of previous seed deposition. The grown ZnO NWs are single crystalline with wurtzite crystal structure and are vertically aligned respect to the growth substrate. Single NWs were then contacted by means of asymmetric contacts, with an electrochemically active and an electrochemically inert electrode, to form NW-based electrochemical metallization memory cells that show reproducible resistive switching behaviour and neuromorphic functionalities including short-term synaptic plasticity and paired pulse facilitation. Besides representing building blocks for NW-based memristive and neuromorphic systems, these single crystalline devices can be exploited as model systems to study physicochemical processing underlaying memristive functionalities thanks to the high localization of switching events on the ZnO crystalline surface.
Improving the accuracy and robustness of RRAM-based in-memory computing against RRAM hardware noise and adversarial attacks
Sai Kiran Cherupally et al 2022 Semicond. Sci. Technol. 37 034001
We present a novel deep neural network (DNN) training scheme and resistive RAM (RRAM) in-memory computing (IMC) hardware evaluation towards achieving high accuracy against RRAM device/array variations and enhanced robustness against adversarial input attacks. We present improved IMC inference accuracy results evaluated on state-of-the-art DNNs including ResNet-18, AlexNet, and VGG with binary, 2-bit, and 4-bit activation/weight precision for the CIFAR-10 dataset. These DNNs are evaluated with measured noise data obtained from three different RRAM-based IMC prototype chips. Across these various DNNs and IMC chip measurements, we show that our proposed hardware noise-aware DNN training consistently improves DNN inference accuracy for actual IMC hardware, up to 8% accuracy improvement for the CIFAR-10 dataset. We also analyze the impact of our proposed noise injection scheme on the adversarial robustness of ResNet-18 DNNs with 1-bit, 2-bit, and 4-bit activation/weight precision. Our results show up to 6% improvement in the robustness to black-box adversarial input attacks.
STDP implementation using multi-state spin−orbit torque synapse
Hamdam Ghanatian et al 2022 Semicond. Sci. Technol. 37 024004
The abundance of data to be processed calls for new computing paradigms, which could accommodate, and directly map artificial neural network architectures at the hardware level. Neuromorphic computing has emerged as a potential solution, proposing the implementation of artificial neurons and synapses on physical substrates. Conventionally, neuromorphic platforms are deployed in complementary metal-oxide-semiconductor technology. However, such implementations still cannot compete with the highly energy-efficient performance of the brain. This calls for novel ultra-low-power nano-scale devices with the possibility of upscaling for the implementation of complex networks. In this paper, a multi-state spin−orbit torque (SOT) synapse based on the three-terminal perpendicular anisotropy magnetic tunnel junction (P-MTJ) is proposed. In this implementation, P-MTJs use common heavy metals but with different cross-section areas, thereby creating multiple states that can be harnessed to implement synapses. The proposed multi-state SOT synapse can solve the state-limited issue of spin-based synapses. Moreover, it is shown that the proposed multi-state SOT synapse can be programmed to reproduce the spike-timing-dependent plasticity learning algorithm.
Dynamic resistive switching devices for neuromorphic computing
Yuting Wu et al 2022 Semicond. Sci. Technol. 37 024003
Neuromorphic systems that can emulate the structure and the operations of biological neural circuits have long been viewed as a promising hardware solution to meet the ever-growing demands of big-data analysis and AI tasks. Recent studies on resistive switching or memristive devices have suggested such devices may form the building blocks of biorealistic neuromorphic systems. In a memristive device, the conductance is determined by a set of internal state variables, allowing the device to exhibit rich dynamics arising from the interplay between different physical processes. Not only can these devices be used for compute-in-memory architectures to tackle the von Neumann bottleneck, the switching dynamics of the devices can also be used to directly process temporal data in a biofaithful fashion. In this review, we analyze the physical mechanisms that govern the dynamic switching behaviors and highlight how these properties can be utilized to efficiently implement synaptic and neuronal functions. Prototype systems that have been used in machine learning and brain-inspired network implementations will be covered, followed with discussions on the challenges for large scale implementations and opportunities for building bio-inspired, highly complex computing systems.
Dual-mode dendritic devices enhanced neural network based on electrolyte gated transistors
Zhaokun Jing et al 2022 Semicond. Sci. Technol. 37 024002
As a fundamental component of biological neurons, dendrites have been proven to have crucial effects in neuronal activities. Single neurons with dendrite structures show high signal processing capability that is analogous to a multilayer perceptron (MLP), whereas oversimplified point neuron models are still prevalent in artificial intelligence algorithms and neuromorphic systems and fundamentally limit their efficiency and functionality of the systems constructed. In this study, we propose a dual-mode dendritic device based on electrolyte gated transistor, which can be operated to generate both supralinear and sublinear current–voltage responses when receiving input voltage pulses. We propose and demonstrate that the dual-mode dendritic devices can be used as a dendritic processing block between weight matrices and output neurons so as to dramatically enhance the expression ability of the neural networks. A dual-mode dendrites-enhanced neural network is therefore constructed with only two trainable parameters in the second layer, thus achieving 1000× reduction in the amount of second layer parameter compared to MLP. After training by back propagation, the network reaches 90.1% accuracy in MNIST handwritten digits classification, showing advantage of the present dual-mode dendritic devices in building highly efficient neuromorphic computing.
Compact model of retention characteristics of ferroelectric FinFET synapse with MFIS gate stack
Md Aftab Baig et al 2022 Semicond. Sci. Technol. 37 024001
In this paper, multiple-fin n- and p-channel HfZrO2 ferroelectric-FinFET devices are manufactured using a gate first process with post metalization annealing. The device transfer characteristics upon program and erase operations are measured and modeled. The drift in the transfer characteristics due to depolarization field and charge injection are captured using the shift in the threshold voltage along with time-dependent modeling of vertical field dependent mobility degradation parameters to develop a physical, computationally efficient, and accurate retention model for ferroelectric-FinFET devices. The modeled conductance is incorporated into deep neural network simulation platform CIMulator to analyze the role of conductance drift due to retention degradation, as well as the importance of the gap between high and low conductance states in improving the image recognition accuracy of neural networks.
OxRAM + OTS optimization for binarized neural network hardware implementation
J Minguet Lopez et al 2022 Semicond. Sci. Technol. 37 014001
Low-power memristive devices embedded on graphics or central processing units logic core are a very promising non-von-Neumann approach to improve significantly the speed and power consumption of deep learning accelerators, enhancing their deployment on embedded systems. Among various non-ideal emerging neuromorphic memory devices, synaptic weight hardware implementation using resistive random-access memories (RRAMs) within 1T1R architectures promises high performance on low precision binarized neural networks (BNN). Taking advantage of the RRAM capabilities and allowing to substantially improve the density thanks to the ovonic threshold selector (OTS) selector, this work proposes to replace the standard 1T1R architecture with a denser 1S1R crossbar system, where an HfO2-based resistive oxide memory (OxRAM) is co-integrated with a Ge-Se-Sb-N-based OTS. In this context, an extensive experimental study is performed to optimize the 1S1R stack and programming conditions for extended read window margin and endurance characteristics. Focusing on the standard machine learning MNIST image recognition task, we perform offline training simulations in order to define the constraints on the devices during the training process. A very promising bit error rate of ∼10−3 is demonstrated together with 1S1R 104 error-free programming endurance characteristics, fulfilling the requirements for the application of interest. Based on this simulation and experimental study, BNN figures of merit (system footprint, number of weight updates, accuracy, inference speed, electrical consumption per image classification and tolerance to errors) are optimized by engineering the number of learnable parameters of the system. Altogether, an inherent BNN resilience to 1S1R parasitic bit errors is demonstrated.
A three-bit-per-cell via-type resistive random access memory gated metal-oxide semiconductor field-effect transistor non-volatile memory with the FORMing-free characteristic
E Ray Hsieh et al 2021 Semicond. Sci. Technol. 36 124002
We present an embedded memory for possible neuromorphic computing applications using a via-type resistive random access memory (RRAM) gated metal-oxide semiconductor field-effect transistor (MOSFET). By this arrangement, the threshold voltage (Vth) of the MOSFET is modulated by the resistance of the via-type RRAM. When the resistance of the via-type RRAM is in a high-resistance state (HRS), the word-line voltage (VWL) is consumed mostly across the via-type RRAM and little is left on the gate dielectric layer of the MOSFET; the Vth of the MOSFET is boosted. In contrast, when the resistance of the via-type RRAM is in a low-resistance state (LRS), with the resistance value much smaller than that of the gate dielectric of the MOSFET, the VWL will be dropped majorly on the gate dielectric of the MOSFET, and the Vth of the MOSFET will be much reduced than that of the MOSFET gated by the via-type RRAM in a HRS. The experimental results show that, in a direct-current mode, the memory window achieves 1 V between a LRS and a HRS of the via-type RRAM gated MOSFET. In an alternating current mode, the LRS can be SET at 10 nanoseconds; the HRS can be RESET at 5 nanoseconds. Furthermore, three-bit-per-cell operation of the via-type RRAM gated MOSFET is demonstrated. The eight conductance states are distributed evenly between 100 micro- and 100 picosiemens with almost isometric gaps in between. The endurance tests were executed for eight conductance states with one million cycles for four pairs. Finally, the retention tests of eight states were kept under 125 °C for one month.
1/f noise in amorphous Sb2Te3 for energy-efficient stochastic synapses in neuromorphic computing
Deokyoung Kang et al 2021 Semicond. Sci. Technol. 36 124001
Recent studies on neuromorphic computing have used stochastic synapses to implement power-efficient stochastic computing inspired by unreliable connections between neurons, such as the blank-out noise in the Synaptic Sampling Machine. In this paper, we propose to generate stochasticity by exploiting intrinsic
noise in phase change memory (PCM) as a synaptic device, negating additional stochastic devices and circuits that deteriorate the synaptic footprints and power consumption. As existing models are limited to demonstrating the spectral density of
noise, we devised a new model based on two-level state theory with a cutoff frequency, resulting in accurate quantification of the finite normalized variance of current (
) of PCM and its dependence on the volume of the cell and the frequency range in which the measurement is taken. We experimentally verified our model by measuring
noise of a phase change bridge cell with an as-deposited amorphous Sb2Te3 as the phase change material. We further analyzed whether the noise in PCM can implement a restricted Boltzmann machine (RBM), in which stochasticity plays a key role, to allow efficient neuromorphic computing. We devised and simulated a spiking neural network (SNN)-based RBM system based on a 832 × 832 PCM synapse array with the intrinsic noise model. As we modeled the normalized synaptic noise with a normal distribution,
and optimal standard deviation between 0.01 and 0.05, the on-chip learning and inference test result showed comparable MNIST accuracy and ∼60 times larger estimated energy efficiency than that of the SNN-based RBM, the stochasticity of which is implemented with power-consuming random walk neuron circuits, demonstrating that the intrinsic
noise of PCM is not a nuisance but an asset to implement efficient neuromorphic computing systems.
Analysis and mitigation of parasitic resistance effects for analog in-memory neural network acceleration
T Patrick Xiao et al 2021 Semicond. Sci. Technol. 36 114004
To support the increasing demands for efficient deep neural network processing, accelerators based on analog in-memory computation of matrix multiplication have recently gained significant attention for reducing the energy of neural network inference. However, analog processing within memory arrays must contend with the issue of parasitic voltage drops across the metal interconnects, which distort the results of the computation and limit the array size. This work analyzes how parasitic resistance affects the end-to-end inference accuracy of state-of-the-art convolutional neural networks, and comprehensively studies how various design decisions at the device, circuit, architecture, and algorithm levels affect the system’s sensitivity to parasitic resistance effects. A set of guidelines are provided for how to design analog accelerator hardware that is intrinsically robust to parasitic resistance, without any explicit compensation or re-training of the network parameters.
Experimental measurement of ungated channel region conductance in a multi-terminal, metal oxide-based ECRAM
Hyunjeong Kwak et al 2021 Semicond. Sci. Technol. 36 114002
Due to the rapid progress of artificial intelligence technology based on neural networks, the amount of required computation has been increasing dramatically. To keep up with the ever-increasing demand, novel analog neuromorphic computing architectures have been intensively studied, where cross-point arrays of resistive memory devices are utilized for high-speed and power-efficient computation. Among various synaptic memory device candidates, a metal oxide-based electrochemical random-access memory (MO-ECRAM) has been attractive due to its complementary metal-oxide-semiconductor-compatibility and superior programmability. In this work, we fabricate a WO3-based MO-ECRAM with multiple terminals and characterize the conductance modulation in the channel regions with and without the gate stack. While the gated region conductance shows a high on/off ratio, the ungated region conductance displays weak modulation with a near-unity on/off ratio. Based on our experimental observation, we propose a lithographical technique to intentionally uncover the channel area and utilize the ungated area’s resistance to limit the maximum conductance of each cross-point element at the individual device level. We conduct a neural network training simulation for MNIST dataset and show that this technique can guarantee robust large array operations for high-performance neural network computation.
Exploiting the electrothermal timescale in PrMnO3 RRAM for a compact, clock-less neuron exhibiting biological spiking patterns
Omkar Phadke et al 2021 Semicond. Sci. Technol. 36 114001
Spiking neural networks (SNNs) are gaining widespread momentum in the field of neuromorphic computing. These network systems integrated with neurons and synapses provide computational efficiency by mimicking the human brain. It is desired to incorporate the biological neuronal dynamics, including complex spiking patterns which represent diverse brain activities within the neural networks. Earlier hardware realization of neurons was (a) area intensive because of large capacitors in the circuit design, (b) neuronal spiking patterns were demonstrated with clocked neurons at the device level. To achieve more realistic biological neuron spiking behavior, emerging memristive devices are considered promising alternatives. In this paper, we propose, PrMnO3 (PMO)-resistive random-access memory (RRAM) device-based neuron. The voltage-controlled electrothermal timescales of the compact PMO RRAM device replace the electrical timescales of charging a large capacitor. The electrothermal timescale is used to implement an integration block with multiple voltage-controlled timescales coupled with a refractory block to generate biological neuronal dynamics. Here, first, a Verilog-A implementation of the thermal device model is demonstrated, which captures the current-temperature dynamics of the PMO device. Second, a driving circuitry is designed to mimic different spiking patterns of cortical neurons, including intrinsic bursting and chattering. Third, a neuron circuit model is simulated, which includes the PMO RRAM device model and the driving circuitry to demonstrate the asynchronous neuron behavior. Finally, a hardware-software hybrid analysis is done in which the PMO RRAM device is experimentally characterized to mimic neuron spiking dynamics. The work presents a realizable and more biologically comparable hardware-efficient solution for large-scale SNNs.
Ferroelectric HfO2-based synaptic devices: recent trends and prospects
Shimeng Yu et al 2021 Semicond. Sci. Technol. 36 104001
Neuro-inspired deep learning algorithms have shown promising futures in artificial intelligence. Despite the remarkable progress in software-based neural networks, the traditional von-Neumann hardware architecture has suffered from limited energy efficiency while facing unprecedented large amounts of data. To meet the performance requirements of neuro-inspired computing, large-scale vector-matrix multiplication is preferred to be performed in situ, namely compute-in-memory. Non-volatile memory devices with different materials have been proposed for weight storage as synaptic devices. Among them, HfO2-based ferroelectric devices have attracted great attention because of their low energy consumption, good complementary-metal-oxide-semiconductor (CMOS) compatibility and multi-bit per cell potential. In this review, recent trends and prospects of the ferroelectric synaptic devices are surveyed. First, we present the three-terminal synaptic devices based on the ferroelectric field effect transistor (FeFET), and discuss the switching physics of the intermediate states, the back-end-of-line integration and the 3D NAND architecture design. Then, we introduce a hybrid precision synapse concept that leverages the volatile charges on the gate capacitor of the FeFET and the non-volatile polarization on the gate dielectric of the FeFET. Lastly, we review two-terminal synaptic devices using the ferroelectric tunnel junction (FTJ) and ferroelectric capacitor (FeCAP). The design margins of the crossbar array with FTJ and FeCAP analyzed.