**2. ReRAM-based neural network implementation**

The emerging ReRAM device has shown superior performance in neuromorphic computing. The ReRAM device is promising to enable highly parallel, ultra-low-power computing in memory for AI applications owing to its structural simplicity, low power consumption, and ease of integration. Hardware implementations of artificial neurons and synapses play an important role in neuromorphic computing, attracting considerable attention over the past few decades. The fundamental function of artificial neurons is to emulate potential accumulation processes and spike generation functions. Meanwhile, artificial synapses are designed to implement various synaptic plasticities and learning weight signals. These plasticities are crucial for the learning function in neuromorphic computing. Based on the artificial neurons and synapses, neuromorphic hardware systems are built for the whole neural network functions. In this section, we will provide an overview of the hardware implementation of ReRAMbased neuromorphic synapses, neurons, and hardware systems.

#### **2.1 ReRAM-based artificial synapses**

The performance of synaptic devices directly impacts the learning accuracy and efficiency of a neuromorphic computing system. ReRAM devices are widely utilized to implement synaptic plasticities, such as signal weighting, short-term potentiation/ depression (STP/STD), long-term potentiation/depression (LTP/LTD), and spikingtime dependent plasticity (STDP).

*Enabling Neuromorphic Computing for Artificial Intelligence with Hardware-Software… DOI: http://dx.doi.org/10.5772/intechopen.111963*

#### **Figure 1.**

*The typical structure of ReRAMs devices and their switching states [62].*

#### **Figure 2.**

*I-V curves for ReRAM devices on unipolar and bipolar operations [62].*

The typical ReRAM device is in a metal/insulator/metal (MIM) structure [62] as is depicted in **Figure 1**, where the device consists of the bottom electrode (BE), top electrode (TE), and oxide layers. As illustrated in **Figure 2**, ReRAM devices can be switched between a high resistance state (HRS) and a low resistance state (LRS) with the unipolar or bipolar operation. This nonlinear feature resembles biological synapses, whose weight changes in response to stimuli passing from pre- to post-synaptic neurons.

In 2010, Lu proposed the first resistive synapse and demonstrated its function of STDP [1]. To achieve the synaptic function of STDP, well-designed shapes of neuronal pulses were also required. Therefore, Yu et al. proposed a metal oxide ReRAM-based synapse and an energy-efficient signal scheme for synapse programming, which involved tuning the pulse amplitude in each time slot [2]. As a further study, Ohno et al. proposed an inorganic synapse using Ag2S [3], showing that a single proposed synapse device exhibits both time-dependent STP and LTP features of a biological synapse by adjusting the repetition rate of input stimuli without the special design needs of neuronal pulses. Later, Li et al. demonstrated a chalcogenide resistive electronic synapse with an Ag/AgInSbTe/Ag structure to implement STDP function [5], in which the synaptic weights were modified with the cooperation of pre- and postsynaptic spikes and the growth of the weights was more stable by utilizing synaptic saturation mechanism. To eliminate the resistant fluctuations issue [6], Gao et al. proposed an oxide-based 3D vertical structure synapse and simulated a single synapse by using the mean value of a group of resistive switching devices.

Challenges in applying the ReRAM devices in neuromorphic computing were further explored and novel ReRAM-based synapses were proposed. Woo et al. analyzed the TiN/HfO2/Ti/TiN ReRAM-based synapse and observed deteriorated accuracy drop of the neuromorphic computing under abrupt SET switching operation [8]. Based on this observation, they proposed an optimized ReRAM-based synapse with an Al electrode on top to improve the accuracy of the RerAM-based neuromorphic computing. To address the issue of abrupt SET switching, Wu et al. found that increasing the temperature led to the transition from abrupt switching to analog switching [9]. Therefore, they proposed a HfO*x*/TEL ReRAM-based synapse, which incorporated a thermal enhanced layer (TEL) to confine heat for realizing analog switching. Kim et al. proposed a Ni/SiN*x*/AlO*y*/TiN ReRAM-based synapse, where the AlO*<sup>y</sup>* layer reduces current overshoot, resulting in a smooth reset switching transition to enhance analog switching performance [10]. Sun et al. proposed an XNOR-ReRAM synapse that enables equivalent XNOR and bit-counting operations to be carried out in parallel in binary neural networks (BNN) [11]. Roy et al. improved the reliability of HfO2 ReRAM devices through processes of Al-doping, ozone treatment, and postdeposition annealing [12].

#### **2.2 ReRAM-based artificial neurons**

Researchers have started to investigate ReRAM-based neuron designs with low design and computation costs for a few years. By leveraging the threshold transition characteristic of ReRAM devices, the dynamic process of pulse generation and resetting of neurons can be accurately represented. In addition, the ReRAM devices hold unique nonlinear features and flexible structures to achieve the rich kinetic properties of neurons. Furthermore, the nanoscale properties of ReRAM devices enable neurons with high power efficiency with the capability of achieving neuron functions with one or a few devices.

The ReRAM-based analog integrate-and-fire (I&F) neurons have been extensively explored. In 2016, Mehonic et al. implemented the Hodgkin-Huxley (H-H) neural model and leaky integrate-and-fire (LIF) neural model using silicon oxide ReRAM devices [13]. As is depicted in **Figure 3**, the charging and discharging process of neurons in the LIF model was successfully implemented. Compared to traditional hybrid analog/digital CMOS silicon neurons, the proposed ReRAM neurons

### *Enabling Neuromorphic Computing for Artificial Intelligence with Hardware-Software… DOI: http://dx.doi.org/10.5772/intechopen.111963*

significantly simplified the hardware design. Similarly, Kwon et al. proposed a ReRAM-based analog integrate-and-fire (I&F) neuron circuit without capacitors, which greatly reduces power consumption, delay, and neuron size [15]. Zhang et al. implemented a new I&F neuron based on an Ag/SiO2/Au threshold switching ReRAM device [16]. The proposed neuron can achieve four critical neuron functions, including all-or-nothing spiking, threshold-driven spiking, a refractory period, and a strengthmodulated frequency response. As the first attempt, Lashkare et al. proposed a Pr0*:*7Ca0*:*3MnO3 (PCMO) ReRAM device to implement an integrate-and-fire (I&F) neuron with all the neuronal functionalities—integration, reset, threshold comparison, spike, etc., by only utilizing a single device without any external control circuits [17].

In addition, the ReRAM-based neuron implementations have been extended to a large scale. For example, spiking neural networks on a large scale were achieved based on PCMO ReRAM-based neurons [17]. Lin et al. implemented neurons with a onetransistor-one-ReRAM (1T1R) structure device and integrated the neurons within the synaptic crossbar to build more dense and high-throughput process element [18].

#### **2.3 Neuromorphic system implementation**

The CMOS-based neuromorphic chips have several limitations that hinder their further development and applications [20–23]. Firstly, on-chip memory based on SRAM is area inefficient and can hardly store the weights of a large-scale neural network. In addition, the power consumption of off-chip storage using DRAM is more than 100 times higher than that of on-chip memory. The ReRAM device, on the other hand, offers advantages such as low programming voltage, fast switching speed, high integration density, and excellent performance scalability, which make it a promising candidate for neuromorphic computing system implementation.

In the last 10 years, ReRAM-based neuromorphic computing systems for artificial neural network implementation have been extensively studied. In 2012, Hu et al. proposed a neuromorphic hardware system using ReRAM crossbar arrays to achieve recall functions of Brain-State-in-a-Box (BSB) network [24]. In the hardware implementation, the resistive crossbar naturally performs the intensive vector-matrix multiplication, which is the basic computation in the neural network model in parallel. The signal strength was represented by the voltage amplitude and analog neurons with circuitry (e.g., amplifier, analog-to-digital converter) were utilized. Two years later, Hu et al. further implemented the training functions of the BSB model with ReRAM crossbar arrays and alleviated the noise issues found in the previous works, and impacted the computing accuracy [25]. They introduced an iterative scheme for training that uses the sign of inputs and magnitude differences between outputs and inputs, which significantly reduced the circuit design complexity. The analog designs of the neuromorphic systems are vulnerable to signal variations and introduce extremely heavy hardware design and area costs due to the large analog circuit components. With the target of solving the above challenges, in 2015, Liu et al. proposed spiking neuromorphic systems, where input signals were represented by spikes and the need for large-scale analog circuits was significantly reduced [26]. In their works, spiking neuromorphic systems with 1T1R resistive crossbar arrays were implemented for feedforward and Hopfield networks on digital image recognition with IBM 130 nm technology. The 1T1R structure is utilized to control the impact of the sneak path and thus guarantee computing accuracy and a novel Integrate-and-Fire Circuit (IFC) with high-speed and low-power consumption was developed.

With the continuous growing of artificial neural networks in scale and complexity, there is an urgent need to improve computing efficiency in speed and energy. Neuromorphic systems for large-scale and complex neural networks were developed accordingly. For example, Yakopcic et al. implemented the first deep convolutional neural network (CNN) on ReRAM-based crossbars with all data represented in 16 values without classification accuracy degradation [27]. The proposed neuromorphic hardware also achieved parallel computing of the convolution operations. Wen et al. implemented a long short-term memory (MLSTM) network using ReRAM crossbars [29]. The neural network training process is in extremely high demand of computing resources, while its speed and energy efficiency are highly constrained. Therefore, researchers also explored neuromorphic computing systems that can support neural network training besides inference. For example, Yao et al. implemented parallel online training for gray-scale face classification using an integrated 1024-cell array [28].
