**2. Human target analysis in 2D and 3D domains**

#### **2.1. Human target analysis with RD video sequence**

The RD video sequence that consists of N time sampled 2D range-Doppler images contains both spatial and temporal characteristics: range and frequency information consists in every RD image, while time information exists among sequential frames. Compared with 1D and 2D forms of radar echoes, the joint time-/range-frequency form of echoes contains all the targets' motion information.

Among these human target analysis systems using the RD video sequence, so far, a representative example is Google Project Soli, which is the first gesture recognition system capable of recognizing a rich set of dynamic gestures based on short-range radar [27, 28]. It is based on an end-to-end trained combination of deep convolutional and recurrent neural networks, and the dataset is comprised of 3D data cubes. Combining CNN and RNN could enhance the ability to recognize different activities that vary in temporal and spatial dimensions. The system can recognize subtle gestures of 10 kinds performed by 10 people. From then on, many researches have been done using radar just like Kinect and Leap Motion in CV [21]. In addition, with the advent of this system, many novel ideas have been proposed based on it [22, 29, 30].

#### **2.2. Human target analysis with HRRP and micro-Doppler profiles**

both generated by micromotions, have their shortcomings: they only contain information from either the time-frequency or time-range domain. Micro-Doppler analysis neglects range

Therefore, in order to analyze the target signature more comprehensively, we describe a new concept called *range-Doppler surface* (RDS). As an alternative to the micro-Doppler profile and HRRP, RDS is a radar target representation extracted from a three-dimensional data cube the range-Doppler (RD) video sequence [11, 12]. The RDS consists of all the important information contained in both HRRP and micro-Doppler signatures. The present study analyzes the RDS using simulated and real radar data. Results suggest a new area of human target

It is worth mentioning that the term "range-Doppler surface" has been presented in prior works [13–15]. It was used for a 2D range-Doppler image that is shown in a 3D perspective. In this chapter we present this term to describe the time-varying range-Doppler isosurface information. RDS is referred to 3D visualization for the first time in this study, and it is indeed

Nowadays, deep learning has become a mainstream method for human activity recognition instead of conventional machine learning methods. Deep learning came into our sight and has emerged as a hot topic in the past few years. It works by learning several layers of representation for modeling the complex relationships among data. It can create high-level features from related low-level ones by means of its hierarchical architecture without artificial extraction from the raw data and specialized knowledge. In this way, it makes activity recognition system more intelligent and versatile. Therefore, deep learning is an applicable approach to

This chapter is organized as follows: In Section 2, human target analysis with the micro-Doppler profile, HRRP, as well as the three-dimensional RD video sequence is described. In Section 3, the range-Doppler surface is described. Then, deep learning for human activity classification is introduced briefly in Section 4. Finally, future directions are drawn in Section 5.

The RD video sequence that consists of N time sampled 2D range-Doppler images contains both spatial and temporal characteristics: range and frequency information consists in every RD image, while time information exists among sequential frames. Compared with 1D and 2D forms of radar echoes, the joint time-/range-frequency form of echoes contains all the

Among these human target analysis systems using the RD video sequence, so far, a representative example is Google Project Soli, which is the first gesture recognition system capable of recognizing a rich set of dynamic gestures based on short-range radar [27, 28]. It is based on an end-to-end trained combination of deep convolutional and recurrent neural networks,

information, while HRRP analysis neglects Doppler information.

analysis and classification.

60 UWB Technology and its Applications

identify human activities.

targets' motion information.

a suitable, precise term to describe this concept.

**2. Human target analysis in 2D and 3D domains**

**2.1. Human target analysis with RD video sequence**

Although containing abundant information of human activity properties, 3D form of human backscattering echoes is complicated to process. As a result, the complexity of the systems using 3D form of echoes is higher compared with those using 2D forms. 2D forms of radar echoes, which mainly refer to HRRP and micro-Doppler profiles, also carry enough human activity information and can be used for human target analysis.

It is a common way to obtain time-Doppler maps, namely, joint time-frequency transformation (JTFA) [23, 24]. Similar to the developments in other fields such as acoustics and speech processing, JTFA can provide additional insight into the analysis, interpretation, and processing of radar signals, and the performance is superior to what has been achieved in the traditional time or frequency domain alone [3].

The short-time Fourier transform (STFT) is the most commonly used time-frequency transform. STFT performs the Fourier transform on a short-time sliding window basis instead of using one long-time window to the entire signal. The square modulus of the STFT is called the spectrogram, which is a nonnegative time-frequency energy distribution without phase information [20]:

$$Spectrogram(t, f) = \left| STFT(t, f) \right|^2\tag{1}$$

The resolution of STFT is determined by the size of window function. There is a trade-off between the time resolution and the frequency resolution [4].

By performing a STFT over time for every range bin, a series of 2D time-Doppler images along range can be acquired. Then, summing the time-Doppler "video" along range, a time-Doppler map is obtained.

Among these three 2D forms of echoes, till now, the time-Doppler maps are most commonly used to analyze human targets. The time-Doppler maps include a wealth of Doppler information changing over time. The main Doppler shift is caused by the bulk speed, while micro-Doppler is produced by rotating or vibrating parts, such as the legs, feet, and hand. By selecting and classifying the micro-Doppler features in the time-Doppler map, human activities can be recognized by various models. For example, G. Klarenbeek et al. applied a LSTM structure with the time-Doppler maps to realize the multi-target human gait classification [26].

Time-range maps contain time-varying range information between the target and the radar. When a person is moving, different components of the human body have different relative distances from the radar at time t. Therefore, various time-range maps produced by different activities can be used to recognize the corresponding activities, although they neglect the Doppler information. In [25], Yuming Shao et al. have employed the time-range maps with a deep CNN to classify human motions, and a good performance was achieved.

#### **3. Range-Doppler surface**

#### **3.1. Range-Doppler processing**

In general, separating the frequency components of different body parts is a vital step for human target analysis. However, the ability to resolve separate frequency components is limited because of the time-varying Doppler shifts in radar echoes. A general way of representing the scattered signals is range-Doppler (RD) processing. And a classical way for RD processing is to apply Fourier transform to samples from a fixed range bin over one coherent processing interval. This interval is theoretically limited by the time during which the target stays in the same range bin:

$$T\_r = \frac{\delta\_r}{\mathcal{U}} \tag{2}$$

In radar, we usually assume that the target remains static in one pulse duration *T*:

is the range resolution of the radar and *Bw*

*rx*(*t*′ , *f* , *t s*

Keystone transform-based range-Doppler processing is proposed in this chapter.

where *<sup>R</sup>* <sup>=</sup> \_\_\_ *<sup>c</sup>*

as

2 *Bw*

FT is performed over *t*

where *sinc*( *<sup>f</sup>* ) <sup>=</sup> *sin*(*<sup>f</sup>* ) \_\_\_\_\_\_

*Srx*(*t*′

*S*˜

*<sup>f</sup>* .

**3.2. Range-Doppler surface construction**

Doppler, and target velocity).

*s*

*vT* ≪ *R* (8)

cates that the Doppler effect in fast time is negligible. By simplifying, Eq. (7) can be rewritten

in Eq. (9) and the Doppler spectrum is given:

*<sup>d</sup>*) = *p*(*t*′ − *τ*0)*sinc*( *f*

However, a target may traverse multiple range cells in one coherent processing interval sometimes. In this situation, applying conventional Fourier transform-based RD processing is not suitable, which will lead to a blur in frequency. To improve the Doppler resolution, a

By performing RD processing on the radar echoes, a sequence of RD images is obtained. The threedimensional RD video is proposed to describe the slow-time evolution of target RD signatures.

Before constructing the RDS, it is essential to detect the target in the range-Doppler domain, since detection allows the extraction of targets and elimination of false alarms. The cell-average constant false alarm rate (CA-CFAR) procedure [16] is a classical approach of detecting a target in noise and clutter. Detection is performed employing a two-dimensional CA-CFAR procedure [17] in the range-Doppler domain. For each range-Doppler image in the range-Doppler video sequence, a sliding 2D window is applied to scan this RD image pixel by pixel. For each pixel, it is claimed as detected if its intensity exceeds an estimated threshold. In **Figure 1**, a typical 2D window is shown. The cell under test covers the target reflections. The reference cells estimate background noise for computing the detection threshold. The guard cells separate the cell under test and reference cells as a barrier. The sizes of these cells strongly affect the performance of the CA-CFAR detection and thus should be tuned carefully according to radar parameters and target characteristics (e.g., signal bandwidth, maximum unambiguous

In **Figure 2a**, the detected scatterers of a simulated human target are shown in a three-dimensional (3D) volume, where the intensities of different scatterers are represented by various colors. Note that the simulated radar system uses the same parameters as used in generating **Figure 3**. Finally, the RDS (see **Figure 2b**) is constructed by creating a surface that has the same intensity value within the 3D range-Doppler-time volume (i.e., range-Doppler video sequence) in **Figure 2a**. Isosurface plots are similar to contour plots in that they both indicate where values are equal. The MATLAB® function *isosurface* is applied to extract the isosurface from the volume using a

) = *p*(*t*′ − *τ*0) *ej*<sup>2</sup>*<sup>f</sup>*

*c t*

*<sup>d</sup>* − *β f*

is the signal bandwidth. This indi-

Toward Deep Learning-Based Human Target Analysis http://dx.doi.org/10.5772/intechopen.81592 63

*<sup>s</sup>* (9)

*<sup>c</sup>*), (10)

The radar transmits a coherent burst of M pulses:

$$S\_{tx} = \sum\_{n=0}^{M-1} p\{t - mT\_s\} e^{j2\pi f\_s t} \tag{3}$$

where *t* is the fast time, *p*(*t*) is the pulse complex envelope, *Tr* is the pulse repetition interval, and *f c* is the center frequency. The echo of a moving point scatterer can be described by a delayed and attenuated version of the transmitted signal in Eq. (3):

$$\mathcal{S}\_{\text{rx}} = \alpha \mathcal{S}\_{\text{rx}}(t - \tau(t)),\tag{4}$$

where *α* is the complex amplitude and *τ*(*t*) is the round-trip time delay. Assuming that the radial velocity *v* is constant during this period, the round-trip delay can be approximated by

$$
\tau(t) = \tau\_o - \frac{2v}{c}t,\tag{5}
$$

where *τ*<sup>0</sup> = <sup>2</sup> *<sup>R</sup>*\_\_\_0 *<sup>c</sup>* is the initial delay, *R*<sup>0</sup> is the initial range, and *c* is the speed of light. Substituting Eqs. (3) and (5) into Eq. (4) leads to

$$S\_{rs}(t) = \alpha \sum\_{n=0}^{M-1} p\left[\left(1 + \frac{2\upsilon}{c}\right)t - \tau\_0 - mT\_r\right] e^{\left[2\upsilon \left(\left(1 + \frac{2\upsilon}{c}\right)t\right]}\right] \tag{6}$$

where all the constant terms have been absorbed and the variable name has been kept as *α*. Then, for notational convenience the fast time *t*′ was introduced: *t*′ = *t* − *mTr* . Substituting *t*′ into Eq. (6), after demodulation the baseband signal is given as

$$\begin{aligned} \text{(6) after decomposition the baseband signal is given as} \\ S\_n(t', m) &= ap \left[ \left( 1 + \frac{2v}{c} \right) t' - \tau\_0 + \frac{2v}{c} m \, T\_r \right] \\ e^{\left( 2\pi i \frac{2v}{c} (t + mT\_r) \right)} \end{aligned} \tag{7}$$

In radar, we usually assume that the target remains static in one pulse duration *T*:

$$vT \ll \delta \mathsf{R} \tag{8}$$

where *<sup>R</sup>* <sup>=</sup> \_\_\_ *<sup>c</sup>* 2 *Bw* is the range resolution of the radar and *Bw* is the signal bandwidth. This indicates that the Doppler effect in fast time is negligible. By simplifying, Eq. (7) can be rewritten as

$$S\_n(t', t\_s) = \exp\{t' - \tau\_a\} e^{(2\pi\theta)^{\varepsilon}\_+} \tag{9}$$

FT is performed over *t s* in Eq. (9) and the Doppler spectrum is given:

$$\tilde{S}\_{cr}(t',f\_d) = \alpha p(t'-\tau\_0)\sin c(f\_d - \beta f\_c),\tag{10}$$

where *sinc*( *<sup>f</sup>* ) <sup>=</sup> *sin*(*<sup>f</sup>* ) \_\_\_\_\_\_ *<sup>f</sup>* .

information. In [25], Yuming Shao et al. have employed the time-range maps with a deep CNN

In general, separating the frequency components of different body parts is a vital step for human target analysis. However, the ability to resolve separate frequency components is limited because of the time-varying Doppler shifts in radar echoes. A general way of representing the scattered signals is range-Doppler (RD) processing. And a classical way for RD processing is to apply Fourier transform to samples from a fixed range bin over one coherent processing interval. This interval is theoretically limited by the time during which the target stays in the

\_\_*r*

*p*(*t* − *mTr*) *ej*2*π<sup>f</sup>*

is the center frequency. The echo of a moving point scatterer can be described by a

*Srx* = *αStx*(*t* − *τ*(*t*)), (4)

where *α* is the complex amplitude and *τ*(*t*) is the round-trip time delay. Assuming that the radial velocity *v* is constant during this period, the round-trip delay can be approximated by

where all the constant terms have been absorbed and the variable name has been kept as *α*.

*<sup>c</sup>* )*t* − *τ*<sup>0</sup> − *mTr*] *ej*2*π<sup>f</sup>*

*<sup>c</sup>* )*t*′ <sup>−</sup> *<sup>τ</sup>*<sup>0</sup> <sup>+</sup> \_2*<sup>v</sup>*

*c t*

*m*=0 *M*−1 *<sup>v</sup>* (2)

, (3)

*<sup>c</sup> t*, (5)

is the initial range, and *c* is the speed of light. Substituting

*c*(1+\_\_2*<sup>v</sup> <sup>c</sup>* )*t*

was introduced: *t*′ = *t* − *mTr*

*<sup>c</sup> m Tr*]

is the pulse repetition interval,

, (6)

. Substituting *t*′

into

(7)

to classify human motions, and a good performance was achieved.

**3. Range-Doppler surface**

62 UWB Technology and its Applications

**3.1. Range-Doppler processing**

*Tr* <sup>=</sup> *<sup>δ</sup>*

The radar transmits a coherent burst of M pulses:

*<sup>τ</sup>*(*t*) <sup>=</sup> *<sup>τ</sup>*<sup>0</sup> <sup>−</sup> \_\_2*<sup>v</sup>*

*<sup>c</sup>* is the initial delay, *R*<sup>0</sup>

Then, for notational convenience the fast time *t*′

Eq. (6), after demodulation the baseband signal is given as

(*t*′

where *t* is the fast time, *p*(*t*) is the pulse complex envelope, *Tr*

delayed and attenuated version of the transmitted signal in Eq. (3):

*m*=0 *M*−1

*ej*2*π<sup>f</sup>*

*<sup>p</sup>*[(<sup>1</sup> <sup>+</sup> \_\_2*<sup>v</sup>*

, *<sup>m</sup>*) <sup>=</sup> *<sup>α</sup>p*[(<sup>1</sup> <sup>+</sup> \_2*<sup>v</sup>*

*c* \_\_2*v <sup>c</sup>* (*t*′ +*mTr*)

*Stx* = ∑

same range bin:

and *f c*

where *τ*<sup>0</sup> =

<sup>2</sup> *<sup>R</sup>*\_\_\_0

Eqs. (3) and (5) into Eq. (4) leads to

*Srx*(*t*) = *α*∑

*Srx*

However, a target may traverse multiple range cells in one coherent processing interval sometimes. In this situation, applying conventional Fourier transform-based RD processing is not suitable, which will lead to a blur in frequency. To improve the Doppler resolution, a Keystone transform-based range-Doppler processing is proposed in this chapter.

By performing RD processing on the radar echoes, a sequence of RD images is obtained. The threedimensional RD video is proposed to describe the slow-time evolution of target RD signatures.

#### **3.2. Range-Doppler surface construction**

Before constructing the RDS, it is essential to detect the target in the range-Doppler domain, since detection allows the extraction of targets and elimination of false alarms. The cell-average constant false alarm rate (CA-CFAR) procedure [16] is a classical approach of detecting a target in noise and clutter. Detection is performed employing a two-dimensional CA-CFAR procedure [17] in the range-Doppler domain. For each range-Doppler image in the range-Doppler video sequence, a sliding 2D window is applied to scan this RD image pixel by pixel. For each pixel, it is claimed as detected if its intensity exceeds an estimated threshold. In **Figure 1**, a typical 2D window is shown. The cell under test covers the target reflections. The reference cells estimate background noise for computing the detection threshold. The guard cells separate the cell under test and reference cells as a barrier. The sizes of these cells strongly affect the performance of the CA-CFAR detection and thus should be tuned carefully according to radar parameters and target characteristics (e.g., signal bandwidth, maximum unambiguous Doppler, and target velocity).

In **Figure 2a**, the detected scatterers of a simulated human target are shown in a three-dimensional (3D) volume, where the intensities of different scatterers are represented by various colors. Note that the simulated radar system uses the same parameters as used in generating **Figure 3**. Finally, the RDS (see **Figure 2b**) is constructed by creating a surface that has the same intensity value within the 3D range-Doppler-time volume (i.e., range-Doppler video sequence) in **Figure 2a**. Isosurface plots are similar to contour plots in that they both indicate where values are equal. The MATLAB® function *isosurface* is applied to extract the isosurface from the volume using a

**Figure 1.** 2D CA-CFAR window [27] (red: cell under test; yellow: guard cells; green: reference cells).

user-defined isosurface threshold. The isosurface connects points that have the specified value much the way contour lines connect points of equal elevation. Note that the difference of the surface color in **Figure 2b** is not due to different intensities, but due to the lighting effect used to illustrate the 3D object in MATLAB®. Selecting a reasonable threshold is important in this procedure, because this affects the final output significantly. Although currently the threshold is set manually, automatic approaches to construct the volume surface are certainly interesting in future studies.

**Figure 2.** Range-Doppler surface of a simulated walking human target.

Toward Deep Learning-Based Human Target Analysis http://dx.doi.org/10.5772/intechopen.81592 65

**Figure 3.** Range-velocity images of one simulated human gait.

Target analysis has been commonly done in the time-range domain or time-frequency domain. As mentioned above, HRRP neglects Doppler information, while micro-Doppler neglects range information. Furthermore, micro-Doppler is difficult to be used in multi-target situations, since the Doppler spectrums of different targets may overlap. The RDS shows the target surface in the 3D range-Doppler-time space. All the important targets' information, which might be contained in HRRP and micro-Doppler, is included in RDS.

The RDS of different body segments is presented in **Figure 4**. The responses of the feet are well separated in either range or Doppler, and the responses of the thorax and hands overlap with each other. The feet have a larger Doppler offset than the thorax and hands.

#### **3.3. RDS of measured human activities**

PulsOn 400 radar system, manufactured by Time Domain Corporation [18], was used to acquire measurement data (experimental setup; see **Figure 5**). Its operational frequency band is 3.1–5.3 GHz, and the signal is modulated by an m-sequence. The transmitted power is −14.5 dBm. The pulse repetition frequency is 200 Hz, and the sampling frequency is 16.39 GHz. More details about this radar are given in the literature [19].

In the measurement, two scenarios were considered: single-person walking and two-people walking. The RDSs generated for these two scenarios are presented in **Figures 6** and **7**, respectively. The RDS for the single-person scenario is similar to the simulated RDS shown

user-defined isosurface threshold. The isosurface connects points that have the specified value much the way contour lines connect points of equal elevation. Note that the difference of the surface color in **Figure 2b** is not due to different intensities, but due to the lighting effect used to illustrate the 3D object in MATLAB®. Selecting a reasonable threshold is important in this procedure, because this affects the final output significantly. Although currently the threshold is set manually, automatic approaches to construct the volume surface are certainly interesting in future studies. Target analysis has been commonly done in the time-range domain or time-frequency domain. As mentioned above, HRRP neglects Doppler information, while micro-Doppler neglects range information. Furthermore, micro-Doppler is difficult to be used in multi-target situations, since the Doppler spectrums of different targets may overlap. The RDS shows the target surface in the 3D range-Doppler-time space. All the important targets' information, which

**Figure 1.** 2D CA-CFAR window [27] (red: cell under test; yellow: guard cells; green: reference cells).

The RDS of different body segments is presented in **Figure 4**. The responses of the feet are well separated in either range or Doppler, and the responses of the thorax and hands overlap

PulsOn 400 radar system, manufactured by Time Domain Corporation [18], was used to acquire measurement data (experimental setup; see **Figure 5**). Its operational frequency band is 3.1–5.3 GHz, and the signal is modulated by an m-sequence. The transmitted power is −14.5 dBm. The pulse repetition frequency is 200 Hz, and the sampling frequency is 16.39 GHz. More details

In the measurement, two scenarios were considered: single-person walking and two-people walking. The RDSs generated for these two scenarios are presented in **Figures 6** and **7**, respectively. The RDS for the single-person scenario is similar to the simulated RDS shown

with each other. The feet have a larger Doppler offset than the thorax and hands.

might be contained in HRRP and micro-Doppler, is included in RDS.

**3.3. RDS of measured human activities**

64 UWB Technology and its Applications

about this radar are given in the literature [19].

**Figure 2.** Range-Doppler surface of a simulated walking human target.

**Figure 3.** Range-velocity images of one simulated human gait.

**Figure 5.** Radar system and experimental setup.

Toward Deep Learning-Based Human Target Analysis http://dx.doi.org/10.5772/intechopen.81592 67

**Figure 6.** Range-Doppler surface of two human targets (threshold = −20 dB).

**Figure 4.** Range-Doppler surface of different body segments (a) RDS of feet, (b) Time-range projection of RDS in (a), (c) Time-velocity projection of RDS in (a), (d) RDS of thorax + hands, (e) Time-range projection of RDS in (d), (f) Timevelocity projection of RDS in (d).

in **Figure 2b**, and the capability of RDS to separate body segments is demonstrated again in **Figure 6d** and **e**. It should be noted that in the processing for real data, static clutter is removed via moving target indication before constructing RDS, which cancels the clutter and also the stationary parts of the human body. More precisely, in each walking step, the reflection from the stationary leg/foot is rejected, while the reflection of the moving leg/foot is retained.

**Figure 5.** Radar system and experimental setup.

**Figure 6.** Range-Doppler surface of two human targets (threshold = −20 dB).

in **Figure 2b**, and the capability of RDS to separate body segments is demonstrated again in **Figure 6d** and **e**. It should be noted that in the processing for real data, static clutter is removed via moving target indication before constructing RDS, which cancels the clutter and also the stationary parts of the human body. More precisely, in each walking step, the reflection from the stationary leg/foot is rejected, while the reflection of the moving leg/foot

**Figure 4.** Range-Doppler surface of different body segments (a) RDS of feet, (b) Time-range projection of RDS in (a), (c) Time-velocity projection of RDS in (a), (d) RDS of thorax + hands, (e) Time-range projection of RDS in (d), (f) Time-

is retained.

velocity projection of RDS in (d).

66 UWB Technology and its Applications

as feature extraction, tracking, or classification. Similar to micro-Doppler or high-resolution range profiles, the RDS has the potential of being used to analyze different types of targets (e.g., people or animals) and also used in different types of radars (e.g., S-band, C-band, or

Toward Deep Learning-Based Human Target Analysis http://dx.doi.org/10.5772/intechopen.81592 69

With the rapid emergence of new deep learning algorithms and architectures, the development of many domains such as speech recognition, visual object recognition, object detection, and even drug discovery and genomics has been accelerated. Deep learning is composed of multiple processing layers to learn high-level representations with multiple levels of abstraction, thus automating the process of feature extraction. Hence, deep models do not need heavy feature engineering and domain knowledge compared with traditional machine learning techniques. What is more, with so many complicated deep-level transformations, very complex functions can be learned, and more classification and recognition problems can be solved. As a result, deep learning has made great contributions in overcoming difficulties in

Next, we will mainly describe several deep learning models, which are used mostly in human

Convolutional neural network (CNN) is inspired by the visual cortex structure which is composed of simple cells and complex cells. It adopts four key ideas: local connections, parameter sharing, pooling, and multilayers. In this way, CNN is able to fully explore the property of raw signals that there are compositional hierarchies, namely, extracting higherlevel features from the lower-level ones. As a result, convolutional neural networks, as one of the representative algorithms of deep learning, have made a remarkable progress in object detection and recognition, natural language processing (NLP), speech recognition, and medical image analysis in the past few years. In human activity recognition field, CNN is one of the most used deep learning models. For instance, Zhenyuan Zhang et al. have adopted this network to realize continuous dynamic gesture recognition using a radar sensor [30], while Youngwook Kim et al. detected and classified human activities using deep convolutional

With the successful application in NLP, recurrent neural network (RNN) has caught researchers' attention. RNNs have shone light on modeling temporal sequences such as texts and speeches because they can mine timing and semantic information in them. From the perspective of network structure, RNN can remember the previous information and use it to influence the output of the following nodes. However, conventional RNN has its own limit: long-term dependencies. To overcome this problem, long short-term memory (LSTM) came into being

artificial intelligence and advancing the development of artificial intelligence.

X-band radar).

target analysis field.

neural networks [32].

**4.2. Recurrent neural network**

**4.1. Convolutional neural network**

**4. Deep learning for human target analysis**

**Figure 7.** Range-Doppler surface of two human targets (threshold = −23 dB).

In **Figure 7**, the RDS of the two-people scenario shows that the backscattering of the human targets is automatically separated in the 3D range-Doppler-time space. This indicates that RDS is not only able to show the range-Doppler signatures of a single extended target but also able to separate (or even track) multiple targets in the range-Doppler video sequence. Additional processing to separate multi-target reflection (e.g., the separating method proposed in [19]) is not required anymore.

As an example, the RDS has been demonstrated for human target analysis using an S-/C-band UWB radar, but RDS itself is in fact a generic tool. It can be used in various applications, such as feature extraction, tracking, or classification. Similar to micro-Doppler or high-resolution range profiles, the RDS has the potential of being used to analyze different types of targets (e.g., people or animals) and also used in different types of radars (e.g., S-band, C-band, or X-band radar).
