Self-Learning Low-Level Controllers

*Dang Xuan Ba and Joonbum Bae*

### **Abstract**

Humanoid robots are complicated systems both in hardware and software designs. Furthermore, the robots normally work in unstructured environments at which unpredictable disturbances could degrade control performances of whole systems. As a result, simple yet effective controllers are favorite employed in lowlevel layers. Gain-learning algorithms applied to conventional control frameworks, such as Proportional-Integral-Derivative, Sliding-mode, and Backstepping controllers, could be reasonable solutions. The adaptation ability integrated is adopted to automatically tune proper control gains subject to the optimal control criterion both in transient and steady-state phases. The learning rules could be realized by using analytical nonlinear functions. Their effectiveness and feasibility are carefully discussed by theoretical proofs and experimental discussion.

**Keywords:** backstepping control, PID control, sliding mode control, gain-learning control, position control, low-level control

#### **1. Introduction**

Precise motion control of low-level systems is one of the most important tasks in industrial and humanoid robotic systems [1–3]. Different from industrial robots which commonly operate in compact regions with simple and almost repetitive missions, humanoid robots perform complicated works and face to unknown disturbances in daily activities. Hence, designing a high-performance controller that is easy to use in real-time implementation for such the maneuver systems is a big challenge [4, 5].

To accomplish motion control in real-time applications, conventional proportional-integral-derivative (PID) controllers are the first selection from engineers and researchers thanks to simplicity in design and acceptable control outcome for uncertain systems [6–11]. Stability of the servo-controlled systems is proven by theoretical analyses, and their flexibility could be enhanced using machine-learning methods such as ordinary or neuro fuzzy-logic-based self-tuning [7, 10, 11], poleplacement adaptation [8], or convolutional learning [9]. However, using linear control signals to suppress the nonlinear behaviors of the robotic dynamics may lead to unexpected transient performance. To overcome this drawback, nonlinear controllers such as sliding mode control (SMC), backstepping control (BSC), or inverse dynamical control have gotten attention from developers [12–17]. Indeed, a robust-integral- sign-error (RISE) controller was studied to consolidate lumped disturbances inside the system dynamics for achieving asymptotic control results [13]. In another direction, a model-based nonlinear disturbance-observer controller was proposed based on the backstepping technique to yield excellent control accuracies [15]. Nevertheless,

extended studies noted that the outstanding control performances are difficult to be preserved with hard control gains employed in diverse real-time operations [18, 19]. **3. Low-level intelligent nonlinear controller**

*Self-Learning Low-Level Controllers*

*DOI: http://dx.doi.org/10.5772/intechopen.96732*

*x*<sup>1</sup> and *x*<sup>2</sup> hold for the following assumptions:

a. The system output *x*<sup>1</sup> is measurable.

**3.1 Robust backstepping control scheme**

Let formulate the main control error as:

angular data with a bounded tolerance ð Þ*υ :*

where *x*1*<sup>d</sup>* is the desired trajectory of the controlled joint.

b. The disturbance *d* and its time derivative are bounded.

*Assumption 1*:

*Assumption 2*:

order.

dynamics Eq. (2) is:

for the disturbance *υ*:

**39**

where *k*<sup>1</sup> is a positive constant.

A new state control error is defined as:

form:

In this subsection, a position controller is developed based on the general model using the backstepping technique and new adaptation laws. The dynamics Eq. (1) can be splitted for low-level subsystems under the following state-space

*x*\_ <sup>1</sup> ¼ *x*<sup>2</sup> þ *υ*

*x*\_ <sup>2</sup> ¼ �*a*1*x*<sup>2</sup> þ *a*2*u* þ *d*

where *<sup>x</sup>*<sup>1</sup> <sup>¼</sup> **<sup>q</sup>***<sup>i</sup>*∣*i*¼1*::<sup>n</sup>* presents a specific joint angle, *<sup>x</sup>*<sup>2</sup> is the measurement joint velocity, *u* ¼ **τ***<sup>i</sup>*∣*i*¼1*::<sup>n</sup>* is the control torque at the specific joint, *υ* is the measurement noise, *a1* is a positive constant presenting the nominal dynamics, *a2* is another positive constant standing for the inverse nominal mass at low-level dynamics, and *d* is the lumped disturbance denoting the deviation of internal dynamics. Note that,

b. The angular velocity ð Þ *x*<sup>2</sup> is bounded and is indirectly measured from the

Before designing the final control signal, additional assumptions are given.

a. The measurement noise *υ* is bounded and differentiable up to the second

c. The desired signal *x*1*<sup>d</sup>* is bounded and differentiable up to the third order.

The time derivative of the control objective *e1* in considering the first equation of

To control the error *e*<sup>1</sup> to zero or to be as small as possible, a virtual control signal is employed to remove the time derivative of the desired signal and to compensate

*e*<sup>1</sup> ¼ *x*<sup>1</sup> � *x*1*<sup>d</sup>* (3)

*e*\_<sup>1</sup> ¼ *x*<sup>2</sup> þ *υ* � *x*\_ <sup>1</sup>*<sup>d</sup>* (4)

*x*2*<sup>d</sup>* ¼ *x*\_ <sup>1</sup>*<sup>d</sup>* � *k*1*e*<sup>1</sup> (5)

(2)

As a result, gain-learning SMC algorithms have been developed for robotic systems [18–21]. The control objective could be minimized by learning processes of robust gains, driving gains or massive gains [22, 23]. In fact, some control gains still need to manually tune for their possibly wide ranges due to nature of each control plant. Thus, it may lead to inconvenience during the operation.

Intelligent methods for automatically tuning all the control gains have been also proposed based on modified backtracking search algorithms (MBSA) combining with a Type-2 fuzzy-logic design [24] or model predictive approaches [25]. The desired gains could be estimated for the best performance by dealing with closedloop optimal constraints. Though promising control results were presented, smooth variation of the gain dynamics need to further consideration.

Gain-learning control approaches under backstepping design provided another interesting direction as well. PID control with a gain-varying technique encoded by the backstepping scheme was formerly studied [26]. Success of the creative control method was confirmed by a thorough theoretical proof and experimental validation results. Since the learning process of all the control gain is generated only by one damping function, versatility of the control design may be limited for diverse working conditions. Improvement on the flexibility of gain selection is thus still an open issue.

In this chapter, an extensive gain-adaptive nonlinear control approach is presented for high-performance motion control of a low-level servo system. The controller is comprised of an inner robust nonlinear loop and an outer gain-learning loop. The inner loop is developed based on a RISE-modified backstepping framework to ensure asymptotic tracking control in the existence of nonlinear uncertainties and disturbances. The second loop contains a new gain-adaptive engine to activate variation gains of the inner loop in real-time applications. Theoretical effectiveness of the proposed controller is concretely proven by Lyapunov-based analyses. Feasibility of the control approach was confirmed by intensive real-time experiments on a legged robot. Their features are presented in detail in the below sections.

#### **2. Problem statements**

General dynamics of a robotic system could be expressed in the following form:

$$\mathbf{M(q)\ddot{q} + C(q, \dot{q})\dot{q} + g(q) + \sigma\_{fr}(\dot{q}) + \mathbf{J}^T \mathbf{f}\_{ext} = \mathbf{\sigma} \tag{1}$$

where **q**, **q**\_ , **q**€ ∈ ℜ*<sup>n</sup>* are respectively the joint position, velocity and acceleration vectors, **M q**ð Þ<sup>∈</sup> <sup>ℜ</sup>*<sup>n</sup>*�*<sup>n</sup>*is the inertia matrix, **C q**ð Þ , **<sup>q</sup>**\_ <sup>∈</sup> <sup>ℜ</sup>*<sup>n</sup>*�*<sup>n</sup>*is the Centrifugal/Coriolis matrix, **g q**ð Þ<sup>∈</sup> <sup>ℜ</sup>*<sup>n</sup>*denotes the gravitational torque, **<sup>τ</sup>***fr*ð Þ **<sup>q</sup>**\_ <sup>∈</sup> <sup>ℜ</sup>*<sup>n</sup>* is the frictional torque, **J** *<sup>T</sup>* is the respective Jacobian matrix, **f***ext* is the external disturbance, and **τ** is control torque at robot joints.

The main control objective here is to find out a proper control signal **τ** that ensures a control error between the system output and a desired profile stabilizing at origin under various complicated environments.

To realize the control objective, conventional linear or nonlinear controllers such as Proportional-Integral-Derivative (PID) and Sliding mode control (SMC) methods are priority selections in industry thanks to their simplicity and robustness. However, such the mission in humanoid robots is a different story in which the systems frequently operate in unknown environments with harshly unpredictable disturbances [27, 28]. Obviously, the required controller is strong robustness, fast adaptation, and easy implementation.
