*2.1.1 Acquisition and processing of kinematic data*

The Microsoft Kinect collects the kinematic data of the Tai-Chi master (for training purposes) and the user. Through Kinect, we can obtain joints' transient position h i *x*, *y*, *z k <sup>t</sup>* and corresponding Quaternion rotation [48] cos *<sup>θ</sup>* 2 � �, sin *<sup>θ</sup>* 2 � �*v* ! D E*<sup>k</sup>* , where *θ* is an angle around unit axis *v* !, *t* is the time, and *k* is the joint identifier. Quaternions [48] are considered to represent the rotation of a rigid body in 3D space using four degrees-of-freedom (DOFs).

Quaternions are superior to many other traditional rotation formulation methods because they completely avoid gimbal-lock [49]. In VIGOR, Quaternions are used in 4D reconstruction over Unity3D platform and acquisition of kinetic signal. On the other hand, as a Quaternion is specified with reference to an arbitrary axis vector it is not a good choice in rotation recognition. In VIGOR, Euler angles h i *α*, *β*, *γ* , which represent the angles rotating around axis Z, X, Y respectively (denoted as h i *yaw*, *pitch*,*roll* in some literature) are adopted in gesture recognition.

**Figure 3.** *Input&output instruments (the optional hardware is highlighted in light color).*

VIGOR stores the captured kinetic data in JavaScript Object Notation (JSON) format, which includes joint position h i *x*, *y*, *z k t* � �, quaternion rotation cos *<sup>θ</sup>* 2 � �, sin *<sup>θ</sup>* 2 � �*v* ! D E*<sup>k</sup> t* � �, tracking status (0: invisible; 1: referred; 2: observable), and potentially forces **f** *k t* � � and moments, etc. Tracking status indicates whether or not the joint is observable by the sensor. The forces and moments are derived by inverse dynamics analysis.

Due to measurement error or unavoidable occlusion, a joint is not always observable or tractable by the kinetic sensor. Spherical linear intERPolation (SLERP) [50] and Kalman filtering techniques (be discussed in Section 3.1) are employed to compensate the missing data. As illustrated in our preliminary online video [22], SLERP can effectively address those short-term missed-tracking joints (namely tracking status = 0 or 1).

#### *2.1.2 Acquisition of tactile data*

Besides Kinect, other acquisition instruments such as accelerometers, orientation sensors, and strain gauges [39] are also considered for the VIGOR system. As indicated above, a foot pressure sensor is used to obtain the ground reaction force *Ft* for inverse dynamic analysis. Furthermore, electromyography (EMG) [39] is selectively employed to evaluate and record the electrical activity produced by skeletal muscles. The EMG signal is characterized by a frequency range of several hertz to over 1 kHz and by amplitudes ranging from fractions of a microvolt to a few thousand microvolts. Electromyographic signals can be analyzed to detect activation level or to analyze the biomechanics of users' movement. To acquire highquality EMG signals from localized muscle region, identification of localized muscle region of users, noise reduction and grounding practices (to eliminate extraneous electrical noise), electrode site preparation and placement (to minimize the detection of irrelevant bioelectrical signals) and appropriate differential signal preamplification and preliminary signal conditioning (to further enhance signal-to-noise ratio) can be conducted. EMG signals can be classified to detect movements of limb. Our active/powered orthosis system, which enables users for movement, has EMG and Internal measurement Unit (IMU) sensors. Those sensors can monitor body movement and muscle activity and send the measurement data to the server.

#### **2.2 Reconstruction of 4D data**

4D kinetic feedback/instruction is reconstructed through virtual reality, tactile actuators, and motoring system that drives the active orthosis. (1) *VR/AR facility*, which can visualize the kinetics of human body in Quaternion format [48, 49] (acceptable by Unity3D VR/AR SDK). (2) *Tactile actuators*, through which VIGOR can directly guide users with somatosensory feedback. Tactile actuators potentially used in VIGOR include Eccentric Rotating Mass (ERM), Linear Resonant Actuator (LRA), Piezo, and Electro-Active polymers (EAP) with high fidelity of sensations, and excellent durability. (3) *Active orthosis* [51], which enables users with direct physical support through functional electrical stimulation (F.E.S) [51] or robotic exoskeletons [45].

#### **2.3 Developing communication and edge-computing protocols**

#### *2.3.1 Real-time, two-way communication*

Two-way communications are of key importance in the proposed system, since the information needs to be exchanged in a real-time manner. The challenges of the communication protocol for the proposed VIGOR include: (1) Real-time

## *VIGOR: A Versatile, Individualized and Generative ORchestrator to Motivate the Movement… DOI: http://dx.doi.org/10.5772/intechopen.96025*

communication: Information in the VIGOR system needs to be conveyed in real time. If there is a significant delay in the communications, synchronization between the Tai-Chi master and user will be lost and the user will experience a disturbed rhythm. (2) High communication throughput: When there are many users, all the corresponding multimodal sensory data and feedback information need to be conveyed in the network, thus incurring a substantial requirement for communication bandwidth. (3) Two-way communications: The communications are between the virtual Tai-Chi master and users with mutual interactions. Therefore, it could be sub-optimal if one-way communications are considered separately. (4) Dynamics awareness: The communications may be optimized together with the physical dynamics of the virtual Tai-Chi master and users (namely the motions).

To address the above challenges, first, VIGOR can be modeled as a cyber physical system (CPS) [52, 53] and then the bandwidth can be analyzed for controlling the physical dynamics. Last, the detailed communication protocol can be designed and evaluated with the whole system.
