**3. Distinction of singularity of surgical operation**

In this study, suturing is chosen as the objective surgical operation for automatic identification, and especially "insertion of a needle" in suturing is selected as the objective surgical operation for distinction of singularity. The flow for distinction of the singularity of the surgical operation "insertion of a needle" is explained as follows.

#### **3.1. Features of the operation**

Computational Intelligence in Electromyography Analysis – 250 A Perspective on Current Applications and Future Challenges

PHANTOM Omni and attached four strain gauges.

ω

ω*stylus*

**2.2. Measurement of surface electromyography** 

**Figure 2.** Sensor allocation for needle driver

**Figure 3.** Allocation of surface electrode

**3. Distinction of singularity of surgical operation** 

the surgical operation "insertion of a needle" is explained as follows.

*gimbal*

As shown in Fig.2, the movement of the needle driver is measured by the haptics device

*Strain3*

*Strain1*

The SEMG signals are measured by three electrodes stuck on the forearm of the operator as shown in Fig.3. The electrode 1 was stuck on the musculus flexor carpi radialis, the electrode 2 was stuck on the musculus extensor carpi ulnaris, the electrode 3 was stuck on the musculus extensor carpi radialis longus, and the earth electrode was stuck on the wrist.

In this study, suturing is chosen as the objective surgical operation for automatic identification, and especially "insertion of a needle" in suturing is selected as the objective surgical operation for distinction of singularity. The flow for distinction of the singularity of

*Strain2*

**Needle driver**

*Strain4*

From measurements of the SEMG signals by three electrodes, the amount of distortion by four strain gauges, and the angular velocity of gimbal and stylus by haptic device, the features are defined as follows.

For identifying the surgical operation, the features of the operation are extracted from the measurements of the movement of the needle driver. Define the features as follows.

$$St\_{ch} = \frac{1}{N} \sum\_{n=1}^{N} strain\_{ch\{n\}} \tag{1}$$

$$\Delta \Omega\_{\text{symbol}} = \frac{1}{N} \sum\_{n=1}^{N} \alpha\_{\text{global}\{n\}} \tag{2}$$

$$
\Delta\_{stplus} = \frac{1}{N} \sum\_{n=1}^{N} o\_{stplus\{n\}} \tag{3}
$$

where *strainch*(*n*) (*ch*=1,2…4) is measured value from each strain gauge, *ωgimbal*(*<sup>n</sup>*) and *ωstylus*(*<sup>n</sup>*) are measured angular velocity from the haptic device, and *n* represents the number of the sampled signals.

The features of any singularity of operation are extracted from operator's SEMG signals. The SEMG signals are measured by sampling frequency Fs=2 kHz, and Fast Fourier Transform (FFT) is performed to each SEMG signal for every N=512 sampled data, which is equivalent to perform FFT every 0.256 seconds.

After filtering the SEMG signals by the fourth order Butterworth type band pass filter with 10 Hz to 1 kHz range, the full wave rectification is carried out. In addition, for normalization, the measured SEMG signal of each electrode is divided by the maximum value of the pre-measured SEMG for each operation. Define the features as follows.

Average absolute value: In order to perform pattern recognition, average absolute value of each electrode is often used, which is given as follows.

$$\text{AMAV}\_{ch} = \frac{1}{N} \sum\_{n=1}^{N} \left| \text{EMG}\_{ch(n)} \right| \tag{4}$$

where *EMGch*(*<sup>n</sup>*) (*ch*=1,2,3) is SEMG signal of each electrode, and *n* represents the number of the sampled signals.

Center-of-gravity: In the case where the singular operation is performed, it is expected that change of the waveform can be observed in the SEMG signal. Therefore, as a value representing change of the waveform of the SEMG signal, the value of center-of-gravity is employed, which is defined as follows.

Computational Intelligence in Electromyography Analysis – 252 A Perspective on Current Applications and Future Challenges

$$\log\_{ch} = \sum\_{n=1}^{N} \left( n \bullet \left| EMG\_{ch(n)} \right| \right) \bigg/ \sum\_{n=1}^{N} \left| EMG\_{ch(n)} \right| \tag{5}$$

Distinction of Abnormality of Surgical Operation on the Basis of Surface EMG Signals 253

*V St St* 1 12 = ⋅ (8)

2 2 *V St St* 2 34 = + (9)

*V*<sup>3</sup> *gimbal stylus* =Ω ⋅Ω (10)

(*i*=1,2) (11)

(12)

In addition, to identify the state of operation of the needle driver using a threshold value,

<sup>1</sup> , <sup>0</sup> *i i*

3 3 3

where *THi* (*i*=1,2,3*H*, 3*L*) is threshold value for each new feature determined through trial and error. On the basis of the threshold criteria for the six operations, a surgical operation is

> /Operation T1 T2 T3 1.Grasping 1 0 0 2.Touch 0 1 0 3.Haulage 1 1 0 4.Insertion 1 1 1(CW) 5.Extraction 1 1 -1(CCW)

6.Neutral Else

In this study, (a)a normal operation and a (b)singular operation are defined as follows. A normal operation is a surgical operation performed in the expected manner. The singular operation is assumed to be the following surgical operations: (b-1)the surgical operation performed at a posture in which the operator's elbow is raised, denoted as "Posture", (b-2)the surgical operation performed in the state in which the operator is straining, denoted as "Straining", and (b-3)rough surgical operation performed suddenly by the operator, denoted

*T CCW V TH*

<sup>&</sup>gt; = − <sup>&</sup>lt;

<sup>&</sup>gt; <sup>=</sup>

1( ) 1( )

*V TH*

*else*

*CW V TH*

3 3

*else*

*H L*

the following new features are defined using the features (1) to (3).

For identifying the surgical operation, the following values are defined.

0

 

*i*

*T*

identified as one of the six operations as shown in Table 1.

Discriminant value

**Table 1.** Logical definition of needle driver operation

as "Sudden". These are illustrated in Fig.5.

**3.3. Distinction of singularity of surgical operation** 

Spectrum ratio: Also, in the case where the singular operation is performed, it is expected that change of distribution of the power spectrum can be observed in the SEMG signal. Therefore, ratio of distribution of the power spectrum of the SEMG signal is also employed.

It is well known that the SEMG signal is distributed in the frequency band between 5 Hz to 500 Hz. Therefore, to see the ratio of the spectrum, frequency band is divided into 5 to 250 Hz and 250 to 500 Hz. Thus, the value of spectrum ratio is defined as follows.

$$Fr\_{ch} = Fh\_{ch} \int Fl\_{ch} \tag{6}$$

where

$$\begin{cases} \left. F \right|\_{ch} = \sum\_{k \neq -2}^{N \not\prime 8} \left| F\_{ch(k\!\!f)} \right|^2 & 5 \sim 250Hz \\\\ \left. F \right|\_{ch} = \sum\_{k \neq -N \not\prime 8 + 1}^{N \not\prime 4} \left| F\_{ch(k\!\!f)} \right|^2 & 250 \sim 500Hz \end{cases} \tag{7}$$

and |*Fch*(*kf*)| is spectrum value in frequency *kf* obtained by Fast Fourier Transform (FFT).

#### **3.2. Automatic identification of surgical operation**

The suturing is divided into six operations as shown in Fig.4.


**Figure 4.** Surgical operations for suturing

In addition, to identify the state of operation of the needle driver using a threshold value, the following new features are defined using the features (1) to (3).

$$V\_1 = St\_1 \cdot St\_2 \tag{8}$$

$$V\_2 = \sqrt{\text{St}\_3^2 + \text{St}\_4^2} \tag{9}$$

$$V\_3 = \mathfrak{Q}\_{gimbal} \cdot \mathfrak{Q}\_{stydus} \tag{10}$$

For identifying the surgical operation, the following values are defined.

Computational Intelligence in Electromyography Analysis – 252 A Perspective on Current Applications and Future Challenges

where

disposable.

( ) ( ) ( )

<sup>=</sup> (5)

*ch ch ch Fr Fh Fl* = (6)

(7)

5 ~ 250

250 ~ 500

1 1

Spectrum ratio: Also, in the case where the singular operation is performed, it is expected that change of distribution of the power spectrum can be observed in the SEMG signal. Therefore, ratio of distribution of the power spectrum of the SEMG signal is also employed. It is well known that the SEMG signal is distributed in the frequency band between 5 Hz to 500 Hz. Therefore, to see the ratio of the spectrum, frequency band is divided into 5 to 250

Hz and 250 to 500 Hz. Thus, the value of spectrum ratio is defined as follows.

<sup>8</sup> <sup>2</sup> ( )

8 1

= +

1. Grasping: the grasping state by closing the gripper of the needle driver.

<sup>4</sup> <sup>2</sup> ( )

and |*Fch*(*kf*)| is spectrum value in frequency *kf* obtained by Fast Fourier Transform (FFT).

*Fl F Hz*

*Fh F Hz*

3. Haulage: the state where the needle driver touches the object with grasping the needle

2

=

*N ch ch kf kf N ch ch kf kf N*

 <sup>=</sup> <sup>=</sup> 

**3.2. Automatic identification of surgical operation** 

The suturing is divided into six operations as shown in Fig.4.

2. Touch: the state where the needle driver touches the objects.

4. Insertion: the state where the needle disposable is inserted. 5. Extraction: the state where the needle disposable is extracted.

6. Neutral: the state where nothing is operating.

**Figure 4.** Surgical operations for suturing

*N N ch ch n ch n n n cog n EMG EMG* = =

$$T\_i = \begin{cases} 1 & V\_i > TH\_i \\ 0 & \text{else} \end{cases}, \quad \text{( $i=1,2$ )}\tag{11}$$

$$T\_3 = \begin{cases} 1 & \text{(CCW)} \quad V\_3 > TH\_{3H} \\ -1 & \text{(CCW)} \quad V\_3 < TH\_{3L} \\ 0 & \text{else} \end{cases} \tag{12}$$

where *THi* (*i*=1,2,3*H*, 3*L*) is threshold value for each new feature determined through trial and error. On the basis of the threshold criteria for the six operations, a surgical operation is identified as one of the six operations as shown in Table 1.


**Table 1.** Logical definition of needle driver operation

#### **3.3. Distinction of singularity of surgical operation**

In this study, (a)a normal operation and a (b)singular operation are defined as follows. A normal operation is a surgical operation performed in the expected manner. The singular operation is assumed to be the following surgical operations: (b-1)the surgical operation performed at a posture in which the operator's elbow is raised, denoted as "Posture", (b-2)the surgical operation performed in the state in which the operator is straining, denoted as "Straining", and (b-3)rough surgical operation performed suddenly by the operator, denoted as "Sudden". These are illustrated in Fig.5.

Computational Intelligence in Electromyography Analysis – 254 A Perspective on Current Applications and Future Challenges

**Figure 5.** Experimental situations for surgical operation

The surgical operation of (4) insertion of a needle in suturing is classified as either normal or singular by using a self-organizing map: SOM. For classifying the surgical operation, the feature vector which is input to the SOM, is defined as follows using the features (4) to (6).

$$\text{Xs} = \left( \frac{\text{MAV}\_1}{\text{MAV}}, \frac{\text{MAV}\_2}{\text{MAV}}, \frac{\text{MAV}\_3}{\text{MAV}}, \frac{\text{cog}\_2}{\text{cog}\_1}, \frac{\text{cog}\_3}{\text{cog}\_1}, \frac{\text{cog}\_3}{\text{cog}\_2}, Fr\_1, Fr\_2, Fr\_3 \right)^T \tag{13}$$

Distinction of Abnormality of Surgical Operation on the Basis of Surface EMG Signals 255

A: Actual operation times 19 times B: Recognition count 21 times

Difference: |A-B| 2 False recognition rate: |A-B|/A\*100 10.5%

Recognition rate 89.5%

B: Recognition count

Recognition rate

Times which was not counted although operation was performed. 0 Times which was counted although operation was not performed. 2

operation times

2.Touch Non

1.Grasping 8 times 9 times 87.5%

3.Haulage 30 times 36 times 80.0% 4.Insertion 19 times 21 times 89.5% 5.Extraction 19 times 22 times 84.2% 6.Neutral 6 times 6 times 100%

**Figure 6.** Suturing performed in experiment

**4.2. Result for automatic identification** 

**Table 2.** Recognition rate for insertion operation

**Table 3.** Recognition rate for automatic identification

Recognition rate for other operations is shown in Table 3.

Operation A: Actual

Recognition rate for insertion is shown in Table 2.

Where *MAV* is an average of *MAVch* (*ch*=1,2,3).

In each state shown in Fig.5, 20 features for normal operation and 60 features for singular operation (20 features for each singular operation) were pre-measured, and total 80 feature vectors defined by (13) are used for batch learning of the SOM. The size of the SOM was determined as hexagon lattice type of 10 x 10.

In addition, k-means method was employed to divide the map into four fields, namely, (a)Normal, (b-1)Posture, (b-2)Straining and (b-3)Sudden.

A feature vector extracted from on-line surgical operation is mapped on the map of the learned SOM, and singular operation is recognized by the distribution on the map. In addition, SOM was built using SOM Toolbox.
