**2. Related research**

The process of computing and generating forces in response to user interactions with virtual objects is known as haptic rendering (Salisbury et al., 1995). The application of haptic rendering algorithms to complex contact scenarios becomes a challenging issue, due to the inherent cost of collision detection that induces slow force updates. The haptic display of virtual objects has been an active area of research throughout the last decade. Previous research in haptic rendering can be mainly classified within two groups: penalty and constraint-based methods.

When it comes to penalty methods, collision response is computed as a function of object separation or penetration depth. McNeely et al. (1999) proposed point-voxel sampling, a discretized approximation technique for contact queries that generates points on moving objects and voxels on static geometry. This approximation algorithm offers run-time performance independent of the environment's input size by sampling the object geometry at a resolution that the given processor can handle. Renz et al. (2001) adapted this method with several modifications for smoother and more stable haptic feedback. In another research project, Gregory et al. (2000) presented a 6-DOF haptic rendering system that combined collision detection based on convex decomposition of polygonal models, predictive estimation of penetration depth and force and torque interpolation. Kim et al. (2003) attempted to increase the stability of force feedback by using contact clustering, but their algorithm for contact queries suffers from the same computational complexity.

Otaduy and Lin (2003) have presented a sensation preserving simplification technique for 6-DOF haptic rendering of complex polygonal models by adaptively selecting contact resolutions. Later, they have also presented a modular algorithm for 6-DOF haptic 2 Will-be-set-by-IN-TECH

the system and to present rigid objects with reasonable stiffness (Shimoga, 1992). However, this update rate is often difficult to reach by haptic rendering methods, especially when working in complex environments. One possible solution is to reduce the computational cost of calculating the haptic response by decreasing the accuracy of the method. However, this can result in the emergence of discontinuities in the response. This leads to find a trade-off between the accuracy of the method, which guarantees a smooth and stable haptic response,

This chapter describes a haptic rendering method to properly compute interacting forces and torques in complex environments, ensuring improved feedback by seeking a compromise between continuity and computational cost. In addition, the proposed method pays particular attention to provide users with comfortable interaction. The method is valid for applications in which the virtual environment is composed of rigid and static objects,

The remainder of this chapter is organized as follows: Section 2 presents an overview of the related research on the area. Afterwards, Section 3 describes the haptic rendering method proposed by the authors, describing in detail all algorithms necessary to render appropriate and stable forces and torques to the user. The proposed method is then evaluated in Section 4 within two different virtual scenarios simulating common collisions during aeronautic maintainability tasks. Aeronautic virtual mock-ups have been selected for algorithms testing due to their high interaction complexity. Finally, conclusions and future

The process of computing and generating forces in response to user interactions with virtual objects is known as haptic rendering (Salisbury et al., 1995). The application of haptic rendering algorithms to complex contact scenarios becomes a challenging issue, due to the inherent cost of collision detection that induces slow force updates. The haptic display of virtual objects has been an active area of research throughout the last decade. Previous research in haptic rendering can be mainly classified within two groups: penalty and

When it comes to penalty methods, collision response is computed as a function of object separation or penetration depth. McNeely et al. (1999) proposed point-voxel sampling, a discretized approximation technique for contact queries that generates points on moving objects and voxels on static geometry. This approximation algorithm offers run-time performance independent of the environment's input size by sampling the object geometry at a resolution that the given processor can handle. Renz et al. (2001) adapted this method with several modifications for smoother and more stable haptic feedback. In another research project, Gregory et al. (2000) presented a 6-DOF haptic rendering system that combined collision detection based on convex decomposition of polygonal models, predictive estimation of penetration depth and force and torque interpolation. Kim et al. (2003) attempted to increase the stability of force feedback by using contact clustering, but their algorithm for

Otaduy and Lin (2003) have presented a sensation preserving simplification technique for 6-DOF haptic rendering of complex polygonal models by adaptively selecting contact resolutions. Later, they have also presented a modular algorithm for 6-DOF haptic

contact queries suffers from the same computational complexity.

and the computational cost.

excluding deformable objects.

directions are drawn in Section 5.

**2. Related research**

constraint-based methods.

rendering, that provides transparent manipulation of rigid models with a high polygon count (Otaduy & Lin, 2006).

Unlike penalty methods, constraint-based methods do not use the interpenetration between rigid objects to calculate collision response. These methods use virtual coupling techniques (Colgate et al., 1995) and restrict the movement of virtual objects on the surface of obstacles.

Zilles and Salisbury (1995) proposed a constraint-based method for 3-DOF haptic rendering of generic polygonal objects. They introduced the "god-object", an idealized representation of the position of the haptic device that is constrained on the surface of obstacles. At each time step, the location of the god-object minimizes the distance to the haptic device and the difference between the two positions provides the force direction. Ruspini et al. (1997) extended this approach by replacing the god-object with a small sphere as well as proposing methods to smooth the object surface and add friction. Later, Ortega (2006) extended the 3-DOF constraint-based method of Zilles and Salisbury by employing a 6-DOF god-object.

The different haptic rendering methods described above have contributed extensively to a better representation of contact events between virtual objects. However, haptic interactions with multiple contacts, which also include geometrical discontinuities, have not yet been adequately accomplished computing unrealistic or unstable haptic feedback in these cases. These type of situations are very common in real scenarios and therefore it is necessary to compute properly a stable haptic response in order to improve the usability of these systems. The proposed haptic rendering method overcomes limitations from previous approaches in this type of collisions.

#### **3. Proposed haptic rendering method**

The haptic rendering method outlined in this chapter computes the force and torque that result when a collision between two type of objects occurs: a virtual tool (mobile object) manipulated by the user of the haptic device and any object in the simulation (static object). Three main modules can be identified in the haptic rendering method proposed (Fig. 2): collision detection, collision response and control module.

The complete haptic rendering sequence could be described as follows: firstly, the control module acquires the position (*Uh*) and orientation (*Rh*) of the haptic device and sends it to the collision detection module. With this information, the module checks for collisions between the mobile object and the static environment. If no collisions occur, it waits for new information from the control module. Otherwise, when a collision event occurs, the contact information of both static and mobile objects (**C***s*, **C***m*) is sent to the collision response module which calculates the interaction force and torque. This haptic feedback approximates the contact force and torque that would arise during contact between real objects (*Fr*, *Tr*). Finally, the collision response module sends this information to the control module, which applies it to the haptic device (*Fh*, *Th*), maintaining a stable system behaviour.

A more complete description of each module can be found in the following sections.

#### **3.1 Collision detection**

The collision detection method presented in this chapter can handle non-convex objects without modifying the original mesh. A technique based on a spatial partition (voxels) has been chosen. Hierarchical methods like octrees have also been tested since they require less

for Complex Interactions 5

Effective Haptic Rendering Method for Complex Interactions 119

Regarding the static object, *BST* set of triangles are those in contact with *Vbnd* (Fig. 5), but that do not intersect with mobile triangles. These triangles are usually in the centre of the contact

> Colliding mobile triangles (**CMT**)

(a) (b)

Internal mobile triangles (**IMT**)

> Contact mobile triangles (**CMT**)

Contact mobile

Fig. 3. Colliding triangles detected (a) and additional triangles that define the intersection

**Intersection volume**

> Boundary voxels (*Vbnd*)

Boundary mobile triangles (**BMT**)

Boundary static triangles (**BST**)

**Mobile triangles not detected**

Contact static triangles (**CMT**) triangles (**CST**)

area surrounded by colliding static triangles.

Colliding static triangles (**CST**)

Internal voxels (*Vint*)

Fig. 4. Internal and boundary mobile triangles

Contact static triangles (**CST**)

Fig. 5. Static boundary triangles

volume (b)

**Haptic Device**

memory storage. However, their computation time is higher than that needed for direct access voxel techniques.

It is well-known that algorithms based on voxel techniques have several disadvantages, such as high memory storage requirements and the selection of voxel size. According to previous experiments (Borro et al., 2004), hashing techniques can solve the first problem by reducing memory storage up to 60 % without performance loss, and the choice of an optimal voxel size can be solved by means of an analytical solution based on the algorithm cost function.

The method, in a pre-process, computes a voxel partition from the virtual scene and assigns each triangle of the static environment to its corresponding voxel. This voxel partition is used only for the static object. In addition, each voxel will have a flag identifying it as internal (*Vint*), external (*Vext*) or boundary (*Vbnd*). The last ones contain the triangles that define the surface of the static object.

Next, at runtime, the partition model is used to detect the set of voxels in collision with the mobile object and to carry out interference checks between triangles.

Fig. 3a shows an example of colliding static triangles (*CST*) and colliding mobile triangles (*CMT*) detected by the method in a virtual collision of a tool with an obstacle. As it can be seen colliding triangles do not provide enough information to delimit the volume that defines the intersection between the objects. Therefore, unlike other existing methods in the literature, the proposed algorithm detects additional triangles in order to calculate the intersection volume correctly (Fig. 3b). The union of the colliding triangles and these additional triangles are referred to as *contact triangles*.

The additional triangles can be classified within three groups: internal mobile (*IMT*), boundary mobile (*BMT*) and boundary static (*BST*). With regard to the mobile object, (*IMT*) are those in contact with *Vint* whereas (*BMT*) are those in contact with *Vbnd*, but that do not intersect with static triangles (*CST*) (Fig. 4).

4 Will-be-set-by-IN-TECH

**Cs**, **Cm**

*Fh*, *Th*

memory storage. However, their computation time is higher than that needed for direct access

It is well-known that algorithms based on voxel techniques have several disadvantages, such as high memory storage requirements and the selection of voxel size. According to previous experiments (Borro et al., 2004), hashing techniques can solve the first problem by reducing memory storage up to 60 % without performance loss, and the choice of an optimal voxel size can be solved by means of an analytical solution based on the algorithm cost function. The method, in a pre-process, computes a voxel partition from the virtual scene and assigns each triangle of the static environment to its corresponding voxel. This voxel partition is used only for the static object. In addition, each voxel will have a flag identifying it as internal (*Vint*), external (*Vext*) or boundary (*Vbnd*). The last ones contain the triangles that define the

Next, at runtime, the partition model is used to detect the set of voxels in collision with the

Fig. 3a shows an example of colliding static triangles (*CST*) and colliding mobile triangles (*CMT*) detected by the method in a virtual collision of a tool with an obstacle. As it can be seen colliding triangles do not provide enough information to delimit the volume that defines the intersection between the objects. Therefore, unlike other existing methods in the literature, the proposed algorithm detects additional triangles in order to calculate the intersection volume correctly (Fig. 3b). The union of the colliding triangles and these additional triangles are

The additional triangles can be classified within three groups: internal mobile (*IMT*), boundary mobile (*BMT*) and boundary static (*BST*). With regard to the mobile object, (*IMT*) are those in contact with *Vint* whereas (*BMT*) are those in contact with *Vbnd*, but that do not

mobile object and to carry out interference checks between triangles.

**Collision Response** 

2. Identify contact areas and zones 3. Compute force direction 4. Compute penetration depth 5. Compute contact point 6. Compute force and torque

**Control**

*Fr*, *Tr*

7. Update collision information

8. Adequate response

**Collision Detection**

1. Detect colliding triangles

**Haptic Device**

voxel techniques.

surface of the static object.

referred to as *contact triangles*.

intersect with static triangles (*CST*) (Fig. 4).

Fig. 2. Diagram of haptic rendering algorithm

*Uh*, *Rh*

Regarding the static object, *BST* set of triangles are those in contact with *Vbnd* (Fig. 5), but that do not intersect with mobile triangles. These triangles are usually in the centre of the contact area surrounded by colliding static triangles.

Fig. 3. Colliding triangles detected (a) and additional triangles that define the intersection volume (b)

Fig. 5. Static boundary triangles

for Complex Interactions 7

Effective Haptic Rendering Method for Complex Interactions 121

Contact area 1

Spheres

**Contact zone 2.1 Contact zone 2.2**

contact has occurred in an area of the static object that is not a continuous surface. In this case, static triangles do not provide enough information to obtain a suitable force direction. Therefore, **nc** is computed as the normalized sum of all normal vectors of the mobile triangles of that area. This solution enables smoother direction transitions when interacting with sharp

After computing the force direction in each area, the following step is the calculation of the minimum distance required to separate the two objects, known as penetration depth (*d*). For that purpose, the method samples the volume of intersection measuring heights throughout this volume to determine the penetration between two objects. These heights are determined by tracing a ray from the centroid of each static triangle in **nc** direction (Fig. 8). If this ray intersects with a mobile triangle (Möller & Trumbore, 1997), the height is defined as the

Fig. 7. Example of two contact areas. The second one has two contact zones

**Contact area 2**

Fig. 6. Example of two contact areas

edges.

**Contact area 1**

Contact area 2

Bounding boxes

At the end, the collision detection module yields two sets of contact triangles, one belonging to the static object (**C***s*) and the other to the mobile object (**C***m*).

$$\mathcal{C}\_s = \{\text{CST} \bigcup \text{BST}\} \tag{1}$$

$$\mathcal{C}\_m = \{\mathsf{CMT} \bigcup \mathsf{BMT} \bigcup \mathsf{IMT}\} \tag{2}$$

#### **3.2 Collision response**

The algorithm proposed follows the well-known penalty methods based on elastic model, thus the haptic response that users feel as a consequence of a collision in the virtual environment is determined by a direction and a penetration value. Both factors have substantial influence on user perception.

The force (*Fr*) and torque (*Tr*) are calculated as follows:

$$\begin{aligned} \mathbf{F}\_r &= \mathbf{K}d\mathfrak{n} \\ \mathbf{T}\_r &= (\mathbf{c}\mathfrak{p} - \mathbf{g}\mathbf{c}) \times \mathbf{F}\_r \end{aligned} \tag{3}$$

where *K* is virtual stiffness, *d* is penetration depth, *n* force direction, *cp* the contact point and *gc* the centre of mass of the virtual tool.

In complex scenarios, it is common to encounter multiple contacts. The lists of triangles (**C***s* and **C***m*) obtained by the collision detection module do not give a priory information about the number of different contacts. Therefore, the method divides these sets of triangles into different contact areas considering their spatial proximity. The forces and torques of each area are then added up to compute the net force and torque.

A certain amount of static triangles form a contact area when they are adjacent. In other words, they share at least one edge. Once the different contact areas have been calculated from **C***s*, it is necessary to identify the mobile triangles in contact that are associated with those areas. To facilitate this process, a sphere is created in each contact area that covers the bounding box defined by its triangles. Finally, each triangle of **C***m* is associated with the contact area depending on the sphere which contains it. Fig. 6 shows an example of the division of contact areas.

The difficulty in calculating an appropriate and stable haptic response increases when the geometry has sharp edges, since haptic instabilities often appear due to abrupt force direction or penetration depth changes. In order to detect these cases, each contact area is sub-divided into different contact zones that provide information about the nature of the geometry in collision.

The collision response module subdivides a collision area into different contact zones taking surface *C*<sup>1</sup> discontinuities into account. When two triangles share an edge and the angle between their normal vectors is lower than a fixed value, the edge is designated as "smooth". Triangles in a contact zone must be interconnected and all the shared edges must be smooth. There will be as many contact zones as necessary to satisfy the condition of smooth connectivity (Fig. 7).

Once the contact areas and zones have been detected, the method computes the contact normal vector (force direction) of each area (**nc**). If the contact area has a unique contact zone, the contact normal is computed as the normalized sum of all normal vectors of the static triangles of that zone. Otherwise, when a contact area has two or more contact zones, the 6 Will-be-set-by-IN-TECH

At the end, the collision detection module yields two sets of contact triangles, one belonging

The algorithm proposed follows the well-known penalty methods based on elastic model, thus the haptic response that users feel as a consequence of a collision in the virtual environment is determined by a direction and a penetration value. Both factors have

*T<sup>r</sup>* = (*cp* − *gc*) × *F<sup>r</sup>*

where *K* is virtual stiffness, *d* is penetration depth, *n* force direction, *cp* the contact point and

In complex scenarios, it is common to encounter multiple contacts. The lists of triangles (**C***s* and **C***m*) obtained by the collision detection module do not give a priory information about the number of different contacts. Therefore, the method divides these sets of triangles into different contact areas considering their spatial proximity. The forces and torques of each area

A certain amount of static triangles form a contact area when they are adjacent. In other words, they share at least one edge. Once the different contact areas have been calculated from **C***s*, it is necessary to identify the mobile triangles in contact that are associated with those areas. To facilitate this process, a sphere is created in each contact area that covers the bounding box defined by its triangles. Finally, each triangle of **C***m* is associated with the contact area depending on the sphere which contains it. Fig. 6 shows an example of the division of contact

The difficulty in calculating an appropriate and stable haptic response increases when the geometry has sharp edges, since haptic instabilities often appear due to abrupt force direction or penetration depth changes. In order to detect these cases, each contact area is sub-divided into different contact zones that provide information about the nature of the geometry in

The collision response module subdivides a collision area into different contact zones taking surface *C*<sup>1</sup> discontinuities into account. When two triangles share an edge and the angle between their normal vectors is lower than a fixed value, the edge is designated as "smooth". Triangles in a contact zone must be interconnected and all the shared edges must be smooth. There will be as many contact zones as necessary to satisfy the condition of smooth

Once the contact areas and zones have been detected, the method computes the contact normal vector (force direction) of each area (**nc**). If the contact area has a unique contact zone, the contact normal is computed as the normalized sum of all normal vectors of the static triangles of that zone. Otherwise, when a contact area has two or more contact zones, the

*F<sup>r</sup>* = *Kdn*

*<sup>C</sup><sup>s</sup>* <sup>=</sup> {*CST BST*} (1) *<sup>C</sup><sup>m</sup>* <sup>=</sup> {*CMT BMT IMT*} (2)

(3)

to the static object (**C***s*) and the other to the mobile object (**C***m*).

**3.2 Collision response**

areas.

collision.

connectivity (Fig. 7).

substantial influence on user perception.

*gc* the centre of mass of the virtual tool.

The force (*Fr*) and torque (*Tr*) are calculated as follows:

are then added up to compute the net force and torque.

Fig. 6. Example of two contact areas

Fig. 7. Example of two contact areas. The second one has two contact zones

contact has occurred in an area of the static object that is not a continuous surface. In this case, static triangles do not provide enough information to obtain a suitable force direction. Therefore, **nc** is computed as the normalized sum of all normal vectors of the mobile triangles of that area. This solution enables smoother direction transitions when interacting with sharp edges.

After computing the force direction in each area, the following step is the calculation of the minimum distance required to separate the two objects, known as penetration depth (*d*). For that purpose, the method samples the volume of intersection measuring heights throughout this volume to determine the penetration between two objects. These heights are determined by tracing a ray from the centroid of each static triangle in **nc** direction (Fig. 8). If this ray intersects with a mobile triangle (Möller & Trumbore, 1997), the height is defined as the

for Complex Interactions 9

Effective Haptic Rendering Method for Complex Interactions 123

**Workspace**

**Degrees of freedom**

**Peak output level**

**Actuators**

**Encoders**

<sup>Δ</sup>*Mh*(*i*) = *<sup>M</sup>r*(*<sup>i</sup>* <sup>−</sup> *<sup>j</sup>*) <sup>−</sup> *<sup>K</sup>*(*Uh*(*i*) <sup>−</sup> *<sup>U</sup>h*(*<sup>i</sup>* <sup>−</sup> *<sup>j</sup>*)) <sup>−</sup> *<sup>M</sup>h*(*<sup>i</sup>* <sup>−</sup> <sup>1</sup>)

In the next sampling period (*i* + 1), the method also takes the new movements performed by

This will continue until the collision response method updates the collision-related information. To determine *n*, the optimal value should approximate the number of control sampling periods that the collision module needs to compute the response. Since this number can vary significantly depending on the number of triangles in collision, a conservative number can be set. However, if *n* is very high, the method may excessively filter the signal. Moreover, *n* can have a fixed value during the entire task or can be modified by means of the

Two different virtual scenarios have been designed to test the effectiveness and stability of the proposed haptic rendering algorithm on complex interactions. A 3-DOF torque feedback device has been used for the experiments. The mechanism was designed and built at CEIT and inspired by past research (Angerilli et al., 2001). Fig. 9 shows the device and its main

The system is controlled by a dSPACE DS1104 board that reads encoder information, processes the haptic control loop and outputs torque commands to the motors. Graphic rendering and

*n*

*Mh*(*i* + 1) = *Mh*(*i*) + Δ*Mh*(*i*) − *K*(*Uh*(*i* + 1) − *Uh*(*i*)) (6)

Encoder pitch DOF

Fig. 9. 3-DOF torque feedback device used for the experiments

*Mh*(*i*) = *Mh*(*i* − 1) + Δ*Mh*(*i*)

average collision response delay in previous sampling periods.

Motor+gearhead

Roll

Roll DOF axis

Cable transmission

the user into account:

**4. Implementation and results**

specifications.

Yaw DOF axis

Pitch DOF axis

pitch DOF Motor pitch DOF

them in *n* subsequent sampling periods:

Pitch-Yaw ±70 º Roll Unlimited **Continuous output level** Pitch-Yaw 0.7 Nm Roll 0.2 Nm

Pitch-Yaw 1.6 Nm Roll 0.9 Nm

Sensing input 3 DOF (Pitch-Yaw-Roll) Feedback output 3 DOF (Pitch-Yaw-Roll)

Pitch-Yaw DC MaxonRE35, 90 W

Roll DC MaxonRE-max21, 6 W

Pitch-Yaw Quantum Devices QD145, 5000 ppr Roll MaxonMR TypeM, 512 ppr

+ Cable Transmissions 13.5:1

+ Maxon Planetary GP22C 29:1

(5)

distance between the centroid and the intersection point. The final value of penetration for each area is computed as an average of all the computed heights.

Fig. 8. Rays traced to compute penetration depth

The last parameter necessary to compute collision torque is the contact point (**cp**). Selecting an inappropriate point such as the most penetrating one for each contact area might lead to non-continuous changes in the haptic feedback (Hasegawa & Sato, 2004). To avoid this effect, the proposed solution is to choose a representative contact point for each contact area. This point is calculated as the average of the midpoints of the boundary voxels associated with each contact area.

#### **3.3 Control module**

The control module receives the ideal interaction force and torque calculated by the collision response method, adapts them to the device's capabilities and applies them to the user at a 1 kHz sampling rate. Since the control loop runs faster than the collision module, several strategies must be implemented in order to avoid abrupt changes in contact force and torque when collision information is not available (Savall et al. (2002)).

There are two main problems that should be taken into account. The first one is the delay that exists in the collision-related information. This information, calculated by the response module, is valid for a previous user position, but not the actual one. The second problem resides in the existence of some control loops without collision-related information, since the response module is slower than the control loop. In order to deal with these problems, a strategy based on intermediate representations (Adachi et al. (1995)) is implemented.

Let *Mh*=(*Fh*, *Th*) be the six-dimensional vector of the force and torque that we want to apply to the user and *Xh*=(*Uh*, *Rh*) be the six-dimensional configuration of the haptic device (*U<sup>h</sup>* represents the position, and *R<sup>h</sup>* the axis angle representation of the rotation). First, the control method updates the force and torque (*Mr*) computed by the response module at a previous sampling period *j*, to the current one *i* (*K* represents virtual stiffness):

$$\mathbf{M}\_h(i) = \mathbf{M}\_r(i-j) - \mathbf{K}(\mathbf{U}\_h(i) - \mathbf{U}\_h(i-j)) \tag{4}$$

These forces and torques might vary quite abruptly if they are applied to the user every time they are updated by the collision response method. Therefore, the control module restores 8 Will-be-set-by-IN-TECH

distance between the centroid and the intersection point. The final value of penetration for

Heights triangles

The last parameter necessary to compute collision torque is the contact point (**cp**). Selecting an inappropriate point such as the most penetrating one for each contact area might lead to non-continuous changes in the haptic feedback (Hasegawa & Sato, 2004). To avoid this effect, the proposed solution is to choose a representative contact point for each contact area. This point is calculated as the average of the midpoints of the boundary voxels associated with

The control module receives the ideal interaction force and torque calculated by the collision response method, adapts them to the device's capabilities and applies them to the user at a 1 kHz sampling rate. Since the control loop runs faster than the collision module, several strategies must be implemented in order to avoid abrupt changes in contact force and torque

There are two main problems that should be taken into account. The first one is the delay that exists in the collision-related information. This information, calculated by the response module, is valid for a previous user position, but not the actual one. The second problem resides in the existence of some control loops without collision-related information, since the response module is slower than the control loop. In order to deal with these problems, a

Let *Mh*=(*Fh*, *Th*) be the six-dimensional vector of the force and torque that we want to apply to the user and *Xh*=(*Uh*, *Rh*) be the six-dimensional configuration of the haptic device (*U<sup>h</sup>* represents the position, and *R<sup>h</sup>* the axis angle representation of the rotation). First, the control method updates the force and torque (*Mr*) computed by the response module at a previous

These forces and torques might vary quite abruptly if they are applied to the user every time they are updated by the collision response method. Therefore, the control module restores

*Mh*(*i*) = *Mr*(*i* − *j*) − *K*(*Uh*(*i*) − *Uh*(*i* − *j*)) (4)

strategy based on intermediate representations (Adachi et al. (1995)) is implemented.

Contact mobile triangles

Contact static

each area is computed as an average of all the computed heights.

Fig. 8. Rays traced to compute penetration depth

when collision information is not available (Savall et al. (2002)).

sampling period *j*, to the current one *i* (*K* represents virtual stiffness):

each contact area.

**3.3 Control module**

Fig. 9. 3-DOF torque feedback device used for the experiments

them in *n* subsequent sampling periods:

$$\begin{aligned} \Delta \mathbf{M}\_{\hbar}(i) &= \frac{\mathbf{M}\_{\hbar}(i-j) - \mathbf{K}(\mathbf{U}\_{\hbar}(i) - \mathbf{U}\_{\hbar}(i-j)) - \mathbf{M}\_{\hbar}(i-1)}{n} \\ \mathbf{M}\_{\hbar}(i) &= \mathbf{M}\_{\hbar}(i-1) + \Delta \mathbf{M}\_{\hbar}(i) \end{aligned} \tag{5}$$

In the next sampling period (*i* + 1), the method also takes the new movements performed by the user into account:

$$\mathbf{M}\_h(i+1) = \mathbf{M}\_h(i) + \Delta \mathbf{M}\_h(i) - \mathbf{K}(\mathbf{U}\_h(i+1) - \mathbf{U}\_h(i)) \tag{6}$$

This will continue until the collision response method updates the collision-related information. To determine *n*, the optimal value should approximate the number of control sampling periods that the collision module needs to compute the response. Since this number can vary significantly depending on the number of triangles in collision, a conservative number can be set. However, if *n* is very high, the method may excessively filter the signal. Moreover, *n* can have a fixed value during the entire task or can be modified by means of the average collision response delay in previous sampling periods.

#### **4. Implementation and results**

Two different virtual scenarios have been designed to test the effectiveness and stability of the proposed haptic rendering algorithm on complex interactions. A 3-DOF torque feedback device has been used for the experiments. The mechanism was designed and built at CEIT and inspired by past research (Angerilli et al., 2001). Fig. 9 shows the device and its main specifications.

The system is controlled by a dSPACE DS1104 board that reads encoder information, processes the haptic control loop and outputs torque commands to the motors. Graphic rendering and

for Complex Interactions 11

Effective Haptic Rendering Method for Complex Interactions 125

**1 2 3**

0 500 1000 1500 2000 2500 3000 3500 4000 4500 5000 5500

0 500 1000 1500 2000 2500 3000 3500 4000 4500 5000 5500

0 500 1000 1500 2000 2500 3000 3500 4000 4500 5000 5500

0 500 1000 1500 2000 2500 3000 3500 4000 4500 5000 5500

0 500 1000 1500 2000 2500 3000 3500 4000 4500 5000 5500

Frame

Fig. 10. Performance of our approach in multiple contacts and geometrical discontinuities

**1 2 3**


*Tx* (Nm)

*Ty* (Nm)

*Tz* (Nm)

Triangles

Computation time (ms)

**X**

**Y Z**

> -2 -1 0 1 2


> Static triangles Mobile triangles

collision detection are performed on a PC running the Windows XP operating system with a Pentium Dual Core 6600, 2GB memory and an NVIDIA GeForce 8600 GT.

#### **4.1 Analysis of multiple contacts and geometrical discontinuities**

The first scenario is composed of two parallelepipeds, one central cylinder and a virtual tool similar to a clamp (see top of Fig. 10). The virtual scenario consists of 25, 000 triangles while the clamp is composed of 1, 500 triangles. The aim is to analyze the behavior of the method when multiple contacts and geometrical discontinuities are involved. For that purpose, a sequence of different collisions has been simulated:


Fig. 10 shows the torque feedback computed in all three axis by the collision response method during the sequence. The figure also shows the number of contact triangles detected and the computational cost in each frame.

Notice in the figure that the torque feedback applied to the user during collision is quite smooth and does not offer abrupt changes or discontinuities that may degrade the user's perception of contact. The last figure shows that the collision response method computes the haptic feedback at an average of 2.5 ms, which is not far from the optimal computation time (1 ms) necessary for a realisic haptic experience. This allows the control module to compute the real forces and torques at 1 kHz using a low number of *n* transitions (described in Section 3.3), and thereby maintaining a stable system behaviour. Specifically, for this experiment, the number of transitions *n* for the control algorithm was set to 5.

#### **4.2 Simulation of a disassembly task**

The task designed for the second experiment is similar to the extraction of a clamp from a pipe, which frequently appears in engine disassembly tasks in aeronautics maintenance. Once the clamp is unfastened, the exit path is established by following the spatial trajectory laid out by the pipe itself. Along said path, the curves and bends of the pipe and other obstacles force the clamp to rotate in space. Therefore, in order to accomplish this task properly, it is important that the haptic feedback restored to the user is realistic. The virtual engine mock-up is defined by 100, 000 triangles while the clamp is composed of 2, 000 triangles.

The torque feedback device used in the previous experiment does not allow any translation. Thus, to be able to displace the clamp within the virtual environment, it is necessary to provide the mechanism with a translational DOF. For that purpose, a linear actuator designed and built at CEIT (Savall et al., 2008) is used. The displacement along this linear DOF is mapped into a displacement of the clamp along the axial direction of the pipe, and forces along this DOF are also displayed when collisions occur. Fig. 11 shows both the designed virtual environment and the haptic device used for the experiments.

A virtual path from right to left along the route of extraction was performed. During this process, different types of collisions occurred between the clamp and the environment. Fig. 12 shows an example of the main possible stages during the extraction:

1. Initial position.

10 Will-be-set-by-IN-TECH

collision detection are performed on a PC running the Windows XP operating system with a

The first scenario is composed of two parallelepipeds, one central cylinder and a virtual tool similar to a clamp (see top of Fig. 10). The virtual scenario consists of 25, 000 triangles while the clamp is composed of 1, 500 triangles. The aim is to analyze the behavior of the method when multiple contacts and geometrical discontinuities are involved. For that purpose, a

1. Collision with the central cylinder in which two contact areas are involved. 2. Collision with the corners of the two parallelepipeds (*C*<sup>1</sup> discontinuities). 3. Collision with four contacts simultaneously, combining the previous cases. Fig. 10 shows the torque feedback computed in all three axis by the collision response method during the sequence. The figure also shows the number of contact triangles detected and the

Notice in the figure that the torque feedback applied to the user during collision is quite smooth and does not offer abrupt changes or discontinuities that may degrade the user's perception of contact. The last figure shows that the collision response method computes the haptic feedback at an average of 2.5 ms, which is not far from the optimal computation time (1 ms) necessary for a realisic haptic experience. This allows the control module to compute the real forces and torques at 1 kHz using a low number of *n* transitions (described in Section 3.3), and thereby maintaining a stable system behaviour. Specifically, for this experiment, the

The task designed for the second experiment is similar to the extraction of a clamp from a pipe, which frequently appears in engine disassembly tasks in aeronautics maintenance. Once the clamp is unfastened, the exit path is established by following the spatial trajectory laid out by the pipe itself. Along said path, the curves and bends of the pipe and other obstacles force the clamp to rotate in space. Therefore, in order to accomplish this task properly, it is important that the haptic feedback restored to the user is realistic. The virtual engine mock-up is defined

The torque feedback device used in the previous experiment does not allow any translation. Thus, to be able to displace the clamp within the virtual environment, it is necessary to provide the mechanism with a translational DOF. For that purpose, a linear actuator designed and built at CEIT (Savall et al., 2008) is used. The displacement along this linear DOF is mapped into a displacement of the clamp along the axial direction of the pipe, and forces along this DOF are also displayed when collisions occur. Fig. 11 shows both the designed virtual environment

A virtual path from right to left along the route of extraction was performed. During this process, different types of collisions occurred between the clamp and the environment. Fig. 12

Pentium Dual Core 6600, 2GB memory and an NVIDIA GeForce 8600 GT.

**4.1 Analysis of multiple contacts and geometrical discontinuities**

sequence of different collisions has been simulated:

number of transitions *n* for the control algorithm was set to 5.

by 100, 000 triangles while the clamp is composed of 2, 000 triangles.

shows an example of the main possible stages during the extraction:

computational cost in each frame.

**4.2 Simulation of a disassembly task**

and the haptic device used for the experiments.

1. Initial position.

Fig. 10. Performance of our approach in multiple contacts and geometrical discontinuities

for Complex Interactions 13

Effective Haptic Rendering Method for Complex Interactions 127

**<sup>4</sup> <sup>5</sup>**

11000 10000 9000 8000 7000 6000 5000 4000 3000 2000 1000 0

11000 10000 9000 8000 7000 6000 5000 4000 3000 2000 1000 0

11000 10000 9000 8000 7000 6000 5000 4000 3000 2000 1000 0

11000 10000 9000 8000 7000 6000 5000 4000 3000 2000 1000 0

11000 10000 9000 8000 7000 6000 5000 4000 3000 2000 1000 0

Frame

11000 10000 9000 8000 7000 6000 5000 4000 3000 2000 1000 0

**6 5 4 3 2 1**

**1**

Static triangles Mobile triangles

**2**

**3**

**Z**

**6**

**X Y**

*Tz* (Nm)

*Fx* (Nm)

Triangles

Computation time (ms)

0

2 4

Fig. 12. Extraction of a clamp from a pipe

*Tx* (Nm)

*Ty* (Nm)

Fig. 11. Haptic system used for the experiment

2. Collision of the clamp with the first obstacle. In this case, the linear actuator restores a force in the *X* axis to avoid translational movement.

3. The clamp is inside the obstacle and, when rotating along the *X* axis, it collides with the upper part of the obstacle.


Fig. 12 shows the torque applied to the user in each axis (*Tr\_x*, *Tr\_y* and *Tr\_z*) and the force exerted by the linear actuator (*Fr\_x*) computed by the collision response method. It also indicates the number of triangles in collision, in addition to the computational cost at each frame. Notice that in this figure data are shown from right to left according to the movement of the tool.

Haptic feedback obtained gives a realistic perception of collision events and allows to perform the task properly. Unlike the previous scenario, designed to study the behavior of the method in situations with multiple collisions, in this case the aim is to simulate a real task. For this reason, although the complexity of the environment is higher, the number of simultaneous contacts decreases because the user corrects trajectory when a collision is detected.

Fig. 13 is an augmentation of the third collision stage of Fig. 12 (frames 3450–3650) for torque feedback in the *x* axis, with and without applying the control algorithms. It can be seen that torque computed by the collision response is smooth and avoids abrupt changes. In addition, control algorithms improve the continuity of the feedback signal and apply it to the user at a sampling rate of 1 kHz. As in the previous example, in this case the number of transitions for the control algorithms is also 5.

12 Will-be-set-by-IN-TECH

2. Collision of the clamp with the first obstacle. In this case, the linear actuator restores

3. The clamp is inside the obstacle and, when rotating along the *X* axis, it collides with

4. Collision of the clamp with the pipe when rotating along the *Z* axis.

5. Collision of the clamp with the pipe resulting in complex multi-axis torque.

Fig. 12 shows the torque applied to the user in each axis (*Tr\_x*, *Tr\_y* and *Tr\_z*) and the force exerted by the linear actuator (*Fr\_x*) computed by the collision response method. It also indicates the number of triangles in collision, in addition to the computational cost at each frame. Notice that in this figure data are shown from right to left according to the movement

Haptic feedback obtained gives a realistic perception of collision events and allows to perform the task properly. Unlike the previous scenario, designed to study the behavior of the method in situations with multiple collisions, in this case the aim is to simulate a real task. For this reason, although the complexity of the environment is higher, the number of simultaneous

Fig. 13 is an augmentation of the third collision stage of Fig. 12 (frames 3450–3650) for torque feedback in the *x* axis, with and without applying the control algorithms. It can be seen that torque computed by the collision response is smooth and avoids abrupt changes. In addition, control algorithms improve the continuity of the feedback signal and apply it to the user at a sampling rate of 1 kHz. As in the previous example, in this case the number of transitions for

contacts decreases because the user corrects trajectory when a collision is detected.

Fig. 11. Haptic system used for the experiment

the upper part of the obstacle.

the control algorithms is also 5.

of the tool.

a force in the *X* axis to avoid translational movement.

6. Final position. The clamp is disassembled.

Fig. 12. Extraction of a clamp from a pipe

for Complex Interactions 15

Effective Haptic Rendering Method for Complex Interactions 129

Adachi, Y., Kumano, T. & Ogino, K. (1995). Intermediate representation for stiff virtual objects,

Angerilli, M., Frisoli, A., Salsedo, F., Marcheschi, S. & Bergamasco, M. (2001). Haptic

Basdogan, C., De, S., Kim, J., Muniyandi, M., Kim, H. & Srinivasan, M. (2004). Haptics in

Borro, D., García-Alonso, A. & Matey, L. (2004). Approximation of optimal voxel

Colgate, J., Stanley, M. & Brown, J. (1995). Issues in the haptic display of tool use, *IEEE*

Gregory, A., Mascarenhas, A., Ehmann, S., Lin, M. & Manocha, D. (2000). Six

Hasegawa, S. & Sato, M. (2004). Real-time rigid body simulation for haptic interactions based on contact volume of polygonal objects, *Computer Graphics Forum* 23(3): 529–538.

simulation of an automotive manual gearshift, *10th IEEE International Workshop on*

minimally invasive surgical simulation and training, *IEEE Computer Graphics and*

size for collision detection in maintainability simulations within massive virtual

*International Conference on Intelligent Robot and Systems*, Vol. 3, Pittsburgh, PA, USA,

degree-of-freedom haptic display of polygonal models, *IEEE Conference on*

*IEEE Virtual Reality Annual International Symposium* pp. 203–210.

*Robot and Human Interactive Communication*, pp. 170–175.

environments, *Computer Graphics Forum* 23(1): 13–23.

*Visualization*, Salt Lake City, Utah, United States, pp. 139–146.

• **X***<sup>h</sup>* : User displacement and rotation of the haptic handle

• **M***<sup>h</sup>* : Force and torque feedback applied to the user

• *d* : Penetration of the mobile tool within a static object

• *n* : Number of transitions for the control algorithm

• **F***<sup>h</sup>* : Force feedback applied to the user • **T***<sup>h</sup>* : Torque feedback applied to the user

• **C***s* : List of static triangles in collision • **C***m* : List of mobile triangles in collision

• **gc** : Centre of mass of the virtual tool

• **n***c* : Force direction of each contact ares

*Applications* 24(2): 56–64.

pp. 140–145.

• **cp** : Collision contact point

• *K* : Virtual object stiffness

• **n** : Force direction

**7. References**

• **V***int* : Internal voxels • **V***ext* : External voxels • **V***bnd* : Boundary voxels

• **F***r* : Collision force computed by the collision response method • **T***r* : Collision torque computed by the collision response method

• **M***r* : Collision force and torque computed by the collision response method

Fig. 13. An augmentation of the third collision stage for torque feedback in the X axis, with and without applying the control algorithms

#### **5. Conclusions and future research**

The real-time computation of the forces and torques in a virtual environment is a complicated task but a key point for the effectiveness of haptic systems. It is known that non-realistic or inappropriate haptic feedback has negative effects on the usability and leads to frustration when manipulating haptic systems. Therefore, it is very important to guarantee smooth and realistic haptic feedback.

This chapter outlines a haptic rendering method that computes a proper haptic response in complex environments. It ensures improved feedback by seeking a compromise between continuity and computational cost. The method avoids abrupt changes in the haptic force direction and magnitude, thereby improving the overall stability of the haptic system.

In order to validate the proposed method, two different scenarios containing complex collision examples, such as multiple contacts and geometrical discontinuities, have been used. The yielded results validate the applicability of the method in these types of interactions.

In terms of future research, the authors plan to analyze the performance of the method from a perceptual perspective carrying out studies of human factors to improve the responsiveness. The authors also hope that the research included in this chapter will provide a better understanding of the many phenomena that challenge the development of improved haptic rendering methods able to display adequate force and torque feedback while preserving stability, and thereby improve performance of current haptic interfaces.
