**3.1 Assisting to sketch using fixed templates**

In order to present fixed templates to the users, we have developed an application that transforms the principal edges of a given input image file (jpg, bmp, tiff, etc.) into trajectories for later HDS interpretation, this way the "trajectories" are generated without any kind pre programming. The first process applied to the image is a median filter to remove the possible noise present, then a equalization and binarization is done (Pratt, 1991), after that, to detect its edges a Canny edge detector (Canny, 1986) is used, then the resulting

Furthermore, the non-real time application displays an interactive drawing window where the user can draw any kind of shapes or write words by changing grasping the HI and by

As mentioned previously, the core idea is to use the HDS as a tool capable of interacting dynamically with the human on the basis of the action he decides to perform. The use of an automated system in the skill transfer process has several advantages: the process could be repeated indefinitely with a high level of precision, time and location flexibility, the user can improve its performance without the assistance of a teacher and his progress can be

The HDS aims to be an interactive bi-directional skill transfer system that can emulate the presence of a human assistant, guiding the user movements on the trajectories chosen by the

In our previous research (Solis et al., 2002) it has been demonstrated the influence that proportional feedback programmed into haptic interface (HI) can have in the development of motor skills, in spite of this experiment was neither co-located nor coherent with the visual stimuli and only could assist to drawn a circle with fixed radio. The HD is our

We have developed two applications where the HDS assists the users in the sketching of simple drawings. The user interacts with the system through different means of feedback

In order to present fixed templates to the users, we have developed an application that transforms the principal edges of a given input image file (jpg, bmp, tiff, etc.) into trajectories for later HDS interpretation, this way the "trajectories" are generated without any kind pre programming. The first process applied to the image is a median filter to remove the possible noise present, then a equalization and binarization is done (Pratt, 1991), after that, to detect its edges a Canny edge detector (Canny, 1986) is used, then the resulting

**3. A transfer skill system for teaching how to draw to unskilled persons** 

controlling the command buttons (Figure 13).

evaluated numerically through automated procedures.

Fig. 14. Haptic Interfaces as assistant of abilities

**3.1 Assisting to sketch using fixed templates** 

multimodal interface solution for teaches to design simple sketches.

user himself (Figure 14).

(visual, force, audio).

edges are s segmented using label connected components in a binary image (Haralick & Shapiro, 1992), subsequently its components that are not more longer than a minimum predefined size are erased. The remaining components are converted to one pixel wide curves applying on them the morphological thin operator (Haralick & Shapiro, 1992).

c) Haptic and Visual Feedback d) Visual Feedback

Fig. 15. The HDS live a virtual tutor.

Finally, the coordinates of the remaining objects (virtual templates) are converted to the HDS reference system controller and stored in a segment memory, which is shared with the HDS controller. The Figure 15a shows an example of original jpg image and Figure 15b shows the results of the image processing.

The GUI displays the processed images and assists to the user to interact with the templates. The application is controlled through four buttons, the first two (top to bottom, Figure 15c) acts like the left and right mouse buttons, the third cleans the screen and the fourth toggles the

Training Motor Skills Using Haptic Interfaces 211

(c) The user "fills" the template (d) The user decides to finalize with

(e) Final sketch without "template"

(b) A circle "template"

assistance of the HDS

(a) Choosing between three different geometric primitives (straight line, circle and arc).

Fig. 16. Interaction with the HDS assistant.

force feedback. The user can move the HDS's end effector (HEE) freely in all its workspace until he presses the first button. On button press, a blue dot is drawn in the end effector position (visual feedback) and a collision detection algorithm looks for collisions between the HEE position and any template (trajectory) inside a radius of 40 mm from the HEE center.

If a collision is detected, the HDS responds using force feedback to maintain the HEE constrained to the nearest point of the contacted template. Force feedback is generated with the following control law:

$$F = -K\_P \Delta - K\_v \stackrel{\bullet}{\Delta} \tag{5}$$

Where is the distance between the HEE Position and its nearest point on the template, and *K*P and *K*v should be small enough to allow a free movement along the trajectory while creating a force feedback outside the trajectory, its values where find experimentally. When the HDS exerts a force of two Newtons or more, the user receives an auditory feedback.

Figure 15c shows a user interaction with the application: it is possible to see the coherence and colocation between haptic and graphical information. The visual feedback presented to the user is shown close up in Figure 15d.

## **3.2 Assisting to sketch using interactive templates**

A second application was developed in order to assist to drawn sketches interactively, the HDS's audio feedback, coherence and co-location between haptic and graphical information are exploited. The GUI developed is minimalist and simple to use (Figure 16). At the moment it can assist to design three different types of virtual geometrical templates (VGTs): straight line, circle and arc. The use of the buttons in order to control the application, the collision detection algorithm and the auditory cues have the same behaviour, that its fixed template counterpart.

The user can move the HEE freely in all the workspace and drawing a free sketch when the first button is pressed. When the user decided to drawn a VGT with the assistance of the HDS, he first, must indicate the VGT's first point, pressing the second button and choosing of a menu the VGT that he want drawn (Figure 16a); this first point could be; a) the beginning of a straight line, b) beginning of an arc or c) the center of a circle. Once that the first point has been selected, he must move the HEE to another position inside the GUI and with the first button must to mark the second and third points depending on which VGT has been selected. The second point is the end of the straight line if a line VGT has been chosen, on other hand if circle VGT was selected, the second point is used to calculate the radius respect to the first point and then generate a circumference around it. If arc VGT is chosen, the three points marked are united interpolating a second order spline curve.

Once that all the control points have been defined, the application draws a virtual "template trajectory" (Figure 16b), after that, the user can move the HDS's end effector (HEE) freely in all its workspace until he presses the first button. On button press, a blue dot is drawn in the end effector position (visual feedback) and a collision detection algorithm looks for collisions between the HEE position and the virtual template. Similarly to the fixed template application, the HDS maintains its HEE constrained to the nearest point of the contacted template and draws a blue line.

force feedback. The user can move the HDS's end effector (HEE) freely in all its workspace until he presses the first button. On button press, a blue dot is drawn in the end effector position (visual feedback) and a collision detection algorithm looks for collisions between the HEE position and any template (trajectory) inside a radius of 40 mm from the HEE center.

If a collision is detected, the HDS responds using force feedback to maintain the HEE constrained to the nearest point of the contacted template. Force feedback is generated with

*FK K P v*

Where is the distance between the HEE Position and its nearest point on the template, and *K*P and *K*v should be small enough to allow a free movement along the trajectory while creating a force feedback outside the trajectory, its values where find experimentally. When the HDS exerts a force of two Newtons or more, the user receives an auditory feedback.

Figure 15c shows a user interaction with the application: it is possible to see the coherence and colocation between haptic and graphical information. The visual feedback presented to

A second application was developed in order to assist to drawn sketches interactively, the HDS's audio feedback, coherence and co-location between haptic and graphical information are exploited. The GUI developed is minimalist and simple to use (Figure 16). At the moment it can assist to design three different types of virtual geometrical templates (VGTs): straight line, circle and arc. The use of the buttons in order to control the application, the collision detection algorithm and the auditory cues have the same behaviour, that its fixed

The user can move the HEE freely in all the workspace and drawing a free sketch when the first button is pressed. When the user decided to drawn a VGT with the assistance of the HDS, he first, must indicate the VGT's first point, pressing the second button and choosing of a menu the VGT that he want drawn (Figure 16a); this first point could be; a) the beginning of a straight line, b) beginning of an arc or c) the center of a circle. Once that the first point has been selected, he must move the HEE to another position inside the GUI and with the first button must to mark the second and third points depending on which VGT has been selected. The second point is the end of the straight line if a line VGT has been chosen, on other hand if circle VGT was selected, the second point is used to calculate the radius respect to the first point and then generate a circumference around it. If arc VGT is chosen, the three points marked are united interpolating a second order spline curve.

Once that all the control points have been defined, the application draws a virtual "template trajectory" (Figure 16b), after that, the user can move the HDS's end effector (HEE) freely in all its workspace until he presses the first button. On button press, a blue dot is drawn in the end effector position (visual feedback) and a collision detection algorithm looks for collisions between the HEE position and the virtual template. Similarly to the fixed template application, the HDS maintains its HEE constrained to the nearest point of the

(5)

the following control law:

template counterpart.

the user is shown close up in Figure 15d.

contacted template and draws a blue line.

**3.2 Assisting to sketch using interactive templates** 

(b) A circle "template"

(a) Choosing between three different geometric primitives (straight line, circle and arc).

(e) Final sketch without "template"

Fig. 16. Interaction with the HDS assistant.

Training Motor Skills Using Haptic Interfaces 213

In the Figure 18, we present the normalized sum of PSD over the subjects of the two groups:

We can obviously observe a strong difference between performances on trial 2, 4 and 6. What is more interesting, we can observe that for the Visual Feedback Only group, learning stops at Trial 4, while the Visual Feedback and Haptics group continues learning to Trail 7. Furthermore, at trial 7 the normalized sum of PSD is lower for the Visual Feedback and Haptics group, which leads us to conclude (within the limits of a small scale experiment with a small number of subjects) that Haptic feedback enables learning for a longer time and

We didn't validate the improvement of the user to drawn simple skills due there aren't restrictions about position, size of the sketch's traces. It is our first attempt to develop a more advanced assistant, which will have more virtual geometric templates and the

The efficacy of the Haptic Desktop to transfer basic skills to drawn simples sketches was demonstrated through the use of two applications, in both the HD acts like a virtual teacher that uses its force feedback and collocated capabilities to retain the cursor's position (controlled by the end effector position) in contact the borders of predefined sketch. The sketch could be either fixed or interactive, the fixed ones are generated from images, and the interactive ones are generated by the user whom can decide where, how big and which

auditory cues will be exploited more properly, i.e. vocal instructions.

primitive geometric element draw (circle, line or arch).

**4.1 Results** 

the PSD sum for trial 1 has been made equal to 1.

Fig. 18. Evolution of normalized sumatory of the PSD

leads to better results for the task at hand.

**5. Conclusions** 

When the user desires finalize with the HDS's assistance, he must press the second button and indicates using a menu, if erase the line recently drawn or maintains it; if he decides maintain it, the virtual template will be erased and only the line drawn made remains (Figure 16.e). In order to design more complex sketches is necessary repeat all the precedent steps and depending of the ability of the user can draw more complex design.
