**1.1 Integration of tactile and kinesthetic feedback toward direct contact simulation**

Tasks that involve direct contact between hand and objects are the most complex manipulative actions that humans can perform. Human ability of exploring, grasping and manipulating tools and objects relies on superior morphological and physical properties of our hands. A sophisticated system of bone, joints and tendon allow our hands to perform complex movement and to control accurately interaction forces.

On the Integration of Tactile and Force Feedback 49

We decided not to include the topic of thermal display and feedback since it would require a

Haptic interaction with objects can happen in two different ways. The first happens when a subject holds a special tool used for interacting with the surrounding environment. A pen, a joystick or a fork are examples of possible tools. The artificial creation of interaction with environment is then completely focused on replicating the information that is exchanged

Alternatively the subject can directly interact with her/his bare fingers with the environment. In this case the artificial recreation of interaction is much more complex and is

We will refer to these two different cases respectively as Mediated Contact interaction and Direct Contact interaction. Hayward in (Hayward, 2008) provided a very clear theoretical

During haptic interaction mediated by a tool the subject holds a tool in his hand and the haptic information regarding the interaction is practically contained in the motion and the forces that are acting on the tool. An ideal device able to perfectly reproduce the haptic interaction can be equipped with a toll moke-up that is a faithful reproduction of the real tool. The problem of perfectly reproducing the tactile and kinesthetic sensory input is then reduced to the problem of imposing to such moke-up the same static and dynamic behavior

For this kind of interaction kinesthetic haptic interface are theoretically very efficient but in many cases their mechanical performance in terms of frequency response, fidelity in force reproduction and force resolution are still insufficient for high fidelity reproduction of the

A lot of research efforts has been applied for improving and optimizing kinesthetic haptic interfaces specifically developed for mediated tool interaction and many systems are

For example, in the field of surgical simulators for training of laparoscopic surgery, catheter insertion, phlebotomy, or, in the domain of Virtual Prototyping and Assembly, mediated

The analysis of the Direct Contact interaction requires looking with higher level of detail to the physics of the phenomena. The problem of perfectly reproducing the phenomena through artificial stimulus can be seen in a dual way. The first consists in defining the haptic interface as a device able to perfectly reproduce the shape and the spatial distributed mechanical impedance of the contact surface. Or, dually, the device can be seen as a generator of pressure field that perfectly reproduce the interaction pressure distribution that

already commercialized and available as an off the shelf product.

contact device have been successfully employed.

**2.1.2 Direct contact interaction** 

focused on reproducing the physical phenomena that occurs at the fingertip level.

description of the difference between these two modes.

**2.1.1 Mediated contact interaction** 

**2.1 Analysis of direct contact interaction: From perception to design of devices** 

special treatise.

through the held object.

of the real tool.

real feelings.

However, as many researches have demonstrated, those abilities are not only based on the high elaborated structural properties of our hands but the haptic sensory input is highly involved in such tasks and it plays a fundamental role.

Basically, there are two types of interaction mode with objects: Active Haptic Sensing and Grasping/Manipulation (Jones & Lederman, 2006). In both of them the multimodal sensing given by the simultaneous tactile and kinesthetic sensory input strongly come into play. It has been largely demonstrated that in Active Haptic Sensing the information gathered by tactile and kinesthetic channels are somehow integrated while performing explorations and recognition tasks. Lederman and Klatzky for example (Lederman & Klatzky, 1999) demonstrated that in absence of distributed tactile information the perceptual capabilities of our fingers are strongly impaired. In addition our ability in grasping/manipulation of objects strongly relies on both the sensory inputs. Several research works have demonstrated how the absence of tactile or kinesthetic sensory input makes our ability worse. Johansson and Flanagan for example in (Johansson & Flanagan, 2007) show how the distributed tactile information are deeply involved even in very simple manual tasks like grasping and lifting an object.

These considerations lead to conclude that a system that is asked to replicate with high level of realism the direct interaction of virtual objects with the human hands should include both tactile and kinaesthetic sensory inputs.

Unluckily, most of present day haptic feedback systems are able to stimulate only kinesthetic or tactile interactions separately. Traditional force feedback devices like Phantom® (Salisbury & Srinivasan, 1997) are typically only able to provide sensory input correlated with kinaesthetic information. Such device are usually employed for realistic rendering of "mediated contacts", however they can be connected to the user body through mechanical components like a thimble that guarantees a constant contact surface for the simulation of interaction with bare fingers. From the functional point of view these class of devices are able to provide as output a controlled force and/or torques and get as input the position and/or orientation of the interaction surface. The system is unable to transmit distributed and programmable sensory input on the skin surface and the exchanged information only relates with global force and/or torque and global position and/or orientation of a body.

The technical challenge of integrating tactile feedback on kinesthetic has been faced only in few research works. In this chapter we are going to summarize the issues that concern the Direct Contact simulation with the integration of tactile and kinesthetic feedback. The topic is treated on both the hardware and software perspective illustrating the basic problems found in mechanical and control electronics integration and in haptic rendering for efficient computation.
