**3. Cross-modal transfer between touch and vision**

In cognitive psychology, amodal perception is usually considered to be present at birth (see Streri, in press; Streri & Gentaz, 2009) as suggested by E. J. Gibson (1969). Beyond the details provided by individual sensory modalities, newborns are able to perceive a multimodal object as unified. However, the links between the haptic and the visual modalities are not fully established and will not be it until about the age of 15 years. Because newborns cannot engage in bimodal visual-haptic exploration of an object, a cross-modal transfer paradigm can be used to uncover the nature of these links and thereby evaluate young infants' ability to match the same object property captured by two modalities. However, cross-modal transfer tasks involve two successive phases (familiarization with an object in one modality and recognition test in a second modality). These tasks require cognitive processes (manual and visual information-processing capacities, memory load, etc.) that can weaken the links between sensory modalities and reveal failures in the establishment of amodal perception. Here we present a series of studies that illustrate these constraints.

#### **3.1 Initial evidence in newborns**

Newborns' visual abilities are weak. Nevertheless, numerous studies have revealed that babies can perceive speaking faces, photographs, objects, pictures, discriminate between large numbers, etc. (Coulon, Guellai and Streri, 2011; Féron, Gentaz, and Streri 2006; Guellai and Streri, 2011; Izard, Sann, Spelke and Streri, 2009; Meary, Kitromilides, Mazens, Graff and Gentaz, 2007; cf. Kellman and Arteberry, 1988, for a review). As discussed above, various studies have provided evidence that newborns are able to detect differences between shapes and textures with their hands (Streri et al. 2000; Molina and Jouen, 1998). All of these findings show that the prerequisites in both modalities are present to obtain cross-modal transfer between these senses.

Intermanual and Intermodal Transfer in Human Newborns:

this object.

**3.3 Shape vs. texture** 

Neonatal Behavioral Evidence and Neurocognitive Approach 325

plausible explanation of these results on lack of bi-directional crossmodal transfer is that, as in newborns, the levels of representation attained through each modality are not sufficiently equivalent to exchange information between sensory modalities. This hypothesis seems to be validated by the fact that if a two-month-old baby is presented with degraded visual stimulation (a bi-dimensional sketch of an 3D object) in which volumetric and textural aspects are missing, leading to a blurred percept, tactile recognition is possible, which is not the case with a visual volumetric object (Streri and Molina 1993). This result means that the infant's hand cannot sufficiently explore the held object to obtain a clear representation of

A number of studies have also revealed that over the course of development, the links between the haptic and the visual modalities are fragile, often not bi-directional, and representation of objects is never complete: this holds not only in infancy (Rose and Orlian 1991; Streri 2007; Streri and Pêcheux 1986), but in children (Gori *et al.* 2008) and adults (Kawashima *et al.* 2002). For example, in a behavioral and PET study on human adults, Kawashima *et al.* found that the human brain mechanisms underlying crossmodal discrimination of object size follow two different pathways depending on the temporal order in which the stimuli are presented. They found crossmodal information transfer to be less accurate with VT transfer than with TV transfer. In addition, more brain areas were activated during VT than during TV. Crossmodal transfer of information is rarely reversible, and is generally asymmetrical even when it is bi-directional. However, in adults, these asymmetries can be due to experience, learning and maturation and the characteristics of these asymmetries cannot be used directly to explain the brains of newborns. To better understand results from newborns and two-month-olds, a comparison with another

property (texture) in bi-directional cross-modal transfer tasks was carried out.

as efficient as (if not better than) vision, this asymmetry does not appear.

The comparison between shape and texture, amodal properties, should allow testing the hypothesis of amodal perception in newborns and to shed light on the processes involved in information-gathering by both sensory modalities. However, shape is best processed by vision, whereas texture is thought to be best detected by touch (see Bushnell and Boudreau 1998; Klatzky *et al.* 1987). According to Guest and Spence (2003), texture is "more ecologically suited" to touch than to vision. In many studies on shape (a macrogeometric property), transfer from haptics to vision has been found to be easier than transfer from vision to haptics in both children and adults (Connolly and Jones 1970; Jones and Connoly 1970; Juurmaa and Lehtinen-Railo 1988; Newham and MacKenzie 1993; cf. Hatwell 1994). In contrast, when the transfer concerns texture (a microgeometric property), for which touch is

Sann and Streri (2007) undertook a comparison between shape and texture in bi-directional crossmodal transfer tasks. They sought to examine how information is gathered and processed by the visual and tactile modalities and, as a consequence, to shed light on the perceptual mechanisms of newborns. If the perceptual mechanisms involved in gathering information on object properties are equivalent in both modalities at birth, then reverse crossmodal transfer would be expected. In contrast, if the perceptual mechanisms differ in the two modalities, then non-reversible transfer should be found. Thirty-two newborns participated in two experiments (16 in crossmodal transfer from vision to touch, and 16 in the reverse transfer). The stimuli were one smooth cylinder and one granular cylinder (a

Streri and Gentaz (2003; see also Streri and Gentaz, 2004) conducted an experiment on crossmodal transfer of shape information from the right hand to the eyes in 24 human newborns (mean age: 62 hours). They used an intersensory paired-preference procedure that included two phases: a haptic familiarization phase in which newborns were given an object to explore manually without seeing it, followed by a visual test phase in which infants were shown the familiar object paired with a novel one. Tactile objects were a small cylinder (10 mm in diameter) and a small prism (10 mm triangle base). Because the vision of newborns is immature and their visual acuity is weak, visual objects were the same 3D shapes, but much larger (45mm triangle base and 100mm in length for the prism and 30mm in diameter and 100mm in length for the cylinder). An experimental group (12 newborns) underwent the two phases successively (haptic then visual) whereas a baseline group (12 newborns) underwent only the visual test phase with the same objects as the experimental group but without haptic familiarization. Comparison of looking times between the two groups provided evidence of crossmodal recognition, with shapes explored by the hands of the experimental group recognized by the eyes. The newborns in the experimental group looked at the novel object for longer than the familiar one. In contrast, the newborns in the baseline group looked equally at both objects. Moreover, infants in the experimental group made more gaze shifts toward the novel object than the familiar object. In the baseline group this was not the case. Thus, this recognition in the experimental group stems from the haptic habituation phase. These results suggest that newborns recognized the familiar object through a visual comparison process as well as a comparison between the haptic and visual modalities. Moreover, the discrepancy between the sizes of the visual and tactile objects was apparently not relevant for crossmodal recognition. Shape alone seems to have been considered by newborns.

#### **3.2 Limits of cross-modal shape transfer**

Sann and Streri (2007) tested transfer from eyes to hand and from hand to eyes in order to ascertain whether this would demonstrate a complete primitive 'unity of the senses.' After haptic habituation to an object (cylinder or prism), the infants were shown the familiar and the novel shape in alternation. After visual habituation with either the cylinder or the prism, the familiar and the novel shape were put in the infant's right hand. The tactile objects were presented sequentially in an alternating manner. Again, visual recognition was observed following haptic habituation, but the reverse was not the case: no haptic recognition was found following visual habituation. Evidence of a visual recognition of shape also depended on the hand stimulated during the familiarization phase. No evidence of crossmodal recognition was found when the left hand was stimulated (Streri and Gentaz, 2004). Thus, cross-modal transfer seems not to be a general property of the newborn human; instead it is specific to certain parts of the body.

To understand this lack of bi-directional crossmodal transfer we must examine the differences between the ways that the two modalities process object shape. Vision processes shapes in a global manner, whereas touch processes information sequentially. Moreover, infants do not use efficient tactile exploratory procedures such as "*contour following*" to establish good representations of shapes (Lederman and Klatzky,, 1987). Earlier research performed on 2-month-old infants and using a bi-directional crossmodal shape transfer task (Streri 1987) revealed that two-month-old infants visually recognize an object that they have previously held, but do not manifest tactile recognition of an already-seen object. A plausible explanation of these results on lack of bi-directional crossmodal transfer is that, as in newborns, the levels of representation attained through each modality are not sufficiently equivalent to exchange information between sensory modalities. This hypothesis seems to be validated by the fact that if a two-month-old baby is presented with degraded visual stimulation (a bi-dimensional sketch of an 3D object) in which volumetric and textural aspects are missing, leading to a blurred percept, tactile recognition is possible, which is not the case with a visual volumetric object (Streri and Molina 1993). This result means that the infant's hand cannot sufficiently explore the held object to obtain a clear representation of this object.

A number of studies have also revealed that over the course of development, the links between the haptic and the visual modalities are fragile, often not bi-directional, and representation of objects is never complete: this holds not only in infancy (Rose and Orlian 1991; Streri 2007; Streri and Pêcheux 1986), but in children (Gori *et al.* 2008) and adults (Kawashima *et al.* 2002). For example, in a behavioral and PET study on human adults, Kawashima *et al.* found that the human brain mechanisms underlying crossmodal discrimination of object size follow two different pathways depending on the temporal order in which the stimuli are presented. They found crossmodal information transfer to be less accurate with VT transfer than with TV transfer. In addition, more brain areas were activated during VT than during TV. Crossmodal transfer of information is rarely reversible, and is generally asymmetrical even when it is bi-directional. However, in adults, these asymmetries can be due to experience, learning and maturation and the characteristics of these asymmetries cannot be used directly to explain the brains of newborns. To better understand results from newborns and two-month-olds, a comparison with another property (texture) in bi-directional cross-modal transfer tasks was carried out.
