**3.2 Effects of visually perceived object size on heaviness perception: Use of the Real Haptics + Virtual Vision environment: Second experiments**

In Sec. 3.1, investigation of the effect of haptic size was made possible by simply removing the Real Vision from the Real Haptics + Real Vision environment. Such a simple and inexpensive experimental set-up has a great advantage since it is easily reproduced and replicated by others. Now we turn to visually perceived size. In experimental paradigms for the role of visual size in heaviness perception, it has been conventional that two or more objects of equal weight but unequal sizes are alternatively or in parallel lifted by subjects using wires, handles, or grip apparatus, and then rated for heaviness (See left in Fig. 4A and 4B) (H.E. Ross, 1969; David & Roberts, 1976; Masin & Crestoni, 1988; Gordon et al., 1991; Mon-Williams & Murray, 2000; Chouinard, et al., 2009; Rabe et al., 2009). Thus the haptically perceived size is kept constant. All studies unanimously reported that visually perceived size affected perceived heaviness: the smaller object felt heavier than the larger object although they were identical in weight.

However, these experimental set-ups clearly involved the factor of inertia tensor (Amazeen, 1999) which has significant impact on perceived heaviness. When lifting objects of different length, width, volume, shape, and orientation, it is possible to haptically perceive the differences in heaviness, even without visual information (Turvey & Carello, 1995). Thus, strictly speaking, we cannot determine conclusively whether the SWI is derived from

with weight (Kawai, 2002b), (6) The Weight/Aperture, the finger span formed during thumb-index finger grasp (the opposition vector), or the width of cube itself, can be derived as a heaviness model as it reflects subjective responses in any weight-density conditions

While it is clear that haptic object size is systematically involved in heaviness perception, the results of this study offer no neurological evidence about how, where and when in the

nervous system this occurs (Flanagan et al., 2008; Chouinard et al., 2009).

Fig. 3. A. Treatment of object size as a single independent factor. B. Weight-density conditions presented to the subjects (A: Reproduced with permission from Kawai, 2002a.

**Haptics + Virtual Vision environment: Second experiments** 

**3.2 Effects of visually perceived object size on heaviness perception: Use of the Real** 

In Sec. 3.1, investigation of the effect of haptic size was made possible by simply removing the Real Vision from the Real Haptics + Real Vision environment. Such a simple and inexpensive experimental set-up has a great advantage since it is easily reproduced and replicated by others. Now we turn to visually perceived size. In experimental paradigms for the role of visual size in heaviness perception, it has been conventional that two or more objects of equal weight but unequal sizes are alternatively or in parallel lifted by subjects using wires, handles, or grip apparatus, and then rated for heaviness (See left in Fig. 4A and 4B) (H.E. Ross, 1969; David & Roberts, 1976; Masin & Crestoni, 1988; Gordon et al., 1991; Mon-Williams & Murray, 2000; Chouinard, et al., 2009; Rabe et al., 2009). Thus the haptically perceived size is kept constant. All studies unanimously reported that visually perceived size affected perceived heaviness: the smaller object felt heavier than the larger object

However, these experimental set-ups clearly involved the factor of inertia tensor (Amazeen, 1999) which has significant impact on perceived heaviness. When lifting objects of different length, width, volume, shape, and orientation, it is possible to haptically perceive the differences in heaviness, even without visual information (Turvey & Carello, 1995). Thus, strictly speaking, we cannot determine conclusively whether the SWI is derived from

(Kawai, 2003a, 2003b).

Copyright © 2002 Springer-Verlag.)

although they were identical in weight.

differences in visual size or inertia tensor, when we adhere to the conventional methodology of the Real Haptic + Real Vision environment (left in Fig.4A and Fig.4B). This is the limitation of the real physical world for experimental control.

We decided, therefore, that the only way to separate the factor of visual size cues completely from other factors such as inertial tensor was to use a combination of 3D motion analysis and 3D computer graphics techniques to create virtual objects for the SWI paradigm. This type of augmented environment was developed as the Virtual Hand Laboratory at Simon Fraser University in Burnaby, Canada, in which 3D graphics (and other displays) were driven by 3D motion and forces. Computer graphics were developed to ensure completely independent manipulation of visual size information by superimposing computer-created graphics of different sizes on a single object (Fig. 4A, top) or two physically identical objects (Fig.4B, bottom); thus, haptic information was kept constant. As seen in Fig. 4, two different experimental set-ups were designed: a single physical grip apparatus was used specifically to record grip and load forces applied on the grip handle (top in Fig 4A), while two physical cubes with identical physical properties were used to investigate the effects of visual size on heaviness (bottom in Fig. 4B). In both environments, visual size varied as a single independent parameter without changing the inertia tensor. As shown in the right of Fig. 4B, each subject wore Crystal Eyes goggles and viewed through a semi-silvered mirror the stereo images (dotted line) on a monitor.

Fig. 4. From Real Haptics + Real Vision (left) to Real Haptics + Virtual Vision (right) environments. (A) a single stimulus presentation and (B) two stimuli presentation (Reproduced with permission from Kawai et al., 2007. Copyright © 2007 Springer-Verlag.)

Computer Graphic and PHANToM Haptic Displays:

**Virtual Haptics environment: Third experiment** 

enabling us to address pictorial depth cues.

cube to a 2D square.

et al. 2007).

Powerful Tools to Understand How Humans Perceive Heaviness 35

conditions with sufficient size differences between standard and comparison cubes of equal mass; that is, (3) when the comparison cube was smaller than 4.0 cm all the subjects perceived it to be heavier than the 5.0 cm standard cube (upper in Fig. 5), and when the comparison cube was larger than 7.0 cm all the subjects perceived it to be lighter (bottom in Fig. 5). (4) Interestingly, whether or not test subjects experienced the SWI was significantly correlated with their sensitivity to weight discrimination, but not their sensitivity to discriminate small differences in visual size, (5) Erroneously programmed motor commands were not systematically correlated to perceived heaviness or experience of the SWI (Kawai

We emphasize that usage of the Real Haptic + Virtual Vision environment was the way to verify effects of only visual size cues on perception of heaviness, while presenting both visual and haptic cues "synchronously" to participants; it is impossible to obtain this evidence from any experimental set-ups in the real world (Real Haptic + Real Vision).

**4. Pictorial depth cues of an object on perceived heaviness: Virtual Vision +** 

In Sec. 3.2, the Real Haptics + Virtual Vision environment ensured a strict manipulation of visual information independently from haptic information and suggested possibilities for presenting any type of visual/graphical stimulus with a range of intensities, unlike the Real Vision environment. Concerning vision, this suggested further study of visual components such as pictorial depth cues or stereopsis that may contribute to perceived object size and heaviness. For haptics, it suggested the development of a haptic display making possible the presentation of any type of haptic stimulus with a range of intensities. This led to the challenging experiment using a Virtual Haptic + Virtual Vision environment. Thus we demonstrate here an experiment using Virtual Vision + Virtual Haptics environment

**4.1 Volumetric information and pictorial depth cues in the size-weight illusion** 

Since volumetric information of objects has long been thought to be critical, numerous studies have focused on the volume of the object as the essential parameter for size in the size-weight illusion (SWI) (Scripture, 1897; H.E. Ross, 1969; Anderson, 1970; Ellis & Lederman, 1993). However, there is no direct evidence as to whether or not volumetric information is encoded in the process of size-weight integration in perceived heaviness. Further, although the dimensions of objects have been examined on reach-to-grasp movements (Westwood et al., 2002; Kwok & Braddick, 2003), no such research has been done on lifting movements or heaviness perception. This study investigated the effects of cues regarding pictorial depth and volumetric size cues on the size-weight integration in perceived heaviness. Weight displays were created using the PHANToM haptic stylus, synchronized and superimposed onto corresponding 2D graphic objects displayed on a 2D monitor, in accordance with manipulation of the stylus (virtual objects) by subjects. It was hypothesized that the degree of the SWI would be weakened, even to the point of disappearing, when the dimensions of pictorial depth cues were reduced from being a 3D

An OPTOTRAK 3D Motion Analysis system tracked infrared emitting markers attached to the physical cubes and the goggles and these 3D position data were used to create the stereographic images of the cubes in real time and subsequently, to analyze the lifting motion of the cubes. The physical cubes of identical size (3.0 x 3.0 x 3.0 cm) and mass (30.0 g) were invisible to subjects. The graphical size of the standard cube placed on the lefthand side of each subject was constant (5.0 x 5.0 x 5.0 cm; indicated as triangles in Fig. 5), while the comparison cube presented on the right-hand side varied from 1.0 x 1.0 x 1.0 to 9.0 x 9.0 x 9.0 cm (See details in Fig. 5). After lifting each pair of cubes, subjects were asked to report whether the comparison cube presented was perceived as Heavier, Lighter, or Similar in heaviness as compared to the standard cube.

Fig. 5. Systematic contribution of visual size cues to human perceived heaviness. (Adapted from Kawai et al., 2007. Copyright © 2007 Springer-Verlag.)

Prior to undertaking trials, care was taken to accurately determine whether or not the augmented objects could produce for subjects a sense of reality, existence, or presence similar to real objects. Every participant had a strong sense of presence of the graphical objects and felt as if they were interacting with a physical object both in terms of force programming (Kawai, et al., 2002) and perceived heaviness (Kawai et al., 2007). It was concluded to be due to the characteristics of this augmented environment that the graphical objects moved without any noticeable delay (timing), in exactly the direction intended (direction), and synchronized with the subject's lifting movement (speed), developing a kind of personal relationship or ownership (Ehrsson, et al.,2004).

Findings indicated that: (1) visual size cues systematically affected heaviness; when the comparison cube was smaller in size than the standard cube, it was perceived to be heavier and vice versa (Fig. 5), (2) visual size cues influenced heaviness for all subjects under

An OPTOTRAK 3D Motion Analysis system tracked infrared emitting markers attached to the physical cubes and the goggles and these 3D position data were used to create the stereographic images of the cubes in real time and subsequently, to analyze the lifting motion of the cubes. The physical cubes of identical size (3.0 x 3.0 x 3.0 cm) and mass (30.0 g) were invisible to subjects. The graphical size of the standard cube placed on the lefthand side of each subject was constant (5.0 x 5.0 x 5.0 cm; indicated as triangles in Fig. 5), while the comparison cube presented on the right-hand side varied from 1.0 x 1.0 x 1.0 to 9.0 x 9.0 x 9.0 cm (See details in Fig. 5). After lifting each pair of cubes, subjects were asked to report whether the comparison cube presented was perceived as Heavier, Lighter, or

Fig. 5. Systematic contribution of visual size cues to human perceived heaviness. (Adapted

Prior to undertaking trials, care was taken to accurately determine whether or not the augmented objects could produce for subjects a sense of reality, existence, or presence similar to real objects. Every participant had a strong sense of presence of the graphical objects and felt as if they were interacting with a physical object both in terms of force programming (Kawai, et al., 2002) and perceived heaviness (Kawai et al., 2007). It was concluded to be due to the characteristics of this augmented environment that the graphical objects moved without any noticeable delay (timing), in exactly the direction intended (direction), and synchronized with the subject's lifting movement (speed), developing a kind

Findings indicated that: (1) visual size cues systematically affected heaviness; when the comparison cube was smaller in size than the standard cube, it was perceived to be heavier and vice versa (Fig. 5), (2) visual size cues influenced heaviness for all subjects under

Similar in heaviness as compared to the standard cube.

from Kawai et al., 2007. Copyright © 2007 Springer-Verlag.)

of personal relationship or ownership (Ehrsson, et al.,2004).

conditions with sufficient size differences between standard and comparison cubes of equal mass; that is, (3) when the comparison cube was smaller than 4.0 cm all the subjects perceived it to be heavier than the 5.0 cm standard cube (upper in Fig. 5), and when the comparison cube was larger than 7.0 cm all the subjects perceived it to be lighter (bottom in Fig. 5). (4) Interestingly, whether or not test subjects experienced the SWI was significantly correlated with their sensitivity to weight discrimination, but not their sensitivity to discriminate small differences in visual size, (5) Erroneously programmed motor commands were not systematically correlated to perceived heaviness or experience of the SWI (Kawai et al. 2007).

We emphasize that usage of the Real Haptic + Virtual Vision environment was the way to verify effects of only visual size cues on perception of heaviness, while presenting both visual and haptic cues "synchronously" to participants; it is impossible to obtain this evidence from any experimental set-ups in the real world (Real Haptic + Real Vision).
