**2.1 Factors related to lifting and holding movements**

Whenever a person attempts to perceive the heaviness of an object or to compare the difference in weight between two or more objects, grasping, lifting and holding movements

obtained from the real world (i.e., Real Haptic + Real Vision). Also, we posit that superimposing computer-generated graphical objects precisely onto real, physical objects (i.e., Real Haptic + Virtual Vision) allows manipulation of visual properties of objects independently from physical properties. Finally, we introduce Virtual Haptic + Virtual Vision conditions, using a unique experimental augmented environment using both computer haptics AND computer graphics displays. This allows for selective experimental manipulations in which the haptic device affects haptic perception while computer generated graphics have an effect on visual perception, by superimposing computergenerated object forces precisely onto the graphical objects existing within the computer display. Presented are fresh findings regarding heaviness perception that enhance the knowledge base for the human perception of heaviness. These basic findings are expected to facilitate future research and development of haptic and graphic computer systems relating

Figure 1 shows a schematic that provides an overview of three categories of factors influencing human perceived heaviness: (1) factors related to the sensorimotor system responsible for lifting and holding an object (Object Lifting Phase), (2) factors or physical properties relating to an object itself (Object), and (3) factors relating to perceiving or judging heaviness (Perceiving Heaviness Phase). These factors act not only individually but also interactively. It should be emphasized here that the human psychological unit of

Fig. 1. Factors influencing human perceived heaviness. Question marks indicate whether or how humans use perceived heaviness for subsequent object lifting. This is controversial and

Whenever a person attempts to perceive the heaviness of an object or to compare the difference in weight between two or more objects, grasping, lifting and holding movements

to human recognition, lifting, transport or manipulation of physical objects.

**2. Factors influencing human perceptual system of heaviness** 

heaviness differs from the physical unit of measuring weight.

requires further investigation (See Sec. 5.1).

**2.1 Factors related to lifting and holding movements** 

must precede these behaviors (MacKenzie & Iberall, 1994). When an object is lifted by an adult, the sensorimotor system functions automatically, without any particular attention, to achieve a safe and effective grip to lift and hold, transport or manipulate the object, maintaining stability. This system is composed of two subsystems: a feedforward system and a feedback system.

A feedforward system works before the initiation of the grasping movement to predict object properties including weight and estimate the required motor commands, based on long-term memories (Gordon et al., 1993) and/or short-term memories if the lifts are repeated during a short period of time (Johansson, 1996). Finally, these programmed motor commands are sent to the related muscle groups to achieve stable lift of the target object. Interestingly, the motor commands are hypothesized to be sent not only to the effectors, i.e., related muscle groups for object lifts, but also for "efferent copies" to be sent to the CNS structures related to sensation, feedback processing or perception (Sperry, 1950; Holst, 1954; McCloskey, 1978).

Once the lift of an object has been initiated, the feedback system acts to optimize the output forces in the muscles through a local reflex feedback via Ia afferents in a muscle spindle (for muscle stretch) and Ib in a Golgi tendon organ (for muscle tension) (Crago et al., 1982; Rothwell, et al., 1982). These peripheral-origin signals ascend via the dorsal column through transcortical loops in the central nervous system (CNS) relating to motor control (S1, M1, basal ganglia, and cerebellum), and ongoing motor commands are probably modified via cortico-cerebellar connections with the CNS-origin signals "copied" to optimize subsequent discharge based on detected errors between the efferent copies and afferent information. The optimized signals, then, may contribute to achieving safer and more stable lift (Brodie & H.E. Ross, 1985). Grip forces applied at the object/digit interfaces are also automatically adjusted, based on the information from mechanoreceptors on glabrous skin, from the initiation of lifts according to frictional forces, object slipperiness (Johansson, 1996; Rinkenauer, et al., 1999) and object torque (Kinoshita et al., 1997).

The "copies" have been termed in such various ways as "sense of effort" (McCloskey, 1978), "collorary discharge" (Sperry, 1950) or "efferent copy" (Holst, 1954). Furthermore, in psychological research such as that for the size-weight illusion (SWI), (Charpentier, 1891 as cited in Murray, et al., 1999; See Sec. 3), efferent copy is replaced by the term "expectation" (H.E. Ross, 1969) or "ease with which could be lifted" (Müller & Schumann, 1889, as cited in Davis, 1973). These copies are thought to play an important role in object perception as well as for motor control. A well-known example is the ability to perceive an object and/or its surroundings as being at rest and clear without blurring when the eyes are moved. This is due to the copied signals relating to self-generated movement being compared with the signals obtained from vision (Holst, 1954). Without this system, we could not perceive an object accurately.

Interestingly, the forces generated when lifting an object correlate with object weight (Johansson, 1996) and that of heaviness perception (Harper & Stevens, 1948; Stevens, 1958). However, as to the correlation between the forces generated and perceived heaviness, researchers differ. Some researchers support their correlation (H.E. Ross, 1969; Davis & Roberts, 1976; Gordon et al., 1991) and others report their dissociation (Flanagan & Beltzner, 2000; Grandy & Westwood, 2006; Chang et al., 2008). What factors give rise to these

Computer Graphic and PHANToM Haptic Displays:

heaviest" might cognitively bias perception of heaviness.

**3. Effects of object "size" on perceived heaviness** 

(Li et al., 2007).

heaviness.

Powerful Tools to Understand How Humans Perceive Heaviness 29

As peripheral or bottom-up processing issues related to perceptual system of heaviness, individual learning and sensitivity in weight discrimination are important factors. The Weber fraction, i.e., weight sensitivity, widely differs among individuals (0.02~0.16) (Holway et al., 1937; Raj et al., 1985). Age, especially, is a crucial factor leading to a decrease of sensitivity to weight or heaviness (Gandevia, 1996; Dijker, 2008). Serious deterioration is also reported for neural disorders including leprous neuropathy (Raj, et al., 1985), lesions to inferior-frontal cortex including PMv (Halstead, 1945), and left parietal and temporal lesions

As for higher level, cognitive-based or top-down processing, individual learning or experience is also an important factor to influence perception of heaviness (Fig. 1). An expectation that a larger object should be heavier than a smaller object, for example, is thought to affect perceived heaviness. Accordingly, when lifted, the larger of two objects of equal weight tends to be perceived as lighter than the smaller (H.E. Ross, 1969; Davis & Roberts, 1976; Gordon et al., 1991; Rabe et al., 2009; Buckingham & Goodale, 2010). This expectation factor is also experienced in relation to object colour (Payne, 1958), object material (Ellis & Lederman, 1999; Buckingham et al., 2009), and even the human conditions of gender and age (Dijker, 2008). For example, when viewed, darker or metallic objects are judged to be heavier than brighter or wooden ones, but are then perceived as lighter when actually lifted under same-weight conditions. When confronting conflicting issues, such as with the SWI, when two objects have different size but are of identical weight, subjects tend to rationalize that if the weight is the same, the larger object should be lighter, rather than depending on the current sensations of heaviness (Mon-Williams & Murray, 2000). This tendency seems to be more pronounced when subjects are subject to the forced-choice condition in which they must choose either Heavier or Lighter (Mon-Williams & Murray, 2000; Buckingham & Goodale, 2010). This raises the question, what is the best way to obtain accurate and natural responses from subjects: using the two category method "Heavier and Lighter", the three category method "Heavier, Lighter, and Similar", the bimanual or unilateral matching methods, or the magnitude estimation method? Other questions posed by experimenters such as, "Which is heavier?" or "Which is the

Object size is the oldest, most-studied factor since Müller and Schuman (1889, as cited in Davis, 1973) and Charpentier (1891, as cited in Murray et al. 1999) reported the phenomenon of the size-weight illusion (SWI): the larger of two objects of equal weight is perceived as lighter than the smaller. The mechanism underlying the SWI has been continuously discussed with various interpretations: Expectation Theory (H.E. Ross, 1969), Information Integration Theory (Anderson, 1970; Masion & Crestoni, 1988), Density Theory (J. Ross & Di Lollo, 1970), Gain Adjustment Theory (Burgess & Jones, 1997), Inertia Tensor Theory (Amazeen, 1999), Bayesian Approach (Brayanov & Smith, 2010), and Throwing Affordance Theory (Zhu & Bingham, 2011). The reasons for these ongoing differences of opinion are that heaviness is affected by various factors (as noted in Sec. 2) and the difficulty of strictly manipulating object size as a single independent variable, uncontaminated by other object properties. As a result, object size remains a wide-open topic regarding perception of

**2.3 Bottom-up and top-down influences on heaviness perception** 

opposing views and which view will eventually prevail remain matters to be resolved, noted by question marks in Fig. 1.

Yet, the motor-related commands for force generation and adaptation, at least partly or indirectly, relate to perceived heaviness. Evidence has shown involvement of the sensorimotor system in human perceptual system of heaviness (McCloskey, 1978). That is, the degree of perceived heaviness is reported to increase due to the effects of fatigue on related muscles (Jones & Hunter, 1983; Buckingham et al., 2009), to partial curarization or a peripheral anaesthesia effect on cutaneous or joint sensation (Gandevia & McCloskey, 1977a), and to muscle vibration on related muscle spindles (Brodie & H.E. Ross, 1984). Neural disorders relating to the sensorimotor system are also reported to affect perceived heaviness. In comparison to normal subjects, for example, it is reliable to be overestimated in patients with paresis (Gandevia & McCloskey, 1977b), in deafferented patients for muscle spindles and Golgi tendon organ (Rothwell et al., 1982), in those with cerebellum disorders (Holms, 1917; Cf. Rabe et al., 2009), and in those with Parkinson's disease (Maschke et al., 2006).

Furthermore, attention should be focused on the lifting conditions, i.e., how to discern the heaviness of an object as a higher order or more cognitive and strategic matter (Fig.1). The manner of lifting, for example, affects perceiving heaviness in various conditions: active lifting, such as jiggling an object (Brodie & H.E. Ross, 1985), tends to make more accurate weight discrimination than passive pressure (Weber, 1834, translated by H. E. Ross & Murray, 1978). In addition, the perceived heaviness when an object is lifted depends on which parts of the hand are in contact, with an increase of perceived heaviness being reported when an object is lifted distally, by the fingertips, compared to when lifted proximally, by the base of fingers or near the palm (Davis, 1974). Holway et al., (1938) reported that the same object is perceived as heavier in the second trial than that in the first. After repeated lifts of sets of heavier objects, the discriminative thresholds decreased in the sets of lighter objects compared to those without such preceding lifts of sets of heavier objects (Holway & Hurvich, 1937). Further, the degree of perceived heaviness changed when lifting two objects simultaneously using both hands compared to that when lifting two objects alternately using only one hand (Jones & Hunter, 1982).

#### **2.2 Object properties and environmental surroundings influence perceived heaviness**

Object weight is a vital factor for perceived heaviness (Harper & Stevens, 1948; Stevens, 1958). Heaviness is, however, not weight. Previous studies demonstrated that heaviness is affected by the input of information regarding size whether visually (H.E. Ross, 1969; Masin & Crestoni, 1988) or haptically (Ellis & Lederman, 1993). Input includes such factors as pressure on the contact-area of the skin (Charpentier, 1891, as cited in Murray et al. 1999), object surface slipperiness (Rinkenauer, et al., 1999), material (Ellis & Lederman, 1999; Buckingham et al., 2009), colour (De Camp, 1917; Payne, 1958), shape (Dresslar, 1894), temperature (Stevens & Green, 1978), inertia tensor whether perceived haptically (Amazeen, 1999) or visually (Streit et al., 2007), and density (J. Ross & Di Lollo, 1970; Grandy & Westwood, 2006). Regarding experimental surroundings (Fig. 1, bottom in middle), changes in gravity have been reported to affect perceived heaviness. Compared to a 1-G normal environment, zero-G reduces perceived heaviness and weight discrimination, while that of 1.8-G increases them (H. E. Ross & Reschike, 1982; H. E. Ross et al., 1984).

opposing views and which view will eventually prevail remain matters to be resolved,

Yet, the motor-related commands for force generation and adaptation, at least partly or indirectly, relate to perceived heaviness. Evidence has shown involvement of the sensorimotor system in human perceptual system of heaviness (McCloskey, 1978). That is, the degree of perceived heaviness is reported to increase due to the effects of fatigue on related muscles (Jones & Hunter, 1983; Buckingham et al., 2009), to partial curarization or a peripheral anaesthesia effect on cutaneous or joint sensation (Gandevia & McCloskey, 1977a), and to muscle vibration on related muscle spindles (Brodie & H.E. Ross, 1984). Neural disorders relating to the sensorimotor system are also reported to affect perceived heaviness. In comparison to normal subjects, for example, it is reliable to be overestimated in patients with paresis (Gandevia & McCloskey, 1977b), in deafferented patients for muscle spindles and Golgi tendon organ (Rothwell et al., 1982), in those with cerebellum disorders (Holms, 1917; Cf. Rabe et al., 2009), and in those with Parkinson's disease (Maschke et al.,

Furthermore, attention should be focused on the lifting conditions, i.e., how to discern the heaviness of an object as a higher order or more cognitive and strategic matter (Fig.1). The manner of lifting, for example, affects perceiving heaviness in various conditions: active lifting, such as jiggling an object (Brodie & H.E. Ross, 1985), tends to make more accurate weight discrimination than passive pressure (Weber, 1834, translated by H. E. Ross & Murray, 1978). In addition, the perceived heaviness when an object is lifted depends on which parts of the hand are in contact, with an increase of perceived heaviness being reported when an object is lifted distally, by the fingertips, compared to when lifted proximally, by the base of fingers or near the palm (Davis, 1974). Holway et al., (1938) reported that the same object is perceived as heavier in the second trial than that in the first. After repeated lifts of sets of heavier objects, the discriminative thresholds decreased in the sets of lighter objects compared to those without such preceding lifts of sets of heavier objects (Holway & Hurvich, 1937). Further, the degree of perceived heaviness changed when lifting two objects simultaneously using both hands compared to that when lifting

**2.2 Object properties and environmental surroundings influence perceived heaviness**  Object weight is a vital factor for perceived heaviness (Harper & Stevens, 1948; Stevens, 1958). Heaviness is, however, not weight. Previous studies demonstrated that heaviness is affected by the input of information regarding size whether visually (H.E. Ross, 1969; Masin & Crestoni, 1988) or haptically (Ellis & Lederman, 1993). Input includes such factors as pressure on the contact-area of the skin (Charpentier, 1891, as cited in Murray et al. 1999), object surface slipperiness (Rinkenauer, et al., 1999), material (Ellis & Lederman, 1999; Buckingham et al., 2009), colour (De Camp, 1917; Payne, 1958), shape (Dresslar, 1894), temperature (Stevens & Green, 1978), inertia tensor whether perceived haptically (Amazeen, 1999) or visually (Streit et al., 2007), and density (J. Ross & Di Lollo, 1970; Grandy & Westwood, 2006). Regarding experimental surroundings (Fig. 1, bottom in middle), changes in gravity have been reported to affect perceived heaviness. Compared to a 1-G normal environment, zero-G reduces perceived heaviness and weight discrimination, while that of

two objects alternately using only one hand (Jones & Hunter, 1982).

1.8-G increases them (H. E. Ross & Reschike, 1982; H. E. Ross et al., 1984).

noted by question marks in Fig. 1.

2006).
