**2.1 Method**

**P2**

**8**

**9**

**7**

Objects were placed on a small table next to the participant. An opaque dark blue curtain was hung between the stimuli and participant and could be slid back and forth along a rod situated 10 feet above, allowing the participant to view or handle the object. This allowed the experimenter to select a designated stimulus to present to the subject, while hiding the remaining 23 stimuli. The stimuli were shown one at a time. Four types of objects can be identified: (a) 12 old objects, 4 from each category prototype, that were presented during learning; (b) 6 new patterns, 2 from each category; (c) 3 prototypes; and (d) 3 midpoint objects. The latter objects were midway between either of two prototypes and, therefore, could not be unambiguously assigned to a single prototype category. A schematic representation of the 24 objects, separated by the three categories and transformational paths is shown in Figure 2.

### **2.2 Procedure**

The learning phase was composed of four study-test trial blocks. On each study block, the 12 learning objects were shown randomly or systematically blocked by category, labeled as A, B, or C for the subject. Following this, the objects were presented in a random order and required verbal classification of the object (A, B, or C). Following their judgment, corrective verbal feedback was provided. For subjects in the systematic condition, the three categories were presented in a counterbalanced order, although patterns belonging to a given category were shown in a random order.

On the transfer test, all 24 objects were presented in a random order, which included the four training patterns in each category (old), the three category prototypes, and nine new objects. As indicated in Figure 2, three of the new patterns were located midway between the two prototypes and were, as a consequence, analyzed separately from the remaining new objects. On the transfer test, the subject was required to make a double judgment to each object. The first judgment was a recognition judgment – is this object old or new? The second judgment was a classification judgment (is it an A, B, or C pattern?).

Haptic Concepts 9

In general, performance improved across learning blocks, learning was more efficient with visual than haptic inspection, and performance was enhanced when study presentation was

Classification errors were unexpectedly rare, with overall error rates ranging between 3-10% among the four modality conditions, with accuracy highest in the VV condition and worst in the HH condition (participants were also tested one week later, and performance

Figure 4 shows the mean hit and false alarm rates as a function of study and test modality (VV, VH, HV, HH), training order (random, systematic), and time of test (immediate, week delay). In general, subjects were able to discriminate old from new objects with fair accuracy, with an overall hit rate of .715 and a false alarm rate of .543. The conditions ordered themselves, from best to poorest old-new discrimination, as VV > VH = HH > HV, with a mean difference

> **0.3 0.4 0.5 0.6 0.7 0.8 0.9**

**P(Old)**

**Old New**

> **Imm Week Imm Week Sys-VV Sys-HH**

**Syst Learning, Switched Modality**

**Imm Week Imm Week Sys-VH Sys-HV**

**Syst Learning, Same Modality**

**Old New**

**Old New**

Fig. 4. Mean hit and false alarm rates as a function of study and test modality (VV, VH, HV, HH), training order (random, systematic), and time of test (immediate, week delay).

**0.3 0.4 0.5 0.6 0.7 0.8 0.9**

**P(Old)**

**Old New**

**2.4 Results – Transfer classification and recognition** 

between hits and false alarms of .304, .176, .167, and .100, respectively.

systematic.

deteriorated a slight 4%).

**0.3 0.4 0.5 0.6 0.7 0.8 0.9**

**P(Old)**

**0.3 0.4 0.5 0.6 0.7 0.8 0.9**

**P(Old)**

**Imm Week Imm Week Rand-VH Rand-HV**

**Random Learning, Switched Modality**

**Imm Week Imm Week Rand-VV Rand-HH**

**Random Learning, Same Modality**

Fig. 2. A schematic representation of the three categories and 24 objects

#### **2.3 Results - Learning**

Figure 3 shows the mean correct classification rate across learning blocks as a function of input modality and training order (systematic, random). The main effects of learning blocks, modality of training, and order of stimulus presentation during learning were each significant.

Fig. 3. Learning across training blocks as a function of modality and order of presentation.

*P3*

**Six Intermediate Exemplars**

**Three Prototypes**

**Exemplars**

**2.3 Results - Learning** 

**A SCHEMATIC REPRESENTATION OF THE THREE CATEGORIES** 

**21 14**

*20 13*

**19 12**

*CATEGORY 3*

**Three Midpoints Twelve Learning** 

*18 11*

**17 10**

*16 9*

**15 8**

*CATEGORY 1 CATEGORY 2*

*P1* **1** *2* **3** *4* **5** *6* **7** *P2*

Figure 3 shows the mean correct classification rate across learning blocks as a function of input modality and training order (systematic, random). The main effects of learning blocks, modality of training, and order of stimulus presentation during learning were each significant.

**Learning**

**Random Haptic Random Visual Syst Haptic Syst Visual**

Fig. 3. Learning across training blocks as a function of modality and order of presentation.

**1234 Trial Blocks**

Fig. 2. A schematic representation of the three categories and 24 objects

**0.7 0.75 0.8 0.85**

**0.9 0.95**

**Proportion Correct**

**1**

In general, performance improved across learning blocks, learning was more efficient with visual than haptic inspection, and performance was enhanced when study presentation was systematic.

#### **2.4 Results – Transfer classification and recognition**

Classification errors were unexpectedly rare, with overall error rates ranging between 3-10% among the four modality conditions, with accuracy highest in the VV condition and worst in the HH condition (participants were also tested one week later, and performance deteriorated a slight 4%).

Figure 4 shows the mean hit and false alarm rates as a function of study and test modality (VV, VH, HV, HH), training order (random, systematic), and time of test (immediate, week delay). In general, subjects were able to discriminate old from new objects with fair accuracy, with an overall hit rate of .715 and a false alarm rate of .543. The conditions ordered themselves, from best to poorest old-new discrimination, as VV > VH = HH > HV, with a mean difference between hits and false alarms of .304, .176, .167, and .100, respectively.

Fig. 4. Mean hit and false alarm rates as a function of study and test modality (VV, VH, HV, HH), training order (random, systematic), and time of test (immediate, week delay).

Haptic Concepts 11

terminal level of learning was virtually the same in each case. Surprisingly, classification on the transfer test, even when switched to a different modality, was remarkably accurate, with error rates ranging from 2-10%; the impact of a test delayed by one week was statistically

The greatest differences occurred in recognition, where again the visual modality generally resulted in superior performance. The visual-visual (VV) condition, compared to the haptichaptic (HH) condition, revealed the general advantage of the visual modality for the same objects, and would be consistent with the general hypothesis that the visual modality

Recognition accuracy was slightly worse in the cross-modality conditions, with better discrimination found for visual study and haptic test than the reverse. This suggests that visual encoding provides considerably more information than haptic encoding, and that this difference remains even following haptic testing. A simple model is to assume that the visual modality encodes more features than does the haptic modality, and that each modality can transfer a proportion of these features to the alternate modality. For example, suppose that 80 features have been encoded and stored for each category following visual learning; for the haptic modality, 40 features are encoded. If 50% of all features can be transferred to the alternate modality, then the number of features available at the time of transfer would be 80(1.0) = 80 for VV, 80(.50) = 40 for VH, 40(1.0) = 40 for HH, and 40(.50) =

This experiment addressed whether categories can be learned when the objects, simultaneously explored visually and haptically, were actually different although from the same category. Following each study block, the subject was tested by presenting the study objects either visually, haptically, or both visually and haptically. This was repeated four

One hypothesis is that cross-modal conflict should retard learning, because of the inconsistency of information available during study. Alternatively, presenting information that is available to both modalities, even when in conflict, could provide additional cues for learning. Since subjects were not told that the objects would be different, and since the differences among the patterns belonging to the same category were not strikingly obvious and encoded by different modalities, it is possible that the visually sensed and felt information for a given 'stimulus' might be integrated into a coherent percept. Since the features encoded visually and haptically could differ, at least for some percentage of the encoded features (Miller, 1972), any integration from the two modalities could, in principle,

Alternatively, the subject could learn two versions for each category, one visual and one haptic, with integration between the modalities playing no role. It is worth stressing that the objects studied visually and haptically for each category were identical; only the pairing on each study trial was inconsistent. Since learning more categories has been found to retard learning but enhance later transfer (Homa & Chambliss, 1975), the formation of multiplemodality categories would predict that learning rate would be slowed by this manipulation

encodes more (or more accurate) information than the haptic modality.

20 for HV, an ordering that matched that obtained in recognition.

**3. Intermodal conflict in category learning and transfer** 

times, followed by a transfer test similar to that used in Experiment 1.

result in a more robust concept.

but produce more accurate later transfer.

significant but minimal in terms of absolute loss.

Testing in an alternate modality provides an index of level of transfer between these modalities. The overall level of discrimination between old and new objects was .304 for the VV condition versus .176 for the VH, which suggests that transfer was substantial but with some loss of information from the visual to the haptic modality. The HH/HV contrast provides an index of conceptual transfer from the haptic to the visual modality. The overall level of discrimination between old and new was .167 for HH; for HV, discrimination dropped to .100. Differences in performance between the HH and VH must reflect encoding (and transfer) from one modality to the other, given a common test modality. No overall differences in recognition discrimination emerged between these conditions (HH = .167; VH = .176), either as main effects or interactions. For the VV vs. HV condition, the difference in discrimination accuracy (VV = .304; HV = .100) was substantial.

In spite of the wide variations in transfer, each of the conditions – transfer to the same or alternate modality – revealed that the ability to discriminate old from new objects was significant even after a week delay. In particular, our expectation that discrimination in the alternate modality would vanish after one week was not supported.

Figure 5 shows the probability each object type (old, new, prototype, midpoint) was called old as a function of learning and transfer modality. In general, subjects were most accurate in identification of old patterns as 'old'; the midpoint, prototype, and new objects were (incorrectly) called 'old' at rates of .459, .539, and .586, respectively. A notable result was that the category prototype, often false alarmed at a higher rate than other new patterns (e.g., Metcalfe & Fisher, 1986), was incorrectly called 'old' no more often than other new objects. This replicates previous studies which have found that the prototype, when composed of continuously variable features, is likely represented as a novel, ideal pattern, not a familiar one (Homa et al., 1993; 2001).

Fig. 5. Probability of calling a stimulus 'old' as a function of condition.

## **2.5 Conclusion**

As expected, the categories were learned more rapidly when presented visually than haptically and when presented in a systematic rather than a random order. However, the

Testing in an alternate modality provides an index of level of transfer between these modalities. The overall level of discrimination between old and new objects was .304 for the VV condition versus .176 for the VH, which suggests that transfer was substantial but with some loss of information from the visual to the haptic modality. The HH/HV contrast provides an index of conceptual transfer from the haptic to the visual modality. The overall level of discrimination between old and new was .167 for HH; for HV, discrimination dropped to .100. Differences in performance between the HH and VH must reflect encoding (and transfer) from one modality to the other, given a common test modality. No overall differences in recognition discrimination emerged between these conditions (HH = .167; VH = .176), either as main effects or interactions. For the VV vs. HV condition, the difference in

In spite of the wide variations in transfer, each of the conditions – transfer to the same or alternate modality – revealed that the ability to discriminate old from new objects was significant even after a week delay. In particular, our expectation that discrimination in the

Figure 5 shows the probability each object type (old, new, prototype, midpoint) was called old as a function of learning and transfer modality. In general, subjects were most accurate in identification of old patterns as 'old'; the midpoint, prototype, and new objects were (incorrectly) called 'old' at rates of .459, .539, and .586, respectively. A notable result was that the category prototype, often false alarmed at a higher rate than other new patterns (e.g., Metcalfe & Fisher, 1986), was incorrectly called 'old' no more often than other new objects. This replicates previous studies which have found that the prototype, when composed of continuously variable features, is likely represented as a novel, ideal pattern,

discrimination accuracy (VV = .304; HV = .100) was substantial.

alternate modality would vanish after one week was not supported.

Fig. 5. Probability of calling a stimulus 'old' as a function of condition.

**0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9**

**Proportion Old**

As expected, the categories were learned more rapidly when presented visually than haptically and when presented in a systematic rather than a random order. However, the

**VV HV VH HH Transfer Objects**

**Old New Pro Mid**

not a familiar one (Homa et al., 1993; 2001).

**2.5 Conclusion** 

terminal level of learning was virtually the same in each case. Surprisingly, classification on the transfer test, even when switched to a different modality, was remarkably accurate, with error rates ranging from 2-10%; the impact of a test delayed by one week was statistically significant but minimal in terms of absolute loss.

The greatest differences occurred in recognition, where again the visual modality generally resulted in superior performance. The visual-visual (VV) condition, compared to the haptichaptic (HH) condition, revealed the general advantage of the visual modality for the same objects, and would be consistent with the general hypothesis that the visual modality encodes more (or more accurate) information than the haptic modality.

Recognition accuracy was slightly worse in the cross-modality conditions, with better discrimination found for visual study and haptic test than the reverse. This suggests that visual encoding provides considerably more information than haptic encoding, and that this difference remains even following haptic testing. A simple model is to assume that the visual modality encodes more features than does the haptic modality, and that each modality can transfer a proportion of these features to the alternate modality. For example, suppose that 80 features have been encoded and stored for each category following visual learning; for the haptic modality, 40 features are encoded. If 50% of all features can be transferred to the alternate modality, then the number of features available at the time of transfer would be 80(1.0) = 80 for VV, 80(.50) = 40 for VH, 40(1.0) = 40 for HH, and 40(.50) = 20 for HV, an ordering that matched that obtained in recognition.
