**2. Graphical access for people with visual impairments**

## **2.1 The persistent challenge of graphical access**

We start by highlighting an important distinction of nonvisual information access between textual and nontextual information sources. Access to printed, text-based material has largely been solved for BVI individuals owing to significant advances over the past 30 plus years in the development of screen-reading software using text-to-speech engines (e.g., JAWS for Windows [2] or VoiceOver for the Mac and iOS-based devices [3]). Indeed, long before these digital speech-based solutions, the Braille code provided a robust system for conveying alpha-numeric information, as well as other literary, mathematical, and musical symbols that are embossed on hardcopy paper (for a review of the history of Braille, see [4]). The development of dynamic, refreshable Braille-display technologies since the 1970s has provided access to the braille code for real-time access to text, often in conjunction with synthetic speech via the aforementioned screen reader software packages. These hardware and software solutions differ widely in their form factor, connectivity, available features, and languages supported but they share a common shortcoming--they are limited to only providing access to textual information. The crux of the problem is that graphical information is almost exclusively rendered visually. In contrast to accessing text-based material, there is no analogous low-cost, intuitive, and commercially available solution for providing individuals who are BVI with dynamic access to visually rendered graphical content. Compounding the problem, compared to the wealth of knowledge that exists about human visual

**111**

these domains.

of these materials.

**2.2 Current solutions for graphical access**

*The Graphical Access Challenge for People with Visual Impairments: Positions and Pathways…*

information processing, there is far less basic research addressing the sensory, perceptual, and cognitive factors that are critical for accurate encoding, interpretation, and representation of graphical information rendered using nonvisual channels such as audition or touch. While earlier studies have evaluated many human information processing characteristics for tangible graphics (i.e., pressure based physical stimuli) [5–9], these results cannot ensure saliency when adopted for rendering digital graphical elements on touchscreen interfaces (see [10, 11] for discussion). The reason stems from the nature of the stimuli and its mechanism of delivery. Vibrations from flat touchscreens provide no direct cutaneous cues as are afforded with traditional raised tangible graphics, and they trigger different sensory receptors compared to what is used when encoding traditional "raised" tactile graphics or

Lack of access to graphical material is more than a mere frustration or hindrance. Indeed, we argue that it represents one of the biggest challenges to the independence and productivity of individuals who are BVI and has had significant detrimental effects on the educational, vocational, and social prospects for this demographic. In support, consider troubling statistics that have estimated that up to 30% of blind people do not travel independently outside of their home [12], that only ~11% of persons who are BVI have a bachelor's degree [13], and that over 70% of this demographic is unemployed or under-employed [14, 15]. This is not an isolated problem: over 12 million people in the U.S. and 285 million people worldwide are estimated as having some form of significant and uncorrected visual impairment [16]. Unfortunately, this problem is rapidly growing, and the current information gap will likely widen without a tractable solution as: (1) the incidence of people experiencing visual impairment is projected to double by 2030 owing to the aging of our population [17], (2) graphics are increasingly being used as the preferred medium of information exchange, and (3) print-based content is rapidly moving to the digital space. The growing reliance on graphical content is especially evident in educational contexts, where it has been estimated that scientific textbooks and journals contain 1.3 graphical representations per page [18]. The inability for students who are BVI to access this rich graphical content certainly helps explain the particularly low inclusion and success of this demographic in STEM disciplines [19, 20]. Outside of information access in education, the lack of accessibility of many sources of information used in daily life also inevitably contributes to the greater social isolation and depression experienced by individuals who are BVI [21]. Without question, a significant component of improving these statistics (and more importantly, benefitting the lives of BVI individuals at large) involves solving the long-standing information gap caused by lack of access to graphical materials in

Traditional approaches to creating accessible, tangible graphics, include the use of: (1) a tactile embosser to produce hardcopy raised graphics (e.g., the Tiger embosser [22]); (2) renderings made on heat-sensitive swell paper (e.g., [23]); (3) physical manipulatives that are pinned or velcroed to a board [24]; or more recently, (4) 3D-printed models or manipulatives [25]. **Figure 1** provides examples

While these techniques certainly work, they also have several significant shortcomings that limit their efficacy as a robust and broadly applicable solution. The principle drawbacks of these solutions include: (1) the authoring process is often slow and cumbersome and typically requires an individual skilled in creating tactile graphics, (2) the equipment can be prohibitively expensive (e.g., a Tiger embosser

*DOI: http://dx.doi.org/10.5772/intechopen.82289*

models.

#### *The Graphical Access Challenge for People with Visual Impairments: Positions and Pathways… DOI: http://dx.doi.org/10.5772/intechopen.82289*

information processing, there is far less basic research addressing the sensory, perceptual, and cognitive factors that are critical for accurate encoding, interpretation, and representation of graphical information rendered using nonvisual channels such as audition or touch. While earlier studies have evaluated many human information processing characteristics for tangible graphics (i.e., pressure based physical stimuli) [5–9], these results cannot ensure saliency when adopted for rendering digital graphical elements on touchscreen interfaces (see [10, 11] for discussion). The reason stems from the nature of the stimuli and its mechanism of delivery. Vibrations from flat touchscreens provide no direct cutaneous cues as are afforded with traditional raised tangible graphics, and they trigger different sensory receptors compared to what is used when encoding traditional "raised" tactile graphics or models.

Lack of access to graphical material is more than a mere frustration or hindrance. Indeed, we argue that it represents one of the biggest challenges to the independence and productivity of individuals who are BVI and has had significant detrimental effects on the educational, vocational, and social prospects for this demographic. In support, consider troubling statistics that have estimated that up to 30% of blind people do not travel independently outside of their home [12], that only ~11% of persons who are BVI have a bachelor's degree [13], and that over 70% of this demographic is unemployed or under-employed [14, 15]. This is not an isolated problem: over 12 million people in the U.S. and 285 million people worldwide are estimated as having some form of significant and uncorrected visual impairment [16]. Unfortunately, this problem is rapidly growing, and the current information gap will likely widen without a tractable solution as: (1) the incidence of people experiencing visual impairment is projected to double by 2030 owing to the aging of our population [17], (2) graphics are increasingly being used as the preferred medium of information exchange, and (3) print-based content is rapidly moving to the digital space. The growing reliance on graphical content is especially evident in educational contexts, where it has been estimated that scientific textbooks and journals contain 1.3 graphical representations per page [18]. The inability for students who are BVI to access this rich graphical content certainly helps explain the particularly low inclusion and success of this demographic in STEM disciplines [19, 20]. Outside of information access in education, the lack of accessibility of many sources of information used in daily life also inevitably contributes to the greater social isolation and depression experienced by individuals who are BVI [21]. Without question, a significant component of improving these statistics (and more importantly, benefitting the lives of BVI individuals at large) involves solving the long-standing information gap caused by lack of access to graphical materials in these domains.

#### **2.2 Current solutions for graphical access**

Traditional approaches to creating accessible, tangible graphics, include the use of: (1) a tactile embosser to produce hardcopy raised graphics (e.g., the Tiger embosser [22]); (2) renderings made on heat-sensitive swell paper (e.g., [23]); (3) physical manipulatives that are pinned or velcroed to a board [24]; or more recently, (4) 3D-printed models or manipulatives [25]. **Figure 1** provides examples of these materials.

While these techniques certainly work, they also have several significant shortcomings that limit their efficacy as a robust and broadly applicable solution. The principle drawbacks of these solutions include: (1) the authoring process is often slow and cumbersome and typically requires an individual skilled in creating tactile graphics, (2) the equipment can be prohibitively expensive (e.g., a Tiger embosser

*Interactive Multimedia - Multimedia Production and Digital Storytelling*

**2. Graphical access for people with visual impairments**

We start by highlighting an important distinction of nonvisual information access between textual and nontextual information sources. Access to printed, text-based material has largely been solved for BVI individuals owing to significant advances over the past 30 plus years in the development of screen-reading software using text-to-speech engines (e.g., JAWS for Windows [2] or VoiceOver for the Mac and iOS-based devices [3]). Indeed, long before these digital speech-based solutions, the Braille code provided a robust system for conveying alpha-numeric information, as well as other literary, mathematical, and musical symbols that are embossed on hardcopy paper (for a review of the history of Braille, see [4]). The development of dynamic, refreshable Braille-display technologies since the 1970s has provided access to the braille code for real-time access to text, often in conjunction with synthetic speech via the aforementioned screen reader software packages. These hardware and software solutions differ widely in their form factor, connectivity, available features, and languages supported but they share a common shortcoming--they are limited to only providing access to textual information. The crux of the problem is that graphical information is almost exclusively rendered visually. In contrast to accessing text-based material, there is no analogous low-cost, intuitive, and commercially available solution for providing individuals who are BVI with dynamic access to visually rendered graphical content. Compounding the problem, compared to the wealth of knowledge that exists about human visual

**2.1 The persistent challenge of graphical access**

are explored in the digital domain.

as touchscreens.

text-based descriptions are only sometimes present, and often do not tell the whole story as they are not designed to do so. Unfortunately, their inclusion is more often the exception than the rule and when available, the description is generally short and imprecise, failing to capture much of the information conveyed by the graphical rendering. One need only to read a few alt tags of graphics on the web to demonstrate how poorly these text descriptions convey what is represented in the graphical depiction. Diversification of design to meet a range of accessibility needs in the digital space can make the information given more valuable to users who must access information in a different way [1]. With more content moving to the electronic space, it is paramount that new solutions for graphical information access

The aim of this chapter is to discuss some of the factors underlying the graphics access problem faced by people who are BVI and to describe the latest class of technologies and techniques that we believe have the most potential to mitigate the problem. We first characterize the persistent challenges that have perpetuated this long-standing information access issue. We then describe some general approaches developed throughout the years to address this challenge. We specifically focus on the role of touchscreen-based smart devices (e.g., phones and tablets), which our group believes is a promising solution moving forward. We then discuss some of the advantages and disadvantages of these devices and share a few ideological positions that we believe must be advanced if we are to truly address the graphical access challenge in the context of new technology development. This chapter sets forth a clear position on the efficacy of this class of information access technology (IAT) and advocates some paradigm shifts in the way that we think about addressing this vexing access problem. It is also meant to serve as a reference for researchers and developers interested in promoting graphical accessibility via new technologies such

**110**

#### **Figure 1.**

*Examples of traditional methods used to convey graphics (e.g., swell paper, embosser, Wikki Stix).*

can cost between \$5,000 and \$15,000, see [22]), (3) the technology is based on single-purpose hardware often requiring individuals to use an "army of devices" in their daily life, (4) the output is a static representation that can quickly become obsolete and is neither easy nor quick to update, and (5) the output is largely restricted to a single modality (i.e., touch). A lengthier discussion of these limitations and the challenges they pose can be found in Ducasse, Brock, and Jouffrais' review of maps for individuals with BVI [26].

Some of these barriers have been addressed through technology development, with the biggest benefit coming from the use of dynamic touch-based interfaces. For instance, a host of refreshable tactual technologies have been developed based on force feedback, refreshable pin arrays, micro fluidics, and moldable alloys. The thorough review by O'Modhrain and colleagues details the pros and cons of each of these approaches [27]. While such technology developments are pushing the boundaries of new haptic technologies as a means for access, these solutions are not widely available nor broadly adopted. This is likely due to several factors including the high cost and lack of commercial availability associated with most of the haptic systems, the in-depth manufacturing and fabrication process required for some of the technologies, and the need for additional hardware that only adds to the host of access devices and technologies already used by BVI persons.

The promise of low-cost, large-format, dot-based graphic displays has been made for decades and some examples are or were commercially available, such as the DotView from KGS Corporation [28] or the Graphic Window by Handytech [29, 30]. Other approaches have exploited auditory solutions, converting the visually-based information into an acoustic format that employs different sonification techniques and auditory parameters (e.g., pitch, loudness, timbre, or tempo) to convey the graphical content [31–33]. Additional efforts have explored utilizing languagebased descriptions to convey graphical information [34, 35]. Auditory and verbal approaches, however, are not optimal as they are based on an interpretive medium that requires cognitive mediation and greater maintenance in attention [36]. Such feedback can also be distracting when accessing information in quiet environments such as classrooms or in a meeting while simultaneously trying to listen to presenters. In addition, we argue that these auditory/linguistic approaches are not as suited to conveying spatial graphics as are touch-based solutions because they do not directly specify spatial relations or provide the necessary kinesthetic feedback that enables spatial organization of information.

The above notable approaches have certainly pushed the possibilities of graphical access, yet it is important to note that simply providing dynamic nonvisual information is not sufficient for conveying and learning graphical materials. In order to effectively meet the larger purpose of what is needed to truly solve the information gap, it is necessary to consider design characteristics that will lead to user acceptance and adoption by the BVI community. These factors include being inexpensive, multi-purpose, multimodal, and readily available. Indeed, many of the solutions discussed above are generally relegated to highly specialized applications and require purpose-built equipment that is designed for specific users, to support

**113**

**Figure 2.**

*The Graphical Access Challenge for People with Visual Impairments: Positions and Pathways…*

specific tasks or needs, in a specific situation or environment. This specificity means that most haptic IATs, even if effective, are too expensive, too limited in their usage applications, too cumbersome, and unduly subject to obsolescence to be viable, long-term information-access solutions for BVI users. There are a growing number of new technologies coming to market that build upon previous work, such as the Graphiti, American Printing House (APH)'s dynamic touch-sensitive pin array [37]; the BLITAB tablet, which is capable of a full page of braille [38]; shapeShift, a refreshable multi-height pin display that can render 3D objects and dynamic movement [39]; and microfluidic-based tablets that are capable of refreshable, raised dots on tablets (e.g., [40, 27]) (see **Figure 2**). Most of these devices, however, are still in the research phase, and many still suffer from high component costs or reliance on hardware-specific platforms, thereby reducing the likelihood of

While the above innovative approaches have various benefits, we posit that a more broadly adoptable solution is to use technology that: (1) provides direct perceptual access to the graphical content, as is the case via visual access, (2) is (or could be) mass marketed and readily available among end users, and (3) is based on a computational platform that can be leveraged for other functions/activities. We argue that this is best accomplished using dynamic touch-based (or multimodal) displays implemented on smart devices (phones/tablets). We believe that interfaces leveraging direct touch access are critical in solving the graphical access problem as touch has much in common with visual spatial perception, sharing many parallels with the visual pathways in the brain (e.g., [41, 42]). For example, both modalities extract the basic features and spatiality of an object in the environment and integrate this information to form a complete, coherent representation of the object

*New innovative solutions being developed for individuals with BVI: upper left—demonstration of shapeShift (multi-height pin array) [39]; upper right—Graphiti (refreshable pin array) [37]; lower left—BLITAB* 

*(refreshable pin tablet) [38]; and lower right—Holy Braille (microfluidic tablet) [40].*

*DOI: http://dx.doi.org/10.5772/intechopen.82289*

such devices becoming a mainstream solution.

#### *The Graphical Access Challenge for People with Visual Impairments: Positions and Pathways… DOI: http://dx.doi.org/10.5772/intechopen.82289*

specific tasks or needs, in a specific situation or environment. This specificity means that most haptic IATs, even if effective, are too expensive, too limited in their usage applications, too cumbersome, and unduly subject to obsolescence to be viable, long-term information-access solutions for BVI users. There are a growing number of new technologies coming to market that build upon previous work, such as the Graphiti, American Printing House (APH)'s dynamic touch-sensitive pin array [37]; the BLITAB tablet, which is capable of a full page of braille [38]; shapeShift, a refreshable multi-height pin display that can render 3D objects and dynamic movement [39]; and microfluidic-based tablets that are capable of refreshable, raised dots on tablets (e.g., [40, 27]) (see **Figure 2**). Most of these devices, however, are still in the research phase, and many still suffer from high component costs or reliance on hardware-specific platforms, thereby reducing the likelihood of such devices becoming a mainstream solution.

While the above innovative approaches have various benefits, we posit that a more broadly adoptable solution is to use technology that: (1) provides direct perceptual access to the graphical content, as is the case via visual access, (2) is (or could be) mass marketed and readily available among end users, and (3) is based on a computational platform that can be leveraged for other functions/activities. We argue that this is best accomplished using dynamic touch-based (or multimodal) displays implemented on smart devices (phones/tablets). We believe that interfaces leveraging direct touch access are critical in solving the graphical access problem as touch has much in common with visual spatial perception, sharing many parallels with the visual pathways in the brain (e.g., [41, 42]). For example, both modalities extract the basic features and spatiality of an object in the environment and integrate this information to form a complete, coherent representation of the object

#### **Figure 2.**

*Interactive Multimedia - Multimedia Production and Digital Storytelling*

review of maps for individuals with BVI [26].

**Figure 1.**

enables spatial organization of information.

access devices and technologies already used by BVI persons.

can cost between \$5,000 and \$15,000, see [22]), (3) the technology is based on single-purpose hardware often requiring individuals to use an "army of devices" in their daily life, (4) the output is a static representation that can quickly become obsolete and is neither easy nor quick to update, and (5) the output is largely restricted to a single modality (i.e., touch). A lengthier discussion of these limitations and the challenges they pose can be found in Ducasse, Brock, and Jouffrais'

*Examples of traditional methods used to convey graphics (e.g., swell paper, embosser, Wikki Stix).*

Some of these barriers have been addressed through technology development, with the biggest benefit coming from the use of dynamic touch-based interfaces. For instance, a host of refreshable tactual technologies have been developed based on force feedback, refreshable pin arrays, micro fluidics, and moldable alloys. The thorough review by O'Modhrain and colleagues details the pros and cons of each of these approaches [27]. While such technology developments are pushing the boundaries of new haptic technologies as a means for access, these solutions are not widely available nor broadly adopted. This is likely due to several factors including the high cost and lack of commercial availability associated with most of the haptic systems, the in-depth manufacturing and fabrication process required for some of the technologies, and the need for additional hardware that only adds to the host of

The promise of low-cost, large-format, dot-based graphic displays has been made for decades and some examples are or were commercially available, such as the DotView from KGS Corporation [28] or the Graphic Window by Handytech [29, 30]. Other approaches have exploited auditory solutions, converting the visually-based information into an acoustic format that employs different sonification techniques and auditory parameters (e.g., pitch, loudness, timbre, or tempo) to convey the graphical content [31–33]. Additional efforts have explored utilizing languagebased descriptions to convey graphical information [34, 35]. Auditory and verbal approaches, however, are not optimal as they are based on an interpretive medium that requires cognitive mediation and greater maintenance in attention [36]. Such feedback can also be distracting when accessing information in quiet environments such as classrooms or in a meeting while simultaneously trying to listen to presenters. In addition, we argue that these auditory/linguistic approaches are not as suited to conveying spatial graphics as are touch-based solutions because they do not directly specify spatial relations or provide the necessary kinesthetic feedback that

The above notable approaches have certainly pushed the possibilities of graphical access, yet it is important to note that simply providing dynamic nonvisual information is not sufficient for conveying and learning graphical materials. In order to effectively meet the larger purpose of what is needed to truly solve the information gap, it is necessary to consider design characteristics that will lead to user acceptance and adoption by the BVI community. These factors include being inexpensive, multi-purpose, multimodal, and readily available. Indeed, many of the solutions discussed above are generally relegated to highly specialized applications and require purpose-built equipment that is designed for specific users, to support

**112**

*New innovative solutions being developed for individuals with BVI: upper left—demonstration of shapeShift (multi-height pin array) [39]; upper right—Graphiti (refreshable pin array) [37]; lower left—BLITAB (refreshable pin tablet) [38]; and lower right—Holy Braille (microfluidic tablet) [40].*

formed in memory. This lends credence to parallel or shared channels in perception [43, 44]. Further, auditory and verbal approaches often involve more cognitive effort and are thus less "perceptual" than touch-based or visually-based information displays [45]. This is not to say that auditory and verbal approaches should be ignored. To the contrary, we believe in synergizing all available modalities, as is done in some capacity on current vibrotactile touchscreen platforms today, and leveraging the appropriate constituent inputs for best supporting the information to be rendered and the task to be performed. While there are various types of haptic displays, each with their own strengths and weaknesses, the position advanced in this paper is that vibrotactile stimulation, when paired with a touchscreen equipped smart device (e.g., phone or tablet) and other output channels, is a highly promising approach for solving the nonvisual graphics access problem. We believe this platform is quickly becoming the de facto gold standard for IAT and offers a solution that has a high likelihood of being accepted and adopted among its end users, which should be the goal of any IAT design.

#### **2.3 Why vibrotactile, touchscreen-based smart solutions?**

We have all experienced our phone vibrating in our pocket to indicate an incoming call or to alert us of an upcoming meeting. However, beyond soliciting our attention, providing simple alerts, signaling a confirmation or error, or any number of other instances of secondary or tertiary cuing, people rarely consider the role of vibrotactile feedback as a primary interaction style. On the one hand, this is surprising given the multitude of common interactions we experience that involve vibration in one capacity or another. Consider the slight detents you feel when spinning the scroll wheel on your computer mouse or the volume dial on your car radio, the signal from your electric toothbrush indicating to brush in another location, the rumble from your game controller indicating an undesired behavior, the alert from the buzzer indicating that your party is being summoned at a restaurant, the vibrating seat in your car indicating that you are backing up near an obstruction, and a myriad of other haptic implementations in current technologies that employ vibrotactile cues for nonvisually conveying relevant information. On the other hand, even if informative, this information is usually either an unintended byproduct of an action, (e.g., vibration from approaching an obstacle), or a secondary cue that is part of a primary interface, (e.g., detents that simply provide frictional control over a spinning wheel/dial). They are often not necessary for its function or primary operation. Indeed, rarely is vibrotactile cuing considered as a primary interaction style. In this chapter, we argue that this need not be the case and that vibrotactile feedback is not only vastly underutilized in current interface design but that vibration can serve as a primary mode of user interaction, especially in conditions where visual access is not possible, such as for use by individuals who are BVI or in eyesfree applications (e.g., driving). We now summarize the current state of research on vibrotactile touchscreen displays before sharing four positions our group believes are needed toward addressing the graphical access challenge moving forward.

### **2.4 Research brief on vibrotactile touchscreen displays**

A growing body of research has demonstrated the efficacy of using touchscreen-based devices and vibrotactile or vibrotactile plus auditory information as a primary interaction style for conveying graphical information. Choi and Kuchenbecker provide an excellent review of vibrotactile displays from both a perceptual and technological perspective, summarizing foundational knowledge in this area and providing implementation guidelines for exemplary applications

**115**

**Figure 3.**

*to look at the screen.*

*The Graphical Access Challenge for People with Visual Impairments: Positions and Pathways…*

[46]. Brewster and colleagues have also done extensive work exploring tactile feedback, particularly from mobile platforms, and have demonstrated important findings illustrating how structured tactile messages (Tactons) can be used to communicate information using different vibration features [47–49]. Other research has demonstrated that vibrotactile feedback enables users to complete scrolling and inputting tasks faster on a mobile device compared to interfaces that lack such feedback [50, 51], and can improve textual reading in braille (e.g., [52–55]). More recent examples have focused on using vibrotactile touchscreen platforms for conveying graphics. A recent project has shown that lines (linear and non-linear) and basic shapes (e.g., circles, triangles, squares) can be successfully interpreted and followed nonvisually through haptic, audio, and haptic-audio access on the touchscreen [5]. Further examples demonstrating the efficacy of this approach were shown when exploring grids [56], graphs [57], maps [58], and nonvisual panning and zooming of large format vibrotactile maps that extended beyond the device's display [11, 59]. In aggregate, this research clearly illustrates the broad potential of this multimodal approach. Work with a prototype system, called a vibro-audio interface (VAI), based on a commercial tablet, has shown near identical accuracy between use of the VAI and hardcopy tactile stimuli for graph interpretation, pattern detection, and shape recognition [60]. In corroboration, studies by Gorlewicz and colleagues have demonstrated no significant differences in the interpretation of a variety of graphics including bar graphs, pie charts, tables, number lines, line graphs, and simple maps that were presented in embossed form and displayed multimodally on a touchscreen created by Vital [61, 62]. Not only do these studies show the efficacy of this interface, but also that this multimodal platform can achieve similar performance to the gold standard of hardcopy graphics. More recent work by our group has also explored the effect of screen size on the

*Touchscreens can leverage both auditory and vibrotactile feedback to convey rich information without the need* 

*DOI: http://dx.doi.org/10.5772/intechopen.82289*

*The Graphical Access Challenge for People with Visual Impairments: Positions and Pathways… DOI: http://dx.doi.org/10.5772/intechopen.82289*

[46]. Brewster and colleagues have also done extensive work exploring tactile feedback, particularly from mobile platforms, and have demonstrated important findings illustrating how structured tactile messages (Tactons) can be used to communicate information using different vibration features [47–49]. Other research has demonstrated that vibrotactile feedback enables users to complete scrolling and inputting tasks faster on a mobile device compared to interfaces that lack such feedback [50, 51], and can improve textual reading in braille (e.g., [52–55]). More recent examples have focused on using vibrotactile touchscreen platforms for conveying graphics. A recent project has shown that lines (linear and non-linear) and basic shapes (e.g., circles, triangles, squares) can be successfully interpreted and followed nonvisually through haptic, audio, and haptic-audio access on the touchscreen [5]. Further examples demonstrating the efficacy of this approach were shown when exploring grids [56], graphs [57], maps [58], and nonvisual panning and zooming of large format vibrotactile maps that extended beyond the device's display [11, 59]. In aggregate, this research clearly illustrates the broad potential of this multimodal approach. Work with a prototype system, called a vibro-audio interface (VAI), based on a commercial tablet, has shown near identical accuracy between use of the VAI and hardcopy tactile stimuli for graph interpretation, pattern detection, and shape recognition [60]. In corroboration, studies by Gorlewicz and colleagues have demonstrated no significant differences in the interpretation of a variety of graphics including bar graphs, pie charts, tables, number lines, line graphs, and simple maps that were presented in embossed form and displayed multimodally on a touchscreen created by Vital [61, 62]. Not only do these studies show the efficacy of this interface, but also that this multimodal platform can achieve similar performance to the gold standard of hardcopy graphics. More recent work by our group has also explored the effect of screen size on the

#### **Figure 3.**

*Touchscreens can leverage both auditory and vibrotactile feedback to convey rich information without the need to look at the screen.*

*Interactive Multimedia - Multimedia Production and Digital Storytelling*

which should be the goal of any IAT design.

**2.3 Why vibrotactile, touchscreen-based smart solutions?**

**2.4 Research brief on vibrotactile touchscreen displays**

A growing body of research has demonstrated the efficacy of using touchscreen-based devices and vibrotactile or vibrotactile plus auditory information as a primary interaction style for conveying graphical information. Choi and Kuchenbecker provide an excellent review of vibrotactile displays from both a perceptual and technological perspective, summarizing foundational knowledge in this area and providing implementation guidelines for exemplary applications

formed in memory. This lends credence to parallel or shared channels in perception [43, 44]. Further, auditory and verbal approaches often involve more cognitive effort and are thus less "perceptual" than touch-based or visually-based information displays [45]. This is not to say that auditory and verbal approaches should be ignored. To the contrary, we believe in synergizing all available modalities, as is done in some capacity on current vibrotactile touchscreen platforms today, and leveraging the appropriate constituent inputs for best supporting the information to be rendered and the task to be performed. While there are various types of haptic displays, each with their own strengths and weaknesses, the position advanced in this paper is that vibrotactile stimulation, when paired with a touchscreen equipped smart device (e.g., phone or tablet) and other output channels, is a highly promising approach for solving the nonvisual graphics access problem. We believe this platform is quickly becoming the de facto gold standard for IAT and offers a solution that has a high likelihood of being accepted and adopted among its end users,

We have all experienced our phone vibrating in our pocket to indicate an incom-

ing call or to alert us of an upcoming meeting. However, beyond soliciting our attention, providing simple alerts, signaling a confirmation or error, or any number of other instances of secondary or tertiary cuing, people rarely consider the role of vibrotactile feedback as a primary interaction style. On the one hand, this is surprising given the multitude of common interactions we experience that involve vibration in one capacity or another. Consider the slight detents you feel when spinning the scroll wheel on your computer mouse or the volume dial on your car radio, the signal from your electric toothbrush indicating to brush in another location, the rumble from your game controller indicating an undesired behavior, the alert from the buzzer indicating that your party is being summoned at a restaurant, the vibrating seat in your car indicating that you are backing up near an obstruction, and a myriad of other haptic implementations in current technologies that employ vibrotactile cues for nonvisually conveying relevant information. On the other hand, even if informative, this information is usually either an unintended byproduct of an action, (e.g., vibration from approaching an obstacle), or a secondary cue that is part of a primary interface, (e.g., detents that simply provide frictional control over a spinning wheel/dial). They are often not necessary for its function or primary operation. Indeed, rarely is vibrotactile cuing considered as a primary interaction style. In this chapter, we argue that this need not be the case and that vibrotactile feedback is not only vastly underutilized in current interface design but that vibration can serve as a primary mode of user interaction, especially in conditions where visual access is not possible, such as for use by individuals who are BVI or in eyesfree applications (e.g., driving). We now summarize the current state of research on vibrotactile touchscreen displays before sharing four positions our group believes are needed toward addressing the graphical access challenge moving forward.

**114**

success of these tasks (e.g., tablets versus smaller mobile platforms), and we have shown that performance on a pattern matching task is equivalent across small and large screen sizes [63]. Even though this is a low resolution output mode, these data show that vibrotactile graphics can still be used effectively and accurately when rendered on the smaller form factor of phone-sized smart devices. This is a positive finding, as the majority of BVI users of smart devices are using mobile phones. A recent review by Grussenmeyer and colleagues provides a thorough survey of how touchscreen-based technologies have been used to support information access by people who are BVI and reiterates the prevalent challenges that exist to bring full inclusion to this population [64]. In short, many of these projects suggest promising pathways forward for vibrotactile touchscreens, supported with empirical evidence and positive qualitative feedback of their capacity to convey multimodal information for the interpretation of visual graphics. Moreover, these platforms offer several significant advantages to one-off information access hardware, with the primary benefits being portability, multi-functional use, relative affordability, and widespread adoption and support by the BVI demographic. Indeed, vibrotactile touchscreens provide a robust multimodal framework, which if continually developed in conjunction with advances in touchscreen-based smart devices, has the potential to become the de-facto, universal means for accessing graphics in a multimodal, digital form (for example, see **Figure 3**). A universal, multimodal platform that is widely available is not only beneficial for the BVI population but extends to many others who benefit from multimodal learning platforms and the brain's capacity to process both redundant and complementary information from different senses.
