**3.2 Map and validate recommendations**

To reiterate, the Think Aloud Protocol and Digital Content Extraction were conducted concurrently. This produced an exhaustive list of recommendations to


#### **Table 1.**

*Synthesised recommendations extracted from think aloud protocol and digital content extraction.*


**Table 2.**

*Cohen's kappa score for mapping of recommendations.*

develop smartphone applications for tech-savvy OAs. Some of these are described earlier along with sample quotations from the participants. There were a total of 131 recommendations for Usability with some overlapping [36]. However, for the sake of preciseness, a subset of synthesised versions of these recommendations mapped against the source from which they were derived are presented in **Table 1**. A total of 14 design patterns were created from these recommendations, and 3 of them are presented in this study.

Each unique recommendation was mapped by all authors against the 7 Usability categories outlined by Morville's Usability honeycomb and then validated using Inter-rater reliability as described in Section 2. The scores for inter-rater reliability exceeded 0.61, indicating a high level of reliability. The values of Cohen Kappa's coefficient (**κ**) for Usability, and all raters (author 1, 2 and 3) are shown in **Table 2**.

#### **4. Transform to design patterns**

We developed numerous design patterns for Usability after conducting Think Aloud and Digital Content Extraction. Patterns, derived from these, also include supporting literature, thus providing further information about how to develop applications for OAs. Other authors have used patterns to describe usability design recommendations, such as [20, 26]. We build on their work by applying a range of recommendations in this format. Design patterns are shown to be beneficial [60]:


**Table 3.** *Make app more accessible with easy log-in id - e.g. fingerprint instead of password.*

• Improve communication between stakeholders through providing a common vocabulary e.g., developers and maintainers.


#### **Table 4.**

*Avoid animations and marquees (e.g. text moving from top to bottom) in the application.*


#### **Table 5.**

*Incorporate auto captioning for disabled users in the app.*


Due to limitations on the size and volume of this chapter, we present a subset of three example design patterns covering sub categories of usability, i.e. Usable, Desirable and Accessible, in **Tables 3**–**5**. The remaining set of design patterns can be accessed at [36]. Thus, these design patterns are the answer to the research question posed earlier:

**Research Question -** What do tech-savvy older adults expect from smart-phone applications?

## **5. Threats to validity**

It is imperative to consider limitations and threats to validity of research [61]. This is particularly important for empirical studies in software engineering, where there is often a multitude of possible threats [62]. First and foremost, the targeted population of this study was tech-savvy older adults, so the findings might not be applicable as-is to age-related counterparts who experience further difficulties in learning and adopting new technology. However, additional findings which included non-tech savvy OAs are presented in [36]. Moreover, for construct validity, field notes taken during the two studies were added to the data collection form on the same day of the sessions. To improve participants' understanding, an information sheet was provided written in English, particularly for Think Aloud sessions. Moreover, Digital Content Extraction involved older adults from across the globe, which may have enhanced the external validity of this research. A known problem with every form of research is that the experience of the researcher can influence both the data collection and analysis positively or negatively. We tried to reduce this bias by involving all three researchers in evaluating the codes and by checking the criteria for data selection. In the context of Think Aloud sessions, the setting was only an approximation of a real-life situation, not the actual day-to-day life usage of technology by older adults. The primary limitation of the examination of online ageing forums was the focus on two forums only. Also, a limited sample of applications was selected from the mainstream app stores using stratified sampling. Despite the limitations, the results follow from the data collected via empirical studies. For reliability, the evaluation of recommendations, through the use of inter-rater reliability, added confidence that the study would yield similar results if other researchers were to replicate our methods.
