**4. Phase 2 resource refinement and current evaluation underway**

**•** Use reputable sources of information about medication—general practitioner, pharmacist,

**•** Actively find information about medications taken by the person with dementia, document

The site also described practical strategies to help the person with dementia to take the right medicines at the right time. It was hoped that these strategies would also reduce familial conflict and stress experienced by the caregivers in their medication management role. The main recommendation was to use blister packaged medications; regularly checking for their ongoing safe usage, and as suggested by Zedler et al. [40] to help improve medication adherence. Blister packs also offer benefits for the PLWD and their caregiver by allowing them to see which medications need to be taken when, and whether or not they have been taken appropriately. Given that many caregivers reported crushing medications in our qualitative study, a page was included to alert caregivers that not all medications are safe to crush. The online resource advises PLWD and/or their caregivers to ask their pharmacist or doctor before crushing any medications or to enquire about alternate formulations (e.g. liquid or transdermal

Finally, the online resource also provided additional information regarding other reputable websites, support groups, translation services and organisations which could help facilitate

We aimed to evaluate the pilot website considering our two guiding design principles: the accessibility of the site and the accessibility of the content, especially for our target audience. An adapted version of the 'Quality checklist for reviewing health information' [41] was used to initially assess the resource. This checklist asks the user to review the content, usefulness, appeal, cultural relevance, readability, presentation, accessibility, credibility and need for improvement of the information resource. The checklist, only available in English, was completed by nine users of the pilot resource all of whom gave positive responses. The survey respondents also provided useful suggestions about additional information regarding specific medications, the need for other formats such as a DVD version of the resource, and translation

Evaluation of the pilot site was also undertaken with caregivers from English and Italian support groups, who also suggested a DVD version of the resource, as evident in the following

"Not sure how well used the online resource will be for the older Italian carers who I suspect may not be well versed in using this sort of media however I think

or nurse.

502 Update on Dementia

these and keep an up-to-date list.

patches) that would avoid the need to crush medications.

safe and effective medication management for PLWD.

**3.5. Evaluation of Phase 1 pilot resource**

*3.5.1. Survey evaluation of the pilot resource*

of the resource in to other languages.

participant quote:

*3.5.2. Support group evaluations of the pilot resource*

Background: The original research group was expanded to include researchers with back‐ grounds in nursing, public health, education and technology. Further funding was obtained which enabled the translation of the resources into a third community language; Macedonian, and the migration of the site to WordPress (https://wordpress.com/). This change to hosting platform was made to facilitate future editing and updating through a what-you-see-is-whatyou-get (WYSIWYG) content management system. The results of the initial pilot study were used to inform the second iteration of the website, which is currently available at: (www.de‐ mentiameds.com) (**Figure 1**).

**Figure 1.** Managing medicines for people with dementia home page.

While the Managing Medicines site was designed to enable maximum accessibility for users, by utilising a larger font size and providing written information in an audio format, there remained the need to evaluate the site. This was completed using a mixture of methodologies, which incorporated readability, usability and server log data analysis.

#### **4.1. Usability testing**

Website users inherently rely on their previous internet experiences to inform their online behaviour. Nielsen[42] found that experienced users are faster at doing things on websites that they use often; they are, more confident at clicking and less afraid that they will break something, better at searching, using the mouse and scrolling. As such, usability testing was undertaken to ensure that the Managing Medicines for People with Dementia website met the needs of the end-users. Previous research has shown that usability testing with five partici‐ pants can identify 85% of problems [42].

A convenience sample of ten participants (age range: 26–79 years of age; mean age 51.8 years) completed the usability testing. Participants were all family caregivers of PLWD, spoke English, had used the internet and were willing to be recorded during the usability testing process. Participants were also asked to complete a short demographic survey, which included questions on age, gender, language spoken at home, country of birth, computer and internet use. Participants were subsequently asked to think-aloud as they completed five set tasks. This allowed the researchers insight into their thought process, personal opinions and reactions to the website. The tasks were designed to mimic the actions that website visitors might have for example, "Please change the language settings from English to Italian", and goals e.g. "Please find the phone number for xxx".

The software program Morae Recorder (TechSmith, Okemos, Michigan) was used to conduct the usability testing. Morae records participant's interactions with the website in the form of visual, audio and mouse movements.

A number of small issues were identified through the usability testing. Most participants encountered difficulties when asked to find specific information. Participants suggested that a dedicated page be added to the site with contact details for support services. Currently, this information is embedded within the website. It was also observed that it would have been useful to have incorporated a short practice into the usability testing, to allow participants to become familiar with how Morae operates.

It should be noted that numerous variables affected task completion time, and the successful completion of tasks. Factors included the telling of personal stories, the presence of children during testing, and phones ringing. This supports the observations from another usability testing study with older adults[43].

Participants who reported over 10 hours internet usage per week were more likely to be successful in completing the tasks. It is likely that this difference was observed due to their increased familiarity with website navigation and increased internet skill level.

Medication Management for People Living with Dementia: Development and Evaluation of a Multilingual Information Resource for Family Caregivers of People Living with Dementia http://dx.doi.org/10.5772/64661 505

#### *4.1.1. Method critique*

While the Managing Medicines site was designed to enable maximum accessibility for users, by utilising a larger font size and providing written information in an audio format, there remained the need to evaluate the site. This was completed using a mixture of methodologies,

Website users inherently rely on their previous internet experiences to inform their online behaviour. Nielsen[42] found that experienced users are faster at doing things on websites that they use often; they are, more confident at clicking and less afraid that they will break something, better at searching, using the mouse and scrolling. As such, usability testing was undertaken to ensure that the Managing Medicines for People with Dementia website met the needs of the end-users. Previous research has shown that usability testing with five partici‐

A convenience sample of ten participants (age range: 26–79 years of age; mean age 51.8 years) completed the usability testing. Participants were all family caregivers of PLWD, spoke English, had used the internet and were willing to be recorded during the usability testing process. Participants were also asked to complete a short demographic survey, which included questions on age, gender, language spoken at home, country of birth, computer and internet use. Participants were subsequently asked to think-aloud as they completed five set tasks. This allowed the researchers insight into their thought process, personal opinions and reactions to the website. The tasks were designed to mimic the actions that website visitors might have for example, "Please change the language settings from English to Italian", and goals e.g. "Please

The software program Morae Recorder (TechSmith, Okemos, Michigan) was used to conduct the usability testing. Morae records participant's interactions with the website in the form of

A number of small issues were identified through the usability testing. Most participants encountered difficulties when asked to find specific information. Participants suggested that a dedicated page be added to the site with contact details for support services. Currently, this information is embedded within the website. It was also observed that it would have been useful to have incorporated a short practice into the usability testing, to allow participants to

It should be noted that numerous variables affected task completion time, and the successful completion of tasks. Factors included the telling of personal stories, the presence of children during testing, and phones ringing. This supports the observations from another usability

Participants who reported over 10 hours internet usage per week were more likely to be successful in completing the tasks. It is likely that this difference was observed due to their

increased familiarity with website navigation and increased internet skill level.

which incorporated readability, usability and server log data analysis.

**4.1. Usability testing**

504 Update on Dementia

pants can identify 85% of problems [42].

find the phone number for xxx".

visual, audio and mouse movements.

become familiar with how Morae operates.

testing study with older adults[43].

While the use of dedicated usability software facilitated recording and analysis of sessions, it would be possible to undertake basic usability testing using traditional audio-visual recording methods and researcher notes. Ideally, usability testing should be undertaken by an expert. However, the use of the dedicated software enabled two research assistants to undertake the usability testing. Another possible method of testing would be a Cognitive Walk Through [43]. This involves an independent health researcher working their way through the website checking that the content is clear, uses plain English, and that the website had consistent formatting with images related to the text.

#### **4.2. User research**

In order to gain feedback from a larger number of site users, a site evaluation survey was compiled and made available using SurveyMonkey. Visitors to the site were asked to complete this online evaluation survey, through a message at the bottom of the left-hand side navigation bar. This message was visible on all site pages. The survey contained 26 items and was available for a four-month period in 2015. It included demographic questions as well as two previously validated tools: a modified version of the Perceived Health Web Site Usability Questionnaire for Older Adults (PHWSUQ) [44] and Chew's three screening questions for health literacy [45]. The evaluation was promoted through personal contacts and via articles in relevant state and national newsletters.

#### *4.2.1. Method critique*

While the use of SurveyMonkey allowed a large response rate, it was flawed in that the survey was only available in English, despite the site being promoted to people from Italian and Macedonian backgrounds. While this decision was made due to financial constraints, it is a significant limitation to the evaluation of the site. Further, the majority of respondents were healthcare workers and not the target audience of family caregivers of people living with a dementia. It is possible that healthcare workers are more comfortable using the internet and completing evaluation surveys. One of the strengths of this method was the ability to incor‐ porate previously validated tools into the survey.

#### **4.3. Web analytics**

Google Analytics (https://www.google.com.au/analytics/) was used to track and report website traffic. This program can be easily configured to monitor a specific website and runs behind the scenes, collecting data and information about site visitors. Data on the number of unique site visits, visit duration and country of origin of the visitor were collected. Notably, many site visitors were from outside of Australia.

#### *4.3.1. Method critique*

One of the main benefits of Google Analytics is that it provides insight in to site users' behaviour and location, and is free to use.

#### **4.4. Readability**

Most health information and educational material on dementia are presented at high reading levels [33, 34, 46]. A recent review of online health information found that sites with informa‐ tion on dementia were the hardest to read when compared to 11 other health conditions [36].

The readability levels of the Managing Medicines for People with Dementia website were evaluated using six readability tools:


For each test, the text from the website was copied and pasted into the tool. We were unable to identify any tools that evaluated readability in Macedonian. Results of the readability tests indicated that the written content was somewhat difficult to understand. This result may be reflective of some of the medical terms used, for example "Alzheimer's" and may be improved by rewording the written content of each page to reduce the number of long sentences. It is recognised that reducing both sentence length and word count can improve upon the overall quality of the website [33]. However, it is worth noting that the information presented on the site is quite complex, which is why caregivers experience so many difficulties in managing the medications of their loved ones. Thus, it may not always be possible to avoid complex sentence structures or word use in medication management resources.

#### *4.4.1. Method critique*

Readability testing proved to be both quick and inexpensive to undertake as it did not require any specialist software. As with other studies, readability testing allowed the identification of content problems not found through usability testing alone [43]. The different readability tools produced different readability scores, which is reflective of the different criteria assessed and different formulae used to calculate the scores. Although the readability levels were found to be relatively high, it is worth noting that the information is also available on the website as an audio version in all three languages. This makes the information more accessible to people with stronger oral literacy skills.

A further Readability Analysis using VocabProfile Compleat (VPC) (http://www.lextutor.ca/ vp/comp/) [47] was conducted as part of our site evaluation, to gain a better understand‐ ing of the readability for immigrants whose second language might be English. This tool is useful to analyse the text according to the most frequently used word families in the English language. VPC categorised all the words throughout the Managing Medicines for People with Dementia website into levels of the most common words used in English with 1000 words in each level: first (K-1 words), second (K-2), third (K-3.) thousand. The frequency scores show which words an immigrant is most likely to be exposed to. Those in the lower thousands would likely be known whereas those in the higher thousands, less so. The results of this analysis revealed that the majority of words fell within the first one thousand level (79.32%) and second thousand level (10.61%) categories overall, suggest‐ ing that the word choice throughout the site is accessible to English as a second lan‐ guage speakers.

This frequency information is important to consider because the greater the number of words that fall outside the first two categories (K 1-2), the greater the potential difficulty that second language speakers of English would likely experience in understanding the website, especially if they have limited proficiency in English. Ideally, websites imparting information to second language speakers should try to utilize words that make up at least 95% of the readers' receptive vocabulary. This figure was determined based on Hsueh-Chao and Nation's [48] description of optimal conditions for extensive reading within an English as a Second Lan‐ guage curriculum being 95–98% of known words and Nation's [49] argument that 98–99% is ideal.

#### **4.5. Quality of information on health websites**

**4.4. Readability**

506 Update on Dementia

levels

test.php)

*4.4.1. Method critique*

with stronger oral literacy skills.

tt\_user=guest), and

evaluated using six readability tools:

Most health information and educational material on dementia are presented at high reading levels [33, 34, 46]. A recent review of online health information found that sites with informa‐ tion on dementia were the hardest to read when compared to 11 other health conditions [36].

The readability levels of the Managing Medicines for People with Dementia website were

**1.** Flesch-Kincaid, readability statistics found in Microsoft Word, based on US school grade

**3.** Simple Measure of Gobbledygook (SMOG - http://www.learningandwork.org.uk/misc/

**4.** Dale-Chall readability formula (http://www.readabilityformulas.com/free-dale-chall-

**5.** Italian Read-IT DyLan Text Tool v2.1.9 (http://www.ilc.cnr.it/dylanlab/apps/texttools/?

For each test, the text from the website was copied and pasted into the tool. We were unable to identify any tools that evaluated readability in Macedonian. Results of the readability tests indicated that the written content was somewhat difficult to understand. This result may be reflective of some of the medical terms used, for example "Alzheimer's" and may be improved by rewording the written content of each page to reduce the number of long sentences. It is recognised that reducing both sentence length and word count can improve upon the overall quality of the website [33]. However, it is worth noting that the information presented on the site is quite complex, which is why caregivers experience so many difficulties in managing the medications of their loved ones. Thus, it may not always be possible to avoid complex sentence

Readability testing proved to be both quick and inexpensive to undertake as it did not require any specialist software. As with other studies, readability testing allowed the identification of content problems not found through usability testing alone [43]. The different readability tools produced different readability scores, which is reflective of the different criteria assessed and different formulae used to calculate the scores. Although the readability levels were found to be relatively high, it is worth noting that the information is also available on the website as an audio version in all three languages. This makes the information more accessible to people

A further Readability Analysis using VocabProfile Compleat (VPC) (http://www.lextutor.ca/ vp/comp/) [47] was conducted as part of our site evaluation, to gain a better understand‐

**2.** Gunning Fog Index readability formula (http://gunning-fog-index.com/)

**6.** Italian Readability Analyzer (http://labs.translated.net/text-readability/).

SMOG-calculator/smogcalc.php?redirectedfrom=niace)

structures or word use in medication management resources.

Almost anyone can create a website and author content that can be accessed by the general public. It has been estimated that 30 million new websites are created every day, with the vast majority relating to health conditions [50]. As the number of people with easy access to the internet continues to rise, it is important that the information that they access be reliable, particularly in relation to health conditions.

A number of organisations have developed codes of ethics and evaluation instruments to determine the reliability and creditability of information on the internet. A third-party approach is often used to evaluate online health information, so that users can determine the quality of information, through displaying either a label, seal or logo on their website [51]. A number of such accreditation systems are available for medical and health websites. Credibil‐ ity, relevance and accuracy are especially important for health websites where information can have a direct impact on the health and well-being of the site user.

Methods: Three website rating tools were used to assess the Managing Medicines for People with Dementia website for accessibility, quality and reliability.


The Suitability Assessment of Materials (SAM) instrument contains six sections: content, literacy demand, graphics, layout and type, learning stimulation and motivation and cultural appropriateness. The SAM rates factors affecting the difficulty of readability and the compre‐ hension relative to understanding the meaning. For each factor, the materials were categorised and rated as not suitable (0), adequate (1) and superior (2) based on objective criteria.

The Health-Related Web Site Evaluation Form consists of 36 questions, covering: content, accuracy, author, currency, audience, navigation, external links and structure. The rating scale criteria were defined as poor (<75%), adequate (75–89%) and excellent (>90%).

The Health on the Net Foundation (HONcode) is a self-regulating non-government organisa‐ tion with a set of principles adopted to standardise the creditability and reliability of health and medical information on the internet [51]. It was developed to provide a code of ethics for medical and health-related information on the internet, for three target audiences, the general public, health professionals and web publishers. It is suggested that if health information websites adhere to the HONcode, it will go a long way to ensuring that the health information provided to the public is both of good quality and reliable. The HONcode certification has eight principles: authority, complementary, confidentiality, attribution, justification, trans‐ parency, financial disclosure and advertising, which are assessed through 12 items.

#### *4.5.1. Method critique*

All of the instruments were found to be quick and easy to administer, and showed that the website provided reliable information and allowed the identification of some minor problems in the website design.

#### **4.6. Future recommendations**

It is important to plan to undertake assessment for usability, readability and accessibility levels when designing a website. Often this is an iterative, multimodal process. The Phase 2 evalu‐ ation highlighted the importance of involving end-users in the design of a health-information website. It showed that a combination of qualitative and quantitative methods can be effec‐ tively utilised to identify design and content problems. Further, the methodologies used were often simple to undertake and relatively cheap to conduct.

The results of the evaluation process will be used to inform the final design changes to the website. The site will subsequently be made available to national bodies within Australia, such as Alzheimer's Australia, for promotion. Further funding will be sought in order to translate information into other community languages.
