**4.1 Determining clinical benefits**

The recognised approach to determine the effectiveness of any intervention is the randomised controlled trial (RCT), in which patients are randomly placed into groups that are designed to have equal statistics other than the intervention under study. This typically involves ensuring that each group has equal statistics for factors such as age, sex, and stage of disease. The best approach is to ensure that the type of intervention being administered is hidden from both the patient and the doctor, in the "double-blind RCT", with the patients receiving either the intervention or placebo in order to ensure they are unaware of the intervention and exclude other factors that might influence the outcome. This approach is often referred as the "gold standard" to determine outcome (see [18]) for details on design of research studies).

However evaluating technology presents difficulties as it is not easy to "hide" the use of technology from patients to implement a double-blind RCT. Alternative approaches are adopted. This usually takes one of two forms: comparing the results from large groups with and without the technology; or comparing the results of each patient with their own data before and after the introduction of the technology, sometimes referred as a "crossover" study. In the first approach, care must be taken to ensure sufficiently large groups for comparison so that they have otherwise matched statistics. The second approach requires care that results are not affected by the deterioration in the condition of the patient over the period of the research, and that seasonal effects on a disease such as flu in the autumn and any other influences of time on a disease are avoided.

The choice of which clinical benefits are assessed also presents difficulties as telehealth is not a treatment, therefore there is no direct outcome that can be measured; rather telehealth impacts diagnosis, decision, and management, and therefore impacts secondary outcomes. In addition, the design of the services that respond to the information collected from telehealth are as important as the design of the telehealth technology; and the intervention must be designed as a complete service, with consideration of the clinical services involved and the communication and relationship between the separate key stakeholders. This important aspect of design of a study is rarely discussed in reports, which not only prevents full comparison of method to be made, but does not allow full advantage of the experience to be gained.

The ultimate analysis of the outcome of assessment of clinical benefit is the systematic review and meta-analysis of outcomes, as these permit rigorous statistical evaluation of benefits of using telehealth over usual care from aggregation of the results from multiple studies. The systematic review has a specific methodology that defines how terms are used to identify trials and how the trials are selected for inclusion and exclusion based on factors such as the number in the trial, duration of the trial, and whether RCT [23]. A systematic review will normally use at least two

reviewers to undertake the selection of papers, with an arbiter to make a final decision on selection. Outcomes from the selected trials may then be summarized and are frequently presented as a funnel plot (also referred as Forest plot) that shows the risk ratio of each study included in the analysis and overall outcome.

#### **4.2 Assessing clinical outcome**

The most common approach that is used to assess clinical outcome is through the use of a validated disease specific questionnaire. This assesses the perceived state of health of a patient and the perceived level of symptoms and their impact on everyday activities. Many such questionnaires exist for diseases such as CHF [24], COPD [25], and asthma [26].

The second approach is to use a questionnaire such as the EQ5D [27] or SF36 [28] to assess the perceived general state of health of a patient. Whilst use of these questionnaires does not assess clinical outcome directly, it can determine the perceived benefits the patient may feel from being monitored.

Direct assessment of clinical outcome is determined through disease specific measures such as ejection fraction in CHF, lung function in COPD, and glucose and HbA1c in diabetes. Secondary measures may be considered such as blood pressure in diabetes. Some projects assess secondary outcomes that include death rate, and extension of life.

#### **4.3 Clinical benefits in patients with CHF**

CHF is a common chronic condition with a prevalence of about 13% amongst those aged 85 years or above in countries such as the UK [29]. Telehealth in CHF has been found to improve health outcome for patients and reduce the number of hospitalisations [30, 31]. Many studies have considered nurse telephone support and remote patient monitoring as equivalent and either report combined outcomes or present results synonymously [32]. Telephone support typically comprises patient follow up, education and counselling delivered via a telephone call made by a specialist nurse. However, patients receiving this form of intervention usually have only mild to moderate CHF symptoms (NYHA class I-II). Remote patient monitoring involves home care of patients using specialist telecare devices to send vital signs directly to the clinician and is often done for patients with severe symptoms of heart failure (NYHA class III-IV) [33].

In reviews that differentiate these approaches and determine the effectiveness of each separately, a technology based approach to patient home monitoring (telemonitoring) is shown to be more effective [34]. In other systematic reviews, telehealth has been compared to a number of alternative approaches, including patient education, specialist (clinician or cardiologist) follow up, nurse home visiting, and telephone support [35–38]. The general conclusion is that telemonitoring alone is insufficient to reduce readmission rates and improve quality of life, and must be integrated with nurse visits and specialist management and follow up, with redesign of the service to support the intervention.

#### *4.3.1 Mortality*

Mortality has been the most reported clinical outcome in the study of impact of telehealth to manage CHF. In [39] ten studies reported mortality as a primary outcome, with five studies reporting outcomes with significant significance. However, although the pooled estimate results showed an overall reduction in all-cause mortality, this was not statistically significant (0.77, 0.61 to 0.98, P = 0.02). Funnel
