**Health Technology Assessment: An Essential Approach to Guide Clinical Governance Choices on Risk Management**

Giovanni Improta1, Antonio Fratini2 and Maria Triassi1 *1Dept. of Preventive Medical Sciences, University of Naples "Federico II", Naples, 2Dept. of Biomedical, Electronic and Telecommunication Engineering University of Naples "Federico II", Naples, Italy* 

#### **1. Introduction**

66 Risk Management for the Future – Theory and Cases

OHSAS 18002:2008 (2008). Occupational health and safety management systems. Guidelines for the implementation of OHSAS 18001:2007. ISBN: 978 0 580 61674 7, p.88. Reniers, G.L.L., Dullaert, W., Ale, B.J.M., Soudan, K. (2005a). Developing an external domino

Reniers, G.L.L., Dullaert, W., Ale, B.J.M., Soudan, K. (2005b). The use of current risk analysis

Salvi, Ol. and Gaston, D. (2004). Risk assessment and risk decision-making process related to

van Duijne, Fr. H., van Aken, D. and Schouten, E.G. (2008). Considerations in developing

Vesely, W.E., Goldberg, F.F., Roberts, N.H., Haasl, D.F. (1981). Fault Tree Handbook. US

Yuhua, D., and Datao, Y. (2005). Estimation of failure probability of oil and gas transmission

Zheng X., Liu M. (2009). An overview of accident forecasting methodologies. Journal of Loss

Prevention in the Process Industries, 18, 119-126.

Suddle, Sh. (2009). The weighted risk analysis. Safety Science, 47, 668-679.

Nuclear Regulatory Commission, Washington, DC.

Prevention in the Process Industries, 22, 4, 484-491.

Industries, 18, 127-138.

Industries, 18, 83–88.

254.

10.1080/1366987042000192192.

prevention framework: Hazwim. Journal of Loss Prevention in the Process

tools evaluated towards preventing external domino accidents. Journal of Loss

hazardous installation in France. Journal of Risk Research, 7: 6, 599-608, DOI:

complete and quantified methods for risk assessment. Safety Science, 46(2), 245-

pipelines by fuzzy fault tree analysis. Journal of Loss Prevention in the Process

Risk management in this chapter is defined as the process of identifying, through the study of all possible sources of errors and problems, the required preventive and corrective actions to reduce risk and whose consequences that compromises the capacity of an organization to reach its own objectives (Del Vecchio and Cosmi, 2003). It has been extensively used in economics, engineering and recently has also been adopted in the fields of public and private health (Gorrod, 2004; Alexander and Sheedy, 2005).

Safety, however, is becoming an imposed target for any health care system so that recent provisions stimulate the application of risk management methodologies (i.e. health risk management) also in clinical environments (Sanfilippo, 2001; Carroll, 2009).

Health Risk Management (HRM) aims to improve the quality of health care, ensure safety and security for patients and sanitary operators encompassing the comprehension of risks associated to the introduction or use of a technology in a clinical environment.

New technologies are not simple objects/products, but like a social practice built within actions and relationships, they are strictly connected to the business setup and act as a basic part of the organization design especially in health care industry.

The knowledge of the risks associated with different technologies (risk assessment) is of extreme importance in the definition of programs and initiatives, at various levels of health care governance (public health authorities, regions government, Ministry of Health), to reduce the incidence of errors and failures.

Healthcare distinguishes itself from other industries in that patient's safety represents a quality dimension of greatest importance.

However, healthcare systems are affected by risks of different nature: risks associated to the personnel professionalism or to the environment appropriateness, risks related to specific equipment use (e.g. magnetic resonance or X-ray), risks related to therapeutic or diagnostic

Health Technology Assessment:

consequences.

technologies.

risks.

organization is aimed to.

The rest of the chapter is organized as follows:

origins and some fundamental concepts;

**2. Origins of health technology assessment** 

2000; Jorgensen and Hvenegaard, 2000).

Battista and Hodge, 1989)

Battista and Hodge, 1999).

HTA approaches with the hierarchic clustering technique;

and its preliminary application to medical technologies; Section 5 (Discussion) reports practical observations and remarks;

An Essential Approach to Guide Clinical Governance Choices on Risk Management 69

HTA process may be carried out at different technologies lifecycle phases (see Figure 1) to obtain a "dynamic" overview of their potential, usage and intended or unintended

Since HRM may take advantage from a complete technology impact overview that includes a comprehensive identification of its associated risks, it becomes important to discover HTA methodologies and their application for the evaluation of medical

The authors aimed to discuss HTA systematic approach and its advantages in assuring correct risk estimation and global patient safety, which is one of the objectives each health

This chapter presents an example on a possible design and implementation of a HTA protocol for the classification of hospitals or health facilities equipment, realized by combining the classic HTA concepts with hierarchic clustering techniques in a multidisciplinary analysis of requirements, cost, impact of logistics, technology associated

The next section (Section 2) presents a brief review on health technology assessment

Section 3 – Actual Research – describes the methods followed for the combination of

Section 4 (Case study) presents the development of the specific classification protocol

During past decades, health care systems of industrialized countries have focussed on the problem of assuring health services to all citizens while reducing the allocation of economic resources (Fleurette and Banta, 2000; France, 2000; Granados *et al*, 2000; Perlett and Busse,

To achieve both the subsistence of the essential health services and the reduction of sanitary costs, almost every state engaged in policies aimed at rationalizing the use of resources by acting on the efficiency of organizations in strengthening service delivery as well as introducing elements of competition between producers or prioritization of health care services to ensure to citizens through public funding. (Sackett, 1980; Banta, 1993, 2000;

The need to evaluate the effectiveness of different diagnostic and therapeutic protocols and technologies compared to the suffering population and, at the same time, the need to a complete knowledge of the service delivery costs originated a multi-disciplinary research area called "Health Technology Assessment" (Blades, 1986; Birch and Donaldson, 1987;

Section 6 (Future work) describes some future and desirable model developments;

pathways but also risks related to the social, ethical or economic impact to the use of technologies and methodologies in healthcare facilities. Each of these aspects have their own specific aspects of investigation making HRM a complex process.

Healthcare is moving towards increased assistance needs with limited resources, both in economics terms, in personnel or space terms, leading to the usage of specific analyses for the acquisition, evaluation and assessment of medical technologies.

The systematic evaluation of properties, effects or other impacts of a medical (or health) technology with a broad multidisciplinary approach is named Health Technology Assessment (HTA).

The main purpose of HTA is to assist policymaking for technology in health care to achieve the most advantageous resource allocation, evaluating the efficacy and the efficiency of the diagnostic and therapeutic pathways as well as related risks and organizational models.

HTA consists in identifying an analytical methodology that allows the optimization of the product adoption/evaluation process, through a careful study of the effective needs of the users, of the available alternative technologies and the relative operational implications on the setup. This type of evaluation requires an interdisciplinary approach of "policy analysis", studying the aspects of safety, cost, benefits, effectiveness, and include critical evaluations of the actual measures and improving the quality of life.1 HTA methodology implies to recognize the actual healthcare needs and evaluate how technologies may answer to those needs while considering the overall implication of their use including the associated risks. It may address the direct and intended consequences of technologies as well as their indirect and unintended consequences.

HTA practices have become widespread and are increasingly present in health systems, so that more and more healthcare facilities monitor the global impact of their medical technologies.

Fig. 1. A link among Technology Lifecycle, Health Technology Assessment and Health Risk Management (derived from Derrico *et al*, 1999 )

<sup>1</sup>Canadian Coordinating Office for Heath Technology Assessment 1995

HTA process may be carried out at different technologies lifecycle phases (see Figure 1) to obtain a "dynamic" overview of their potential, usage and intended or unintended consequences.

Since HRM may take advantage from a complete technology impact overview that includes a comprehensive identification of its associated risks, it becomes important to discover HTA methodologies and their application for the evaluation of medical technologies.

The authors aimed to discuss HTA systematic approach and its advantages in assuring correct risk estimation and global patient safety, which is one of the objectives each health organization is aimed to.

This chapter presents an example on a possible design and implementation of a HTA protocol for the classification of hospitals or health facilities equipment, realized by combining the classic HTA concepts with hierarchic clustering techniques in a multidisciplinary analysis of requirements, cost, impact of logistics, technology associated risks.

The rest of the chapter is organized as follows:

68 Risk Management for the Future – Theory and Cases

pathways but also risks related to the social, ethical or economic impact to the use of technologies and methodologies in healthcare facilities. Each of these aspects have their own

Healthcare is moving towards increased assistance needs with limited resources, both in economics terms, in personnel or space terms, leading to the usage of specific analyses for

The systematic evaluation of properties, effects or other impacts of a medical (or health) technology with a broad multidisciplinary approach is named Health Technology

The main purpose of HTA is to assist policymaking for technology in health care to achieve the most advantageous resource allocation, evaluating the efficacy and the efficiency of the diagnostic and therapeutic pathways as well as related risks and organizational models.

HTA consists in identifying an analytical methodology that allows the optimization of the product adoption/evaluation process, through a careful study of the effective needs of the users, of the available alternative technologies and the relative operational implications on the setup. This type of evaluation requires an interdisciplinary approach of "policy analysis", studying the aspects of safety, cost, benefits, effectiveness, and include critical evaluations of the actual measures and improving the quality of life.1 HTA methodology implies to recognize the actual healthcare needs and evaluate how technologies may answer to those needs while considering the overall implication of their use including the associated risks. It may address the direct and intended consequences of technologies as well as their indirect

HTA practices have become widespread and are increasingly present in health systems, so that more and more healthcare facilities monitor the global impact of their medical technologies.

Fig. 1. A link among Technology Lifecycle, Health Technology Assessment and Health Risk

specific aspects of investigation making HRM a complex process.

the acquisition, evaluation and assessment of medical technologies.

Assessment (HTA).

and unintended consequences.

Management (derived from Derrico *et al*, 1999 )

1Canadian Coordinating Office for Heath Technology Assessment 1995


#### **2. Origins of health technology assessment**

During past decades, health care systems of industrialized countries have focussed on the problem of assuring health services to all citizens while reducing the allocation of economic resources (Fleurette and Banta, 2000; France, 2000; Granados *et al*, 2000; Perlett and Busse, 2000; Jorgensen and Hvenegaard, 2000).

To achieve both the subsistence of the essential health services and the reduction of sanitary costs, almost every state engaged in policies aimed at rationalizing the use of resources by acting on the efficiency of organizations in strengthening service delivery as well as introducing elements of competition between producers or prioritization of health care services to ensure to citizens through public funding. (Sackett, 1980; Banta, 1993, 2000; Battista and Hodge, 1989)

The need to evaluate the effectiveness of different diagnostic and therapeutic protocols and technologies compared to the suffering population and, at the same time, the need to a complete knowledge of the service delivery costs originated a multi-disciplinary research area called "Health Technology Assessment" (Blades, 1986; Birch and Donaldson, 1987; Battista and Hodge, 1999).

Health Technology Assessment:

processes.

adopted.


technology;

implementation of the decision maker.

and to refine HTA approaches and methodologies.

assessment, in some main steps that include:

 the problem addressed by the technology; real clinical needs (needs assessment);

overview of the assessment results;

distribution of results and subsequent advice and recommendations.

the *identification* of the assessment object/topic in order to clarify:

the *evaluation* of the technology that for HTAs has to include:

An Essential Approach to Guide Clinical Governance Choices on Risk Management 71

During the last decades then, national and international societies contributes to consolidate

HTA has been recognized as a process that involves a number of different actions rather than a single research. These may include identification of technologies that require an assessment, collection and data analysis, the synthesis of existing information and the

The fundamental steps of an HTA process can be summarized, as well as in a technology

requirements or constraints the technology under investigation has to fit.

clinical outcomes, costs of the entire life cycle of technology; the analysis of all the collected data and the technology rating;

 the collection of key data in terms of general impact: technical, clinical, social, ethical as well as economical; it involves the comparison of different technologies according to criteria of quality, evaluating the clinical efficacy (benefits), safety,

This process, and its multidisciplinary evaluations, characterize (or it should do) all HTA

the consolidation and synthesis of all the analysis in order to give a synthetic

the production of recommendation on the applicability and adoption of the

Finally, the *dissemination* of information phase comes from the evaluation and

However, as in every dynamic process, *monitoring the effectiveness* of the assessment conclusion helps in refining methodologies and in assuring the correctness of the decision

Crucial in the entire HTA process is the identification of professionals selected for the study. HTA analyses uses different methods, depending on the purpose of the assessment; some of them used economic approach: cost-analyses (Drummond *et al.*, 1987), some a managerial approach SWOT (Strengths, Weaknesses, Opportunities and Threats) analyses, logical framework (Armstrong, 1982; Kahveci *et al.*, 2006) and some other a more pluralistic approach: Multiple Attribute Decision Making (MADM) analyses, hierarchical clustering (Agrawal *et al*., 1991). Each of these methods have their own particular or partial judgment aspects. In the past years scientific community stimulated the debate on other possible approaches and tool in HTA process and hierarchic approach as used in decision support theory have been recently proposed. One of the most interesting and used hierarchic model is the Analytic Hierarchy Process (AHP) that began with (Saaty 1980) and is often referred

Early studies of technology evaluation were performed in the mid-1960s: the Committee on Science and Astronautics of the House of Representatives of the United States Congress, showed the need for new methods of analysis that can clarify the *economic and social impact due to the introduction and development of new technologies*.

In 1972, United States Congress creates the Office of Technology Assessment (OTA) with a pubic law (92-484) with the task of developing and deploying technology assessment, authoritative analysis of complex scientific and technical issues, and demonstrates its usefulness to politicians.

Technical information needed by policymakers is frequently not available, or not in the right form. A policymaker cannot judge the merits or consequences of a technological program within a strictly technical context. He has to consider social, economic, and legal implications of any course of action*2*.

In Europe the importance of the assessment was accepted about a decade later than in the U.S., when the World Health Organization (WHO), within the program "Health for Hall"; in a first phase, the response by governments has focussed on introducing policies aimed at controlling the spread of technologies in logic of cost containment.

This phase did not produce significant results in terms of technology assessment, it however allowed the introduction of methods of economic evaluations, and in particular the concept of cost-effectiveness in health care (Weisbrod, 1961; Cochrane, 1972; Bush *et al*, 1973; Sorkin, 1975; Drummond, 1994).

The change in technology and resource allocation assessment was a great enhancement for the development of another key asset: the Evidence-Based Medicine, whose objective is to find a relationship between the empirical evidence and current clinical practice in order to improve the quality and effectiveness of care at the level of individual patients (Sackett *et al.*, 1996).

2Emilio Daddario, U.S. Congress, House of Representatives 1967

During the last decades then, national and international societies contributes to consolidate and to refine HTA approaches and methodologies.

HTA has been recognized as a process that involves a number of different actions rather than a single research. These may include identification of technologies that require an assessment, collection and data analysis, the synthesis of existing information and the distribution of results and subsequent advice and recommendations.

The fundamental steps of an HTA process can be summarized, as well as in a technology assessment, in some main steps that include:

	- the problem addressed by the technology;
	- real clinical needs (needs assessment);
	- requirements or constraints the technology under investigation has to fit.
	- the collection of key data in terms of general impact: technical, clinical, social, ethical as well as economical; it involves the comparison of different technologies according to criteria of quality, evaluating the clinical efficacy (benefits), safety, clinical outcomes, costs of the entire life cycle of technology;
	- the analysis of all the collected data and the technology rating;

This process, and its multidisciplinary evaluations, characterize (or it should do) all HTA processes.


70 Risk Management for the Future – Theory and Cases

Early studies of technology evaluation were performed in the mid-1960s: the Committee on Science and Astronautics of the House of Representatives of the United States Congress, showed the need for new methods of analysis that can clarify the *economic and social impact* 

In 1972, United States Congress creates the Office of Technology Assessment (OTA) with a pubic law (92-484) with the task of developing and deploying technology assessment, authoritative analysis of complex scientific and technical issues, and demonstrates its

Technical information needed by policymakers is frequently not available, or not in the right form. A policymaker cannot judge the merits or consequences of a technological program within a strictly technical context. He has to consider social, economic, and legal

In Europe the importance of the assessment was accepted about a decade later than in the U.S., when the World Health Organization (WHO), within the program "Health for Hall"; in a first phase, the response by governments has focussed on introducing policies aimed at

This phase did not produce significant results in terms of technology assessment, it however allowed the introduction of methods of economic evaluations, and in particular the concept of cost-effectiveness in health care (Weisbrod, 1961; Cochrane, 1972; Bush *et al*, 1973; Sorkin,

The change in technology and resource allocation assessment was a great enhancement for the development of another key asset: the Evidence-Based Medicine, whose objective is to find a relationship between the empirical evidence and current clinical practice in order to improve the quality and effectiveness of care at the level of individual patients (Sackett *et al.*, 1996).

*due to the introduction and development of new technologies*.

controlling the spread of technologies in logic of cost containment.

Fig. 2. HTA process steps (derived from Panelius *et al.* 1988, FinOHTA 1997)

2Emilio Daddario, U.S. Congress, House of Representatives 1967

usefulness to politicians.

1975; Drummond, 1994).

implications of any course of action*2*.


Finally, the *dissemination* of information phase comes from the evaluation and implementation of the decision maker.

However, as in every dynamic process, *monitoring the effectiveness* of the assessment conclusion helps in refining methodologies and in assuring the correctness of the decision adopted.

Crucial in the entire HTA process is the identification of professionals selected for the study.

HTA analyses uses different methods, depending on the purpose of the assessment; some of them used economic approach: cost-analyses (Drummond *et al.*, 1987), some a managerial approach SWOT (Strengths, Weaknesses, Opportunities and Threats) analyses, logical framework (Armstrong, 1982; Kahveci *et al.*, 2006) and some other a more pluralistic approach: Multiple Attribute Decision Making (MADM) analyses, hierarchical clustering (Agrawal *et al*., 1991). Each of these methods have their own particular or partial judgment aspects. In the past years scientific community stimulated the debate on other possible approaches and tool in HTA process and hierarchic approach as used in decision support theory have been recently proposed. One of the most interesting and used hierarchic model is the Analytic Hierarchy Process (AHP) that began with (Saaty 1980) and is often referred

Health Technology Assessment:

**3.1 Adopted methodology** 

paragraphs.

decision.

problem itself.

investigation.

may be expressed as:

An Essential Approach to Guide Clinical Governance Choices on Risk Management 73

In order to achieve the desired classification, HTA and clustering technique methodologies have been combined; the advantages obtained with the combination proposed and the combination methods are completely discussed and clarified in the next

The protocol has been structured following a hierarchical assessment approach, similar to AHP (Saaty, 1980, Saaty, 1982, 1990), based on the definition of the goal to achieve, the criteria and evaluation parameters and their relative and global incidence in the overall

A hierarchical breakdown of the problem in N different criteria (or cluster), which groups properties and attributes of alternatives, helps in a better evaluation of the

For each cluster are then recognized properties or attributes (or cluster elements) in a variable number. It is worth mentioning that, in health environments, these properties and attributes are not always directly and objectively measurable (i.e. revenues *versus* degree of patient technology acceptance) and, in order to obtain a comprehensive and concise

The non-objective measurable parameters/attributes can also be quantified and then made comparable by using expert opinions expressed in linguistic variables and converted into numerical values (usually using the ordinal scale used in AHP and a pairwise comparison procedure with the aim of producing a square matrix, whose element aij indicates the

Synthetic assessment of the degree of importance of the single Aj with respect to the others (weights) are calculated by normalizing the global importance of individual factors, i.e. the sum of each element of a row, with respect to the sum; it keeps unchanged the relationships between the factors and makes the sum of all weight obtained equal to 1, which is

Assuming gather experts evaluations so that aik = aij\*ajk (i. e. assuming to know n-1 matrix elements and obtaining the remaining matrix elements from the properties of consistency and reciprocity) is not necessary to evaluate the AHP technique Consistency Index (CI) as it

Weights obtained are aggregated together with the hierarchical Saaty's composition

In our case, the final equipment classification is obtained by *scoring* the equipment based on the evaluated importance of the criteria and their properties which helps in correctly combining the specific characteristics/condition of the equipment under

Based on this principle, the overall score of the generic alternative A with respect to the goal

assessment reducing *subjective* bias, these are aggregated together in clusters.

relative importance of the element with respect to criteria AJ).

has hypothesised a perfect consistency in judgments (CI = 0).

principle, which allows a priority listing of alternatives to the goal.

mathematically convenient in weighted sums.

as his method. In general, hierarchic processes are interesting methodologies since their ability to grade choices in meeting conflicting objectives and its wide variety of decision problems applications.

However, HTA is often adopted in process of technology acquisition and hierarchic approach has been proposed in some cases (Sloane *et al.,* 2003); in our opinion however this approach can also be conveniently used as HTA in case of evaluation on the state of technologies in an hospital or in general any health facilities.

### **3. Actual research**

Every health facility (HF) manages a great number of medical technologies; however, in their "hospital" life cycle, technologies will be replaced by recent ones, especially those with lower efficiency and possible risks related to their age and use.

In general, HF managers demand regular surveys regarding status of the technologies in the same structures; hence, HTA procedure should also be applied to assess the state of capital equipment in order to facilitate further management decisions on technologies investments or to consider equipment immediate disposal, planned disposal or relocation (assigning it to other departments).

The authors intended to present an example of assessment protocol, realized by combining hierarchical clustering techniques with HTA multi-criteria approach, aimed to help clinical engineers and clinicians to concur in manager decisions on the capital equipment of a hospital or a health facility, in particular, the solution proposed was intended to allow an easy equipment ranking in terms of disposal/relocation planning.

The goal of the assessment was to highlight the *state* of single device/technology, classifying it in three main categories:

	- *Relocation*: the equipment could respond more effectively to the need of other departments within HF; it complies to technical standard and it is still properly functioning.
	- *Disposal:* the equipment does not respond effectively to the needs of the HF, it could be however conveniently employed by other institutions or health structures; the alternatives in this case are:
		- *Trading:* new technologies vendor discounts or equipment selling.
		- *Donation:* the equipment can be used in other emerging countries, in emergency environments.

In order to achieve the desired classification, HTA and clustering technique methodologies have been combined; the advantages obtained with the combination proposed and the combination methods are completely discussed and clarified in the next paragraphs.

#### **3.1 Adopted methodology**

72 Risk Management for the Future – Theory and Cases

as his method. In general, hierarchic processes are interesting methodologies since their ability to grade choices in meeting conflicting objectives and its wide variety of decision

However, HTA is often adopted in process of technology acquisition and hierarchic approach has been proposed in some cases (Sloane *et al.,* 2003); in our opinion however this approach can also be conveniently used as HTA in case of evaluation on the state of

Every health facility (HF) manages a great number of medical technologies; however, in their "hospital" life cycle, technologies will be replaced by recent ones, especially those with

In general, HF managers demand regular surveys regarding status of the technologies in the same structures; hence, HTA procedure should also be applied to assess the state of capital equipment in order to facilitate further management decisions on technologies investments or to consider equipment immediate disposal, planned disposal or relocation (assigning it to

The authors intended to present an example of assessment protocol, realized by combining hierarchical clustering techniques with HTA multi-criteria approach, aimed to help clinical engineers and clinicians to concur in manager decisions on the capital equipment of a hospital or a health facility, in particular, the solution proposed was intended to allow an

The goal of the assessment was to highlight the *state* of single device/technology, classifying



 *Relocation*: the equipment could respond more effectively to the need of other departments within HF; it complies to technical standard and it is still properly

 *Disposal:* the equipment does not respond effectively to the needs of the HF, it could be however conveniently employed by other institutions or health structures;

*Donation:* the equipment can be used in other emerging countries, in

*Trading:* new technologies vendor discounts or equipment selling.


technologies in an hospital or in general any health facilities.

lower efficiency and possible risks related to their age and use.

easy equipment ranking in terms of disposal/relocation planning.

following a corrective technical maintenance;

this case the possible alternatives are:

the alternatives in this case are:

emergency environments.

problems applications.

**3. Actual research** 

other departments).

it in three main categories:

functioning.

efficiently.

The protocol has been structured following a hierarchical assessment approach, similar to AHP (Saaty, 1980, Saaty, 1982, 1990), based on the definition of the goal to achieve, the criteria and evaluation parameters and their relative and global incidence in the overall decision.

A hierarchical breakdown of the problem in N different criteria (or cluster), which groups properties and attributes of alternatives, helps in a better evaluation of the problem itself.

For each cluster are then recognized properties or attributes (or cluster elements) in a variable number. It is worth mentioning that, in health environments, these properties and attributes are not always directly and objectively measurable (i.e. revenues *versus* degree of patient technology acceptance) and, in order to obtain a comprehensive and concise assessment reducing *subjective* bias, these are aggregated together in clusters.

The non-objective measurable parameters/attributes can also be quantified and then made comparable by using expert opinions expressed in linguistic variables and converted into numerical values (usually using the ordinal scale used in AHP and a pairwise comparison procedure with the aim of producing a square matrix, whose element aij indicates the relative importance of the element with respect to criteria AJ).

Synthetic assessment of the degree of importance of the single Aj with respect to the others (weights) are calculated by normalizing the global importance of individual factors, i.e. the sum of each element of a row, with respect to the sum; it keeps unchanged the relationships between the factors and makes the sum of all weight obtained equal to 1, which is mathematically convenient in weighted sums.

Assuming gather experts evaluations so that aik = aij\*ajk (i. e. assuming to know n-1 matrix elements and obtaining the remaining matrix elements from the properties of consistency and reciprocity) is not necessary to evaluate the AHP technique Consistency Index (CI) as it has hypothesised a perfect consistency in judgments (CI = 0).

Weights obtained are aggregated together with the hierarchical Saaty's composition principle, which allows a priority listing of alternatives to the goal.

In our case, the final equipment classification is obtained by *scoring* the equipment based on the evaluated importance of the criteria and their properties which helps in correctly combining the specific characteristics/condition of the equipment under investigation.

Based on this principle, the overall score of the generic alternative A with respect to the goal may be expressed as:

Health Technology Assessment:

definition of the team members.

**4.1 HTA team** 

manager.

attribute.

constituents.

selected variables.

An Essential Approach to Guide Clinical Governance Choices on Risk Management 75

The centre is specialized in advanced research on malignant hyperthermia, liver cells

The centre has its own imaging department that include different technologies, one of which was evaluated with the developed model; this case represented an ideal opportunity to test

As in each HTA process (see Figure 2), after a first identification of the assessment object (ultrasound device), a panel of experts (*HTA team*) has to be defined to proceed in concretizing the evaluation strategy and to reveal the relative weight of the different evaluation fields as described above. Still, the process of team choice is of extreme importance for the entire assessment and particular attention have been paid to the

Different professional were involved in the HTA protocol development; in our opinion HTA team may benefit from including professionals of structures that are frequently involved in the process of acquisition or evaluation and use of technologies. The team was then composed of five professionals: the hospital general manager, the sanitary manager, a clinical engineer, the superintendent and the risk protection and prevention

The choice of including individual used to do assessment processes facilitates the

Additionally, since periodic controls of hospital technology are mandatory, clinical engineering services hold information related to every equipment existing in the hospital and the collected data is often the result of monitoring procedure and checklist filling, by scoring3 each voice of these checklist it will immediately score the related cluster

identified, the application of the described methodology for the development of the

The multidimensional analysis, typical of a health technology assessment process as mentioned above, determined the identification of five clusters related to key areas of the assessment: S-Safety, L-Legal, O-Organizational, E-Economic, T-technological their

The protocol was intended for the classification of a wide variety of equipment hence the definition of elements and thresholds was derived from previous experience of different professionals, as discussed in the next paragraphs. The figure below reports some of the

mutations, clinical trials on animals, it is also a biomedical data repository.

the acceptance of method by centre employees and the model results.

identification and scoring of criteria and their constituents.

Once the team has been formed and the information sources

3The process of assignment of a value to a monitoring checklist field

assessment protocol took place as reported below.

**4.2 Protocol development** 

$$C = \frac{\sum\_{i=1}^{k} P\_i \cdot V\_i}{\sum\_{i=1}^{k} P\_i} \tag{1}$$

where:

k is the cluster numbers

Pi is the weight of cluster

Vi is the total score of equipment with respect to i-cluster

C is the total score of equipment

$$\begin{aligned} V\_j &= \frac{\sum\_{j=1}^n p\_j \cdot \mathbf{v}\_j}{\sum\_{j=1}^n p\_j} \\ &\quad \sum\_{j=1}^n p\_j \end{aligned} \tag{2}$$

where:

n is the element numbers

pj is the weight of element j with respect to cluster

vj is the score of element j

Vj is the total score of equipment for that criteria

Finally, the process ends with a classification of the equipment based on its specific *score;* in particular, since we hypothesized four different alternatives, classification is achieved choosing three different thresholds and comparing the obtained equipment score with those values. In case of partial evaluation (that is the evaluation based only on some cluster) the sum will include only the aspect under investigation.

Every assessment strategy however, although theoretically correct, has to be tested in real environment to discover its acceptability and practical applicability; thus, the described methodology has then been examined in an HTA case study of an hospital.

#### **4. Case study**

This case study describes the application and refinement of the adopted methodology in a hospital HTA focussed on technology evaluation of hospital equipment: our case consider a preliminary application on an ultrasound device.

The hospital has around 900 beds and serves a population of more than a million people, it is a national centre of excellence and has agreement with different emerging countries for the exchange of personnel and technology.

Among its structures, the hospital includes a biotechnology research centre born in 2000 with a consistent acquisition of specialized personnel already present in a research division founded in 1986 in the same hospital.

The centre is specialized in advanced research on malignant hyperthermia, liver cells mutations, clinical trials on animals, it is also a biomedical data repository.

The centre has its own imaging department that include different technologies, one of which was evaluated with the developed model; this case represented an ideal opportunity to test the acceptance of method by centre employees and the model results.

As in each HTA process (see Figure 2), after a first identification of the assessment object (ultrasound device), a panel of experts (*HTA team*) has to be defined to proceed in concretizing the evaluation strategy and to reveal the relative weight of the different evaluation fields as described above. Still, the process of team choice is of extreme importance for the entire assessment and particular attention have been paid to the definition of the team members.

#### **4.1 HTA team**

74 Risk Management for the Future – Theory and Cases

*i*1

*k*

*k* 

*Pi Vi*

*Pi i*1

*pj vj*

*pj j*1

(1)

(2)

*C*

*Vj*

*j*1

*n*

Finally, the process ends with a classification of the equipment based on its specific *score;* in particular, since we hypothesized four different alternatives, classification is achieved choosing three different thresholds and comparing the obtained equipment score with those values. In case of partial evaluation (that is the evaluation based only on some cluster) the

Every assessment strategy however, although theoretically correct, has to be tested in real environment to discover its acceptability and practical applicability; thus, the described

This case study describes the application and refinement of the adopted methodology in a hospital HTA focussed on technology evaluation of hospital equipment: our case consider a

The hospital has around 900 beds and serves a population of more than a million people, it is a national centre of excellence and has agreement with different emerging countries for

Among its structures, the hospital includes a biotechnology research centre born in 2000 with a consistent acquisition of specialized personnel already present in a research division

methodology has then been examined in an HTA case study of an hospital.

*n* 

Vi is the total score of equipment with respect to i-cluster

pj is the weight of element j with respect to cluster

Vj is the total score of equipment for that criteria

sum will include only the aspect under investigation.

preliminary application on an ultrasound device.

the exchange of personnel and technology.

founded in 1986 in the same hospital.

where:

where:

**4. Case study** 

k is the cluster numbers Pi is the weight of cluster

n is the element numbers

vj is the score of element j

C is the total score of equipment

Different professional were involved in the HTA protocol development; in our opinion HTA team may benefit from including professionals of structures that are frequently involved in the process of acquisition or evaluation and use of technologies. The team was then composed of five professionals: the hospital general manager, the sanitary manager, a clinical engineer, the superintendent and the risk protection and prevention manager.

The choice of including individual used to do assessment processes facilitates the identification and scoring of criteria and their constituents.

Additionally, since periodic controls of hospital technology are mandatory, clinical engineering services hold information related to every equipment existing in the hospital and the collected data is often the result of monitoring procedure and checklist filling, by scoring3 each voice of these checklist it will immediately score the related cluster attribute.

Once the team has been formed and the information sources

identified, the application of the described methodology for the development of the assessment protocol took place as reported below.

#### **4.2 Protocol development**

The multidimensional analysis, typical of a health technology assessment process as mentioned above, determined the identification of five clusters related to key areas of the assessment: S-Safety, L-Legal, O-Organizational, E-Economic, T-technological their constituents.

The protocol was intended for the classification of a wide variety of equipment hence the definition of elements and thresholds was derived from previous experience of different professionals, as discussed in the next paragraphs. The figure below reports some of the selected variables.

 3The process of assignment of a value to a monitoring checklist field

Health Technology Assessment:

**4.3 Weights and ratings** 

criteria (see table 2).

Organizational Aspects (O)

clusters.

entire criteria set.

An Essential Approach to Guide Clinical Governance Choices on Risk Management 77

Cluster weights assignment has been obtained using pair-wise comparison; preferences

Pair-wise comparison highlighted the substantial equivalence between Legal and Safety

A successive pair-wise comparison was performed for organizational (O), economic (E) and

**Economical Aspects (E)** 

O

TE

On the other hand, cluster elements were compared using a ratings approach instead of pair wise comparison since their large number and their relative incidence on all the identified

The rating approach allowed us to define individual weight of elements for each criterion; the table depicts an example of weighting procedure for an element with respect to all the

**Legal and social aspects (T) Weight %** 

<sup>1</sup>50 %

**Sum of** 

<sup>2</sup>O = 4 PO= 66.67%

<sup>1</sup>T = 1 PT= 16.665%

**Scores Weight %** 

E = 1 PE= 16.665%

*Preference Score Parity 1 Miminal 2 Little 3 Medium 4 Maximal 5 Absolute 6* 

were expressed with a scale from 1 to 6 as reported in table.

Table 1. Preference scored, higher scores mean higher preference

Safety (S) S L

Table 2. Pair wise comparison of safety and legal criteria

**Technical Aspects (T)** 

> O 2

Technical Aspects (T)

technological (T) approach resulting in the percentages reported below.

Table 3. Weighting of Technical, Economical and Organizational criteria

Fig. 3. Evaluation clusters elements chosen for protocol implementation

HTA protocol developed is based on a hierarchical classification of evaluation criteria (S, L, O, E, T) and their constituents (si, li, oi, ei, ti); in particular, each criteria is compared against the other through a pair wise comparison and weighted in order to obtain a numerical equivalent of its relevance with respect to the totality of the clusters.

However, for this case study, the protocol has been modelled with different pathways since safety (S) and the legal-ethical-social (L) aspects, may in practice individually determine an *Out of order* condition.

A primary classification on legal (L) and safety (S) aspects is realized, and if equipment get an acceptable score the classification goes further with the organizational, economic and technological scoring. A first threshold was then identified for that first classification, meaning the minimum adequacy of the equipment with respect to the compliance to regulatory standards and relative related risks.

In case of non-sufficient evaluation the process stops with an *Out of order* classification outcome, saving time not proceeding through the overall assessment.

However, technology classification is strictly dependant on the weights and ratings assigned to cluster and cluster elements; this is always related to the purpose of the assessment and the HTA team choices that for the specific case study have been further explained.

#### **4.3 Weights and ratings**

76 Risk Management for the Future – Theory and Cases

Fig. 3. Evaluation clusters elements chosen for protocol implementation

equivalent of its relevance with respect to the totality of the clusters.

outcome, saving time not proceeding through the overall assessment.

*Out of order* condition.

regulatory standards and relative related risks.

HTA protocol developed is based on a hierarchical classification of evaluation criteria (S, L, O, E, T) and their constituents (si, li, oi, ei, ti); in particular, each criteria is compared against the other through a pair wise comparison and weighted in order to obtain a numerical

However, for this case study, the protocol has been modelled with different pathways since safety (S) and the legal-ethical-social (L) aspects, may in practice individually determine an

A primary classification on legal (L) and safety (S) aspects is realized, and if equipment get an acceptable score the classification goes further with the organizational, economic and technological scoring. A first threshold was then identified for that first classification, meaning the minimum adequacy of the equipment with respect to the compliance to

In case of non-sufficient evaluation the process stops with an *Out of order* classification

However, technology classification is strictly dependant on the weights and ratings assigned to cluster and cluster elements; this is always related to the purpose of the assessment and

the HTA team choices that for the specific case study have been further explained.

Cluster weights assignment has been obtained using pair-wise comparison; preferences were expressed with a scale from 1 to 6 as reported in table.


Table 1. Preference scored, higher scores mean higher preference

Pair-wise comparison highlighted the substantial equivalence between Legal and Safety criteria (see table 2).


Table 2. Pair wise comparison of safety and legal criteria

A successive pair-wise comparison was performed for organizational (O), economic (E) and technological (T) approach resulting in the percentages reported below.


Table 3. Weighting of Technical, Economical and Organizational criteria

On the other hand, cluster elements were compared using a ratings approach instead of pair wise comparison since their large number and their relative incidence on all the identified clusters.

The rating approach allowed us to define individual weight of elements for each criterion; the table depicts an example of weighting procedure for an element with respect to all the entire criteria set.

Health Technology Assessment:

An Essential Approach to Guide Clinical Governance Choices on Risk Management 79

A first threshold of sufficiency (S0=6) was identified and only equipment that passes this score proceed to the rest of classification, otherwise it is classified out of service (see Figure 4). Equipment that proceeds with the rest of assessment process is ranked based on the

> *COTE PO* \**VO PT* \**VT PE* \**VE PO PT PE*

For this last classification, three thresholds were established S1 (= 4), S2 (= 5) and S3 (= 6),

 *Class 2* with S1 ≤ C < S2 that identifies equipment that does not respond effectively to the needs of that facility (e.g. non complete standards compliance), but could be usefully employed elsewhere (e.g. other country), and then *donated* to other institutions

 *Class 3* with S2 ≤ C < S3, which identifies the equipment that could be better employed inside the hospital (i.e. based on the percentage/frequency of its usage) and then

the *Class 4* with C ≥ S3 identifies the equipment that, at the time of the assessment, fully

However, threshold were chosen on the HTA team practical experience, they have to be confirmed by feedback results obtained by applying protocol in preliminary studies as for

comply with the adopted constraints and do not need intervention.

*Class 1* with C < S1 that identifies equipment that still be declared "*out of service*";

(4)

Economical, Organizational and Technological criteria as shown below;

Based on the obtained score the equipment is classified.

Fig. 4. Evaluation protocol pathways and classification

which led to the identification of 4 classifications:

our case study as presented in the next section.

or health care facilities;

*relocated* ;


Table 4. Individual ranking of S2 parameter

Finally, as will be detailed below, clusters elements rating was achieved by utilizing the data collected by clinical engineering services; each cluster elements (equipment characteristics/conditions) was therefore scored by means of continuous monitoring and reference checklist analysis.

A value vj for each element of the cluster on a scale from 1 to 10, was assigned where 10 represents the best condition.

Once values and weights are assigned the technology assessment can be finalized: the process proceeds by valuing each technology aspects thorough the pathways described in the next paragraph, the final score is computed and compared with determined thresholds in order to *classify* technology.

#### **4.4 Classification**

As already stated, the protocol includes different pathways; this solution was adopted because legal and safety criteria individually assess the compliance of the device to national and internationals rules and the risks associated to the use of equipment itself.

This first protocol step allows us to classify equipment that has to be immediately declared *Out of Service*; scores for each of the step of the protocol were computed as weighted averages based on the obtained cluster estimations and weights previously assigned to each cluster.

A partial evaluation is computed based on the formula reported below to obtain the assessment:

$$C\_{SL} = \frac{P\_S \, \text{\*} \, V\_S + P\_L \, \text{\*} \, V\_L}{P\_S + P\_L} \tag{3}$$

A first threshold of sufficiency (S0=6) was identified and only equipment that passes this score proceed to the rest of classification, otherwise it is classified out of service (see Figure 4). Equipment that proceeds with the rest of assessment process is ranked based on the Economical, Organizational and Technological criteria as shown below;

$$C\_{O\to} = \frac{P\_O \, \text{\*}\, V\_O + P\_T \, \text{\*}\, V\_T + P\_E \, \text{\*}\, V\_E}{P\_O + P\_T + P\_E} \tag{4}$$

Based on the obtained score the equipment is classified.

78 Risk Management for the Future – Theory and Cases

S 4

L 3

1

Finally, as will be detailed below, clusters elements rating was achieved by utilizing the data collected by clinical engineering services; each cluster elements (equipment characteristics/conditions) was therefore scored by means of continuous monitoring and

A value vj for each element of the cluster on a scale from 1 to 10, was assigned where 10

Once values and weights are assigned the technology assessment can be finalized: the process proceeds by valuing each technology aspects thorough the pathways described in the next paragraph, the final score is computed and compared with determined thresholds

As already stated, the protocol includes different pathways; this solution was adopted because legal and safety criteria individually assess the compliance of the device to national

This first protocol step allows us to classify equipment that has to be immediately declared *Out of Service*; scores for each of the step of the protocol were computed as weighted averages based on the obtained cluster estimations and weights previously assigned to each

A partial evaluation is computed based on the formula reported below to obtain the

*CSL PS* \**VS PL* \**VL PS PL*

and internationals rules and the risks associated to the use of equipment itself.

<sup>T</sup><sup>T</sup>

S 5

L 3

OE 1

S = 19 PS|S2 = 77.94 %

L = 9 PL|S2 = 13.23 %

O = 3 PO|S2 = 2.94 %

E = 1 PT|S2 = 4.42 %

<sup>2</sup>T = 3 PE|S2 = 1.47 %

(3)

L O T E

<sup>O</sup>OT

S 5

3

**S2: Safety standard compliance** 

**Sum of** 

5

Table 4. Individual ranking of S2 parameter

reference checklist analysis.

represents the best condition.

in order to *classify* technology.

**4.4 Classification** 

cluster.

assessment:

S S

**Scores Weight %** 

<sup>L</sup><sup>L</sup>

Fig. 4. Evaluation protocol pathways and classification

For this last classification, three thresholds were established S1 (= 4), S2 (= 5) and S3 (= 6), which led to the identification of 4 classifications:


However, threshold were chosen on the HTA team practical experience, they have to be confirmed by feedback results obtained by applying protocol in preliminary studies as for our case study as presented in the next section.

Health Technology Assessment:

Table 6. Equipment scoring results

*SL*

**5. Discussion** 

provided.

criteria (see Table 3), ultrasound score was:

*PV PV P V <sup>C</sup> PPP*

*oo EE TT*

*OET*

relocation of the equipment to another hospital department.

economic and/or organizational impact and may allow:

planned out of service in particular *Relocation.* 

An Essential Approach to Guide Clinical Governance Choices on Risk Management 81

As done for the first step and using the previously computed weights of the assessment

On the base of the considered threshold, the proposed model classified the equipment as

The equipment complies with safety and legal standards but it usage percentage and its efficacy for that centre is not sufficient to maintain in use the device; however, since the overall state of the ultrasound achieves a good usability, the model correctly suggested a

Technology assessment cannot replace the clinical governance decision makers as these topics are often related to variables dependent on their sensitivity; however, HTA certainly improve management processes through a more effective use of information and knowledge available. HTA leads to a wider risk analysis and a better health needs assessment, it makes possible an extensive knowledge of the technology characteristics, its effects on individuals health, its

improved selection processes: for the selection of technologies to adopt through an

 efficient management of procurement processes, since a better understanding of the overall characteristics of the technology can enhance negotiation skills in dealing with suppliers; the preparation of all the organizational, professional and financial resources necessary for effective and efficient use of technology in order to increase the level of performance

Generally, HTAs are mainly related to technology or equipment purchase; results are presented in the form of reports or indicators to help decision makers in their conclusions. However, in our knowledge few of them have been dedicated to a classification of the

The proposed methodology, based on the requirements and constraints often suggested by decision makers themselves, provides an indicator (a numerical value) through which the equipment may be classified; using the algorithm all the information associated with the assessment are synthesized to allow managers in easily getting an overall picture of capital

explicit comparison between the "*needs*" (health needs, resources available);

overall state of hospital equipments especially with relocation/donation purposes.

equipment state and usage implications in the hospital facility.

5.204 66.67+6.075 16.665+5.935 16.665 5.471 100

 Criteria Score S Safety 7.415 L Legal 9.238 O Organizational 5.204 E Economic 6.075 T Technological 5.935

#### **4.5 Preliminary application of protocol**

The ultrasound is placed at the Biotechnology centre of the hospital. It was purchased in 1998 at a cost of € 75,000, this price refers to the unit acquisition cost and includes two linear probes and a convex probe; moreover, the vendor signed of a full risk contract type on this equipment.4

The ultrasound equipment has been classified through the hierarchy protocol proposed, monitoring checklist has been examined and the assessment process started.

As explained in the previous section the first classification was made based on Safety and Legal criteria.

As example, we reported a complete score of ultrasound equipment for the Safety criteria; in case of non applicable evaluation o some of the listed variables the parameter itself is non-considered in the total criteria scoring process. The condition of nonapplicability of parameter evaluation has been identified by using hash (#) symbol (see table below).


Table 5. Safety criteria elements evaluation

Safety an Legal properties analysis achieved the first step of the examination protocol; equation 3 was computed with previously evaluated weights (see Table 2) and scores obtained by monitoring checklist recognition.

$$C\_{SL} = \frac{P\_S \cdot V\_S + P\_L \cdot V\_L}{P\_S + P\_L} = \frac{7.415 \cdot 50 + 9.238 \cdot 50}{100} = 8.327\%$$

The value obtained is above S0 threshold therefore, the assessment process can continue to the evaluation of Organizational, Technological and Economic aspects; in Table 6 are reported the scores of ultrasound with respect to each criteria:

<sup>4</sup>The contract covers preventive and corrective maintenance program for equipment full technical support including workforce and spare parts.


Table 6. Equipment scoring results

As done for the first step and using the previously computed weights of the assessment criteria (see Table 3), ultrasound score was:

$$C\_{SL} = \frac{P\_o \cdot V\_o + P\_E \cdot V\_E + P\_T \cdot V\_T}{P\_O + P\_E + P\_T} = \frac{5.204 \cdot 66.67 + 6.075 \cdot 16.665 + 5.935 \cdot 16.665}{100} = 5.471\%$$

On the base of the considered threshold, the proposed model classified the equipment as planned out of service in particular *Relocation.* 

The equipment complies with safety and legal standards but it usage percentage and its efficacy for that centre is not sufficient to maintain in use the device; however, since the overall state of the ultrasound achieves a good usability, the model correctly suggested a relocation of the equipment to another hospital department.

#### **5. Discussion**

80 Risk Management for the Future – Theory and Cases

The ultrasound is placed at the Biotechnology centre of the hospital. It was purchased in 1998 at a cost of € 75,000, this price refers to the unit acquisition cost and includes two linear probes and a convex probe; moreover, the vendor signed of a full risk contract type on this

The ultrasound equipment has been classified through the hierarchy protocol proposed,

As explained in the previous section the first classification was made based on Safety and

As example, we reported a complete score of ultrasound equipment for the Safety criteria; in case of non applicable evaluation o some of the listed variables the parameter itself is non-considered in the total criteria scoring process. The condition of nonapplicability of parameter evaluation has been identified by using hash (#) symbol (see

S1 Visual assessment 8 S2 Safety standard compliance 10 S3 Overall safety conditions 10 S4 User manual availability 10 S5 User manual suitability 10 S6 Service manual availability 1 S7 Service manual suitability # S8 Device alarms: suitability and manageability # S9 Equipment caution or critical issues 7

S11 State of use compared to the rooms and facilities 10

Safety an Legal properties analysis achieved the first step of the examination protocol; equation 3 was computed with previously evaluated weights (see Table 2) and scores

The value obtained is above S0 threshold therefore, the assessment process can continue to the evaluation of Organizational, Technological and Economic aspects; in Table 6 are

4The contract covers preventive and corrective maintenance program for equipment full technical

S10 Update training for the personnel than adverse events

*SS LL*

*PV PV <sup>C</sup> P P*

reported the scores of ultrasound with respect to each criteria:

*S L*

Table 5. Safety criteria elements evaluation

obtained by monitoring checklist recognition.

*SL*

support including workforce and spare parts.

Evaluated Parameter Score

involving the equipment <sup>1</sup>

7.415 50 9.238 50 8.327 100

monitoring checklist has been examined and the assessment process started.

**4.5 Preliminary application of protocol** 

equipment.4

Legal criteria.

table below).

Technology assessment cannot replace the clinical governance decision makers as these topics are often related to variables dependent on their sensitivity; however, HTA certainly improve management processes through a more effective use of information and knowledge available.

HTA leads to a wider risk analysis and a better health needs assessment, it makes possible an extensive knowledge of the technology characteristics, its effects on individuals health, its economic and/or organizational impact and may allow:


Generally, HTAs are mainly related to technology or equipment purchase; results are presented in the form of reports or indicators to help decision makers in their conclusions.

However, in our knowledge few of them have been dedicated to a classification of the overall state of hospital equipments especially with relocation/donation purposes.

The proposed methodology, based on the requirements and constraints often suggested by decision makers themselves, provides an indicator (a numerical value) through which the equipment may be classified; using the algorithm all the information associated with the assessment are synthesized to allow managers in easily getting an overall picture of capital equipment state and usage implications in the hospital facility.

Health Technology Assessment:

governance:

support decision tool.

risk management.

**7. References** 

An Essential Approach to Guide Clinical Governance Choices on Risk Management 83

association with diagnostic or therapeutic accuracy, access time to medical services and it

Evaluation of health technologies must be a continuous activity to be conducted before their introduction but also, and with more consideration, during their entire lifecycle. Organizations with high reliability, where healthcare is included, aim to reach high safety standards by focusing the activity not on avoiding isolated failures, but on the ability of rendering the entire

This work suggested some new ideas and future development of the proposed model; in

In conclusion, in our opinion two main improvements will become widespread in clinical

 evolved monitoring checklist: checklist designed in collaboration with clinical engineers and hospital professionals can contribute to obtain a feasible evaluation of

hospital information systems integration: assessment models may be integrated in HFs'

In a period in which HF information is increasing, it is reasonable to imagine an evolved information/computer support system able to store data provided by monitoring procedure and to proceed rapidly to technology classification on predetermined rules and thresholds. Intelligent decision support systems will be able to highlight to HF managers the state and the needs of capital equipment of the entire facility, in a more complete approach to hospital

Agrawal, VP, Kohli, V., Gupta, S. *"Computer aided robot selection: A multiple attribute decision making approach"*. 1991, International Journal of Production Research, Vol. 29, pp. 1629-1644 Alexander C. and Sheedy E. "*The Professional Risk Managers Handbook: A Comprehensive Guide* 

Armstrong J.S. *"The Value of Formal Planning for Strategic Decisions*". 1982, Strategic

Banta D, Oortwijn W. "*Health Technology Assessment and Health Care in the European Union*". International Journal of Technology Assessment in health Care 2000; 16(2):626-635 Banta H.D. and Luce B. R..: *"Health Care Technology and its Assessment: an international* 

Bates D.W. and Gawande A.A.: "*Improving safety with information technology*" New Engl J

Battista R. "*Innovation and diffusion of health-related technologies. A conceptual framework"* 1989

Battista R. Hodge M.J *" The Evolving Paradigm of Health Tecnology Assessment : Reflections for* 

Birch S, Donaldson C. *"Applications of cost benefit analysis to health care: departures from welfare* 

*to Current Theory and Best Practices"*. 2005, PRMIA Publications.

*perspective".* Oxford University Press, 1993, pp 197-203.

Management Journal, issue 3 pp:197–211.

Int J Technol Asses Health Care ; 5(2):227-48

*the Millennium*", CMAJ ,1999, 160 (10) pp 1464-1467

*economic theory."* J Health Economics 1987; 6: 211-25.

Med; 2003; 348(25): pp. 2526-34

influences the safety of personnel that makes use of those facilities.

system robust and practical against relative human and operational risks.

characteristics/condition of capital equipment;

information system to allow continuous

particular, more model application are desirable to enhance the protocol efficacy.

Modern hospitals own a huge amount of equipments, in some case these technologies an underused or not used properly, in some other case equipments are used still if they do not properly comply with safety standard or simply can not be used because not more compliant with emerging ones. Still, it is not common to find HFs relocating technologies, after a HTA, in order to achieve better performance in a different hospital area.

In all of this cases, by appropriately indicate threshold, the proposed model may evaluate if the technology under investigation meets the requirements/provisions of use, and how much convenient may result a specific intervention (disposal/relocation/replacement).

A considerable importance in the application of this protocol has to be assigned to the design of the evaluation checklist and the px weight attribution in pair wise comparisons, appropriate design choices allow for reduction of operator-dependent errors and the correct evaluation of technologies.

In our opinion, the choice of including professionals from the clinical engineering service of the hospital may became a common practice; it gives the chance to score cluster elements in a easy and continuous way. However, clinical engineering services are often outsourced, this leads to a difficult integration with hospital information systems while it also makes difficult equipment monitoring data retrieval.

Monitoring checklists have to be discussed and refinements may be suggested to allow a limited choice to the evaluator, rather than proposing a free answer, reducing the bias of the individual operator with respect to the overall assessment.

The classification system proposed in this paper was built to be easy to use and to allow an immediate interpretation to non-experts. As well as hierarchic methods, the model may led to a broader evaluations (i.e. with a larger number of clusters) or more focused assessments; in general, the use of an automated assessment protocol may be profitably integrated in information systems as a tool for a better management of technologies.

The model has been tested for the evaluation of hospital equipment and gave encouraging feedbacks; it can provide regular analysis on the state of technological equipment of an HF.

A crucial and delicate point was the HTA team building; the evaluations reflect the experience, knowledge and sensibility of individuals involved in the study.

In our opinion, as well as all HTA methodologies the model presented should improve healthcare operators and individual consciousness with respect to the whole healthcare system.

However, the application of any HTA methodology is dependant on the acceptability of users (patients, personnel, managers) and on the integration that the methodology may achieve with HF information systems.

Nevertheless, in this our first experience, the received feedbacks were encouraging and the information level of the structure was fine enough to start thinking a HTA methodology integration.

#### **6. Future work and closure**

Efficiency of medical equipment and devices, from which nowadays the totality of medical services depends, has direct and indirect influences on the quality of the offered service in association with diagnostic or therapeutic accuracy, access time to medical services and it influences the safety of personnel that makes use of those facilities.

Evaluation of health technologies must be a continuous activity to be conducted before their introduction but also, and with more consideration, during their entire lifecycle. Organizations with high reliability, where healthcare is included, aim to reach high safety standards by focusing the activity not on avoiding isolated failures, but on the ability of rendering the entire system robust and practical against relative human and operational risks.

This work suggested some new ideas and future development of the proposed model; in particular, more model application are desirable to enhance the protocol efficacy.

In conclusion, in our opinion two main improvements will become widespread in clinical governance:


82 Risk Management for the Future – Theory and Cases

Modern hospitals own a huge amount of equipments, in some case these technologies an underused or not used properly, in some other case equipments are used still if they do not properly comply with safety standard or simply can not be used because not more compliant with emerging ones. Still, it is not common to find HFs relocating technologies,

In all of this cases, by appropriately indicate threshold, the proposed model may evaluate if the technology under investigation meets the requirements/provisions of use, and how much convenient may result a specific intervention (disposal/relocation/replacement).

A considerable importance in the application of this protocol has to be assigned to the design of the evaluation checklist and the px weight attribution in pair wise comparisons, appropriate design choices allow for reduction of operator-dependent errors and the correct

In our opinion, the choice of including professionals from the clinical engineering service of the hospital may became a common practice; it gives the chance to score cluster elements in a easy and continuous way. However, clinical engineering services are often outsourced, this leads to a difficult integration with hospital information systems while it also makes

Monitoring checklists have to be discussed and refinements may be suggested to allow a limited choice to the evaluator, rather than proposing a free answer, reducing the bias of the

The classification system proposed in this paper was built to be easy to use and to allow an immediate interpretation to non-experts. As well as hierarchic methods, the model may led to a broader evaluations (i.e. with a larger number of clusters) or more focused assessments; in general, the use of an automated assessment protocol may be profitably integrated in

The model has been tested for the evaluation of hospital equipment and gave encouraging feedbacks; it can provide regular analysis on the state of technological equipment of an HF. A crucial and delicate point was the HTA team building; the evaluations reflect the

In our opinion, as well as all HTA methodologies the model presented should improve healthcare operators and individual consciousness with respect to the whole healthcare system. However, the application of any HTA methodology is dependant on the acceptability of users (patients, personnel, managers) and on the integration that the methodology may

Nevertheless, in this our first experience, the received feedbacks were encouraging and the information level of the structure was fine enough to start thinking a HTA methodology

Efficiency of medical equipment and devices, from which nowadays the totality of medical services depends, has direct and indirect influences on the quality of the offered service in

after a HTA, in order to achieve better performance in a different hospital area.

evaluation of technologies.

difficult equipment monitoring data retrieval.

achieve with HF information systems.

**6. Future work and closure** 

integration.

individual operator with respect to the overall assessment.

information systems as a tool for a better management of technologies.

experience, knowledge and sensibility of individuals involved in the study.

In a period in which HF information is increasing, it is reasonable to imagine an evolved information/computer support system able to store data provided by monitoring procedure and to proceed rapidly to technology classification on predetermined rules and thresholds.

Intelligent decision support systems will be able to highlight to HF managers the state and the needs of capital equipment of the entire facility, in a more complete approach to hospital risk management.

#### **7. References**


**5** 

*France* 

**Preventing Societal Health Risks Emerging in** 

The traditional distinction between preventive and curative medicine has probably served, among others, programmatic and educational purposes, but does not look so robust (and valid) when used to back-up choices for innovative technologies or justify managerial economic options. Obviously, the line between both some secondary or tertiary preventive actions and many curative ones is often void, and above all a simple question of optical

Indeed, in our opinion, prevention and precaution are the underlying humanistic driving forces that have, or should have, always presided to the management of health risks. And this should remain untouched. Is this merely an ideological choice? In any case, the implementation of these concepts is also definitely not detrimental to long term economical

Nanotechnology refers to the design, characterization, production and application of structures, devices and systems that have novel physical, chemical and biological properties, by controlling shape and size at the nanometer scale1 (Ebbesen and Jensen, 2006). While this multidisciplinary scientific field is undergoing explosive development, it is also subject to many controversial debates among scientists, but not only2. The design

1 A nanometer (nm) is equal to one billionth of a meter (10-9 meter); formerly called millimicron (1/1000 of a micron), the nano scale was denoted by the symbol mµ. As an illustration, the diameter of a red blood cell is roughly 1 000 nanometers (10-6 meter). 2 A 3 days Conference on Risks associated with nanoparticles and nanomaterials was organised by the French National Institute on Research and Security (INRS) in April 2011, to discuss occupational health research issues. A public debate on Nanotechnologies was held in France from October 15th 2009 to February 24th 2010, organized by the National Commission on Public Debate (CNDP) the National

considerations, another real asset, nor a limiting factor to progress and innovation.

official and independent body (full details are available from:

http://www.debatpublicnano.org/index.html).

**1. Introduction** 

perspective.

**the Development Of Nanomedicine –** 

*2Centre National de Recherche Scientifique LRGP-UPR 3349, Nancy,* 

**What Should Prevail?** 

Roberte Manigat1, Florent Allix2,

Céline Frochot2 and Jean Claude André2,3 *1Ministère du Travail, de l'Emploi et de la Santé, Paris,* 

*3Centre National de Recherche Scientifique INSIS, Paris,* 

