**Extending Health Information System Evaluation with an Importance‐Performance Map Analysis**

Mohd Idzwan Mohd Salleh, Rosni Abdullah and Nasriah Zakaria

Additional information is available at the end of the chapter

http://dx.doi.org/10.5772/68122

#### **Abstract**

Evaluation of a health information system is necessary for determining effective use and for enhancing the productivity of medical practitioners. However, the current system evaluation toolkit does not recommend specific areas required for further improvement. The objective of this chapter was to identify those constructs and their attributes that were the most suitable candidates for managerial intervention by applying partial least squares structural equation modeling. In doing so, the quantitative survey was adopted from the past studies together with new items creation representing system quality, records quality, service quality, and knowledge quality as the predictors while effective use and user performance as the outcomes. When extending the findings in importance‐ performance map analysis, two‐system quality attributes (workflows fit and work styles fit) and all‐knowledge quality attributes exhibited higher importance rank for managerial actions. The chapter also provides a valuable recommendation for the policy and decision‐ makers at the managerial level on how to apply the proposed system evaluation method in producing more efficient strategic‐planning strategies for further system upgrades and new implementation at health facilities.

**Keywords:** summative evaluation, health information system, effectiveness, partial least squares structural equation modeling, importance‐performance map analysis

### **1. Introduction**

The widespread implementation and adoption of health information systems (HISs) around the world are believed to improve access and use of health data in ensuring high quality of care and health system efficiency and fostering clinical research [1, 2]. The acceleration of HIS implementations will further enhance sharing of health information electronically across

© 2017 The Author(s). Licensee InTech. This chapter is distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/3.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

different clinical settings [3] that eventually generate quality benefits and minimize medical costs from avoiding unnecessary clinical trials, examinations, and treatments [4]. Therefore, the management and presentation of HISs are vital to accelerate patient care and its continuity across health institutions [5]. The success of system implementation relies upon a high quality of information outputs from HISs required to make timely and accurate clinical decisions by various health practitioners [6]. Besides enabling care continuity, HIS is regarded as the wealthiest source of clinical evidence to support continuous communication among individual clinicians and surgical team works [7]. With the use of HIS, it is not only capable to reduce human errors [8] but also contributes to an increased adherence to clinical guidelines and deterrence of medical errors [9, 10], thereby delivering greater patient safety and medication management [11].

In Malaysia, the expenditures of customized HISs are fully supported by the government in the efforts to retain a higher standard of patient care [12, 13]. All new public hospitals should be equipped with HISs designed from multiple vendors hired by the government. Although the investment of IS can improve health service, it will also present more costs in mainte‐ nance, hardware replacements, end‐user trainings, and system upgrades [14, 15]. Increasing medical costs [16] and enormous budget cuts among local hospitals have demanded for a comprehensive evaluation of HIS to investigate the most possible strengths and weaknesses for further improvements. In reality, the effectiveness of HIS adoption among implemented government hospitals had never been assessed since its first kick‐off at Selayang Hospital in 1999. Hospitals with HISs are repeatedly distributing user satisfaction surveys without concentrating on significant success factors and impact on the performance of the health personnel. They conducted these surveys to satisfy the auditing needs but the results were still insufficient in recommending which critical attributes for improving system use and user productivity. As a consequence, the government hospitals were still incapable of choosing the right HIS and vendors and even assessing its performance after implementation [17]. A systematic IS evaluation will not only promote efficient use and medical cost savings but also cope with unresolved issues of clinicians' heavy workloads and shortage of specialists in this multi‐racial country [14, 15].

Identifying the needs of health workforce and acknowledging the characteristics of HIS are essential to their productivity that must be emphasized in any evaluation studies [18, 19]. For that reason, recognizing the main attributes of HIS can improve health practitioners' perfor‐ mance from their daily use. Strategies to upgrade an HIS could not precede with an absence of in‐depth knowledge about the most significant HIS characteristics in predicting user productivity. Consequently, there will be wasted expenses on any system upgrades without careful understanding of the potential system impacts or benefits to the user performance, thereby introducing dissatisfaction and risks of system failure [18].

Unfortunately, there is little evidence on the prior HIS research in measuring the influence of IS attributes toward satisfaction and productivity of medical practitioners [20, 21]. Besides, the previous evaluation works did not completely assess the importance and performance of multiple HIS attributes especially in ranking those attributes with high importance for managerial attention. There are only two current studies attempted to prioritize different HIS quality measures among small samples acquired in one public hospital [18, 19]. Furthermore, the current trends in examining HIS use and user satisfaction in the scholarly publications are still plenty by ignoring core success drivers that will predict user and organizational impacts. By contrast, there are many empirical studies on HIS evaluation concerning the effects of system quality, information quality, service quality, usage, user satisfaction, and net benefits in the developed and developing countries [22, 23], but none of them address on the critical quality or success factors required for managerial response. Most studies only present significant results without recommending specific measures or indicators that will guide the hospitals in prioritizing the most important indicators for improving effective use and health personnel productivity.
