**Transforming Risk Assessment Tools from Paper to Electronic**

Daniel Bergeron1 and Kristiina Hyrkäs1,2 *1Maine Medical Center, Portland Maine; 2University of Southern Maine USA* 

### **1. Introduction**

280 Novel Approaches and Their Applications in Risk Assessment

Kunetz, G. (1966). *Principles of direct current resistivity prospecting*, Geoexploration

Lino, M., Mériaux P., Royet P. (2000). *Méthodologie de diagnostic des digues appliquée aux levées* 

Loke, M.H. & Barker, R.D. (1996). *Rapid least-squares inversion of apparent resistivity pseudosections using a quasi-Newton method, Geophysical Prospecting* Vol. 44: 211*–*226. Mallet, C. & Bretar, F. (2008). Full-waveform topographic LiDAR: State-of-the-art. *ISPRS* 

Marescot, L., Rigobert, S., Palma Lopes, S., Lagabrielle, R., & Chapellier, D. (2006). A general

McNeill, J. D. (1980). *Electromagnetic terrain conductivity measurement at low induction numbers*,

Mériaux, P. & Royet, P. C. (2001). *Surveillance, Maintenance and Diagnosis of Flood Protection* 

Mériaux, P., Vennetier, M., Aigouy, S , Hoonakker, M. & Zylberblat, M. (2006). Diagnosis

Oldenburg, D. W. & Li, Y. G. (1999). Estimating depth of investigation in dc resistivity and

Palacky, G.J. (1991). Resistivity characteristics of geological targets, *in* M*.* N. Nabighian (ed.), *Electromagnetic Methods in Apllied Geophysics Vol. 1,* Soc. Explor. Geophys., pp. 53–129

Park, S. K. & Van, G.P. (1991). Inversion of pole-pole data for 3-D resistivity structure

Radzicki, K., Bonelli, S. (2010). Thermal Seepage Monitoring in the Earth Dams with Impulse Response Function Analysis Model, *8th ICOLD European Club*, Innsbruck, Austria Samyn, K., Bitri A., Blondel, A. & Lebert, F. (2009). *Détermination de la présence de karst sous les* 

Sjödahl P., Torleif, D. & Zhou, B. (2006). 2.5D resistivity modeling of embankment dams to assess influence from geometry and material properties, *Geophysics*, Vol. 71(No. 3):, 107-114. Stummer P., Maurer, H., & Green, A.G. (2004). Experimental design: electrical resistivity

Tarantola, A. & Valette, B (1982). Inverse Problems = Quest for Information. *Journal of* 

Tong, L.-T. & Yang, C.-H. (1990). Incorporation of topography into two-dimensional

Tsourlos, P.I., Szymanski, J.E., & Tsokas, G.N. (1999). The effect of terrain topography on

Yoon, J. S., Shin, J. I. & Lee, K. S. (2008). Land cover characteristics of airborne LiDAR

intensity data: a case study, IEEE *Geoscience and Remote Sensing Letters,* Vol. 9; 463-

commonly used resistivity arrays, *Geophysics*, Vol. 64:, 1357-1363

*à la présence de ces conduits souterrains naturels*. BRGM/RP-57650-FR. Shan, J. & Toth, C. K. (Eds) (2009). *Topographic Laser Ranging and Scanning. Principles and* 

*levées domaniales du bassin de la Loire et réduction du risque d'effondrement de la digue lié* 

data sets that provide optimum subsurface information, *Geophysics*, Vol. 69(No. 1):

*Technical Note TN-6*, Geonics, Ltd., Mississauga, Ontario, Canada.

approach for DC apparent resistivity evaluation on arbitrarily shaped 3D

*Dikes - A Practical Handbook for Owners and Operators*, ISBN 978-2-7592-0037-5,

and management of plant growth on embankment dams and dykes*, ICOLD, 22nd* 

*Journal of Photogrammetry and Remote Sensing*, Vol. 64: 1–16.

structures, *Journal of Applied Geophysics* Vol. 60: 55–67.

*Conference on Large Dams*, Barcelone, Spain, pp. 551–567

beneath arrays of electrodes, *Geophysics* Vol. 56: 951–960.

resistivity inversion. *Geophysics,* Vol. 43: 1450-1469.

IP Surveys, *Geophysics* Vol. 64(No. 2): 403–416.

monographs 1.

Cemagref Editions

*Processing*. CRC Press.

*Geophysics*, Vol. 50: 159-170.

120-139.

466.

*de Loire moyenne*, Cemagref éditions, Paris.

As the use of electronic health records (EHR) continues to grow, there is an increasing need for the development and translation of risk assessment tools for use in electronic media. The federal government of the United States is compensating hospitals that are meeting the usage guidelines of electronic health information systems (Kocher et al. 2010). Evidence suggests that use of electronic health records improves patient safety and outcomes (Radicki & Sittig, 2011). This incentive has dramatically increased interest in EHR and content that is easily translated to an electronic format.

Risk assessment is defined in this chapter as a process of evaluating a potential hazard, likelihood of suffering, or any adverse effects (Mosby, 2008). Assessment of a risk, and subsequent prevention is becoming standard practice in healthcare (Carayon et al. 2006). Hospitals in the United States are being held accountable for preventable iatrogenic events like; pressure ulcers, falls, catheter associated urinary tract infections, and central line associated infections (United States Federal Register, 2011). Use of risk assessment tools is thus necessary to prevent these and other hospital-acquired problems. Our hospital first converted a Fall and Injury Risk Assessment tool from paper to electronic (Chapman et al. 2011). Since that time we have been translating many of our risk assessment tools into the electronic medium. The purpose of this chapter is to describe the processes necessary to transform and implement a risk assessment tool in an electronic environment.

### **2. The process of transforming a risk assessment tool**

The translation of risk assessment tools from paper to electronic is a multi-layered, complex process that aims at maintaining their integrity, scientifically tested properties and feasibility in clinical settings, thus, many factors need to be considered. Planning for this type of change requires an interdisciplinary team of nurses and Information Technology (IT) workers (see Wenzel 2002). The goal of this team is to determine the requirements of a successful transformation and implementation. The steps in preparation for this conversion include the following phases: (1.) Workflow Analysis, (2.) Design and Building an electronic version of the tool, (3.) Testing and Signoff, (4.) Training of staff/users of the tool, (5.) ' Go-Live' and Support, and ( 6.) Reporting (Courtney et al. 2005). Using electronic applications

Transforming Risk Assessment Tools from Paper to Electronic 283

calculation errors, this led to inaccurate portrayal of patient's fall risk level (i.e. no risk, moderate and high risk). When the risk assessment tool was converted, we made a special effort during this process to make sure that the tabulations were completed automatically in the electronic version of the tool by the health information system. This new function reduced

**Interviews.** Literature shows (AHRQ, 2011) that a workflow is not just composed of concrete tasks or a set of sequences, but that it also includes multiple complex mental and cognitive processes (see also Lopez et al. 2010). A workflow analysis based on interviewing can yield more important knowledge of these types of information processing activities. For example, during a risk assessment, nurses look for, and record certain observations and detailed characteristics regarding the assessments. The mental and cognitive processes that occur during this practice may include comparison of the current observation to previous-ones, or to certain standards or reference values. During paper-based risk assessment, mental and cognitive processes (e.g. reasoning) are often necessary and required to reach a conclusion of a certain amount or degree of risk. Analysis of these processes and the content of the related information are important considerations when designing an electronic assessment tool, because these help in identifying processes that may change the representation of the assessment in an electronic medium. During the interview, it is important to ask why a clinician takes some actions over others. Reasoning, calculations and critical thinking are examples of the mental and cognitive processes that are embedded in many risk assessments. A moderator/interviewer can collect the information from the interviews for the work-flow

**Observations.** Unlike the previous methods, observation can unveil discrete work processes that may not be readily shared during interviews and focus groups. It is possible that a combination of work and information flows are so complex and multi-layered that these are just simply too difficult to describe by a clinician. An observer focuses on watching the practice and the specific activities of the workflow as they are being performed (AHRQ, 2011). As nurses perform a risk assessment, the observer needs to note, most of all, the required activities (i.e. obtaining appropriate tools for measurement), their sequence (i.e. in the Fall Risk Tool we use, risk for falling precedes assignment of injury risk), and other

Considerations of the future work-flow are important, but sometimes difficult to anticipate beforehand. In one instance, we found that we did not design a future workflow for Fall Injury Risk Assessment that included recommendations for interventions. When revisiting the workflow and using the observation method, we noted that once the assessment was completed at the bedside, the nurse then needed to recommend interventions to prevent falls. In addition, nurses needed to perform recommended interventions and document their completion. An integral part of the process for preventing falls was missed. The result of the translation process was indeed an electronic assessment tool, however the observations revealed that the nurse needed to go to a paper document and note the interventions there. This activity made the workflow inefficient and led to dissatisfaction with the tool. Subsequently, the electronic assessment tool was redesigned to allow nurses to select

mathematical errors to zero and advanced the proper identification of the level of risk.

analysis by using a tape recorder or by taking hand-written notes.

appropriate interventions and to document their completion.

related tasks (AHRQ, 2011).

to deliver these risk assessment tools provide new opportunities for improving data integrity, decision support, and patient safety.

#### **2.1 Workflow analysis**

When a change in practice, such as use of an electronic risk assessment, is initiated, a workflow analysis is necessary to ensure that the design of the tool and requirements of the work/practice meet the needs of the clinician. Workflow analysis can be conducted in several different ways and examples are available in the current literature (Karsh, 2009). Most of the time the analysis begins with documentation of the current state of workflow for the process, in other words, a risk assessment conducted by a clinician. From the initial analysis, a 'future state' workflow is developed by the interdisciplinary team. The new workflow is created by combining the original workflow used in risk assessment with the functionality of the EHR system to create future state workflow.

The approaches that are described in the literature (Karsh, 2009) and commonly used in practice for workflow analysis emphasize different types of techniques for evaluating, documenting and re-designing work processes. These methods include: focus groups, interviews and observations.

**Focus Groups**. The use of focus groups is a helpful and quick method in determining if there is 'a standard' way of executing an assessment and related tasks (e.g. targeted interventions) in practice (AHRQ, 2011). The focus groups in this context mean meeting with several clinicians and staff who are familiar with the assessment tool and who can describe the steps used to complete the current risk assessment tool. During the focus group meetings, the 'moderator' collects information from clinicians and staff regarding the work flow process and attempts to capture the essential and concrete steps to obtaining the assessment. These may include, for example, looking for the patient's chart, assembling associated tools for measurement, bringing the assessment tool to the patient's bedside, and many more. The focus group method is helpful for looking at clinicians' and staff's group behaviors and especially the levels of consensus and agreement regarding the use of the assessment instrument and the steps of the workflow. The 'moderator' captures and summarizes the information most commonly on the flip-chart pages, sticky notes or a notepad for further use (AHRQ, 2011). The information gained from focus groups provides a broad overview of the workflow with a variety of nuances from clinicians and staff, but it is important to point out that if very detailed information is required, one-on-one interviews allow a deeper exploration of workflow. (AHRQ, 2011)

The workflow analysis can reveal very important and unexpected issues. For example, in a recent conversion of a fall risk assessment tool at our hospital, we found out that the predominant workflow often led to misleading risk assessment scores and ultimately incorrectly targeted interventions. The workflow analysis revealed that the main reason for this inaccuracy was, in fact, caused by 'user variability'. Although the assessment tool being converted from a paper document to an electronic version of the instrument had already been found to be superior in both its specificity and sensitivity (Chapman et al. 2011), we detected that there were still inaccurate assessments. The original assessment tool was a paper document and all tabulations were calculated manually by the nurse. If there were any

to deliver these risk assessment tools provide new opportunities for improving data

When a change in practice, such as use of an electronic risk assessment, is initiated, a workflow analysis is necessary to ensure that the design of the tool and requirements of the work/practice meet the needs of the clinician. Workflow analysis can be conducted in several different ways and examples are available in the current literature (Karsh, 2009). Most of the time the analysis begins with documentation of the current state of workflow for the process, in other words, a risk assessment conducted by a clinician. From the initial analysis, a 'future state' workflow is developed by the interdisciplinary team. The new workflow is created by combining the original workflow used in risk assessment with the

The approaches that are described in the literature (Karsh, 2009) and commonly used in practice for workflow analysis emphasize different types of techniques for evaluating, documenting and re-designing work processes. These methods include: focus groups,

**Focus Groups**. The use of focus groups is a helpful and quick method in determining if there is 'a standard' way of executing an assessment and related tasks (e.g. targeted interventions) in practice (AHRQ, 2011). The focus groups in this context mean meeting with several clinicians and staff who are familiar with the assessment tool and who can describe the steps used to complete the current risk assessment tool. During the focus group meetings, the 'moderator' collects information from clinicians and staff regarding the work flow process and attempts to capture the essential and concrete steps to obtaining the assessment. These may include, for example, looking for the patient's chart, assembling associated tools for measurement, bringing the assessment tool to the patient's bedside, and many more. The focus group method is helpful for looking at clinicians' and staff's group behaviors and especially the levels of consensus and agreement regarding the use of the assessment instrument and the steps of the workflow. The 'moderator' captures and summarizes the information most commonly on the flip-chart pages, sticky notes or a notepad for further use (AHRQ, 2011). The information gained from focus groups provides a broad overview of the workflow with a variety of nuances from clinicians and staff, but it is important to point out that if very detailed information is required, one-on-one interviews

The workflow analysis can reveal very important and unexpected issues. For example, in a recent conversion of a fall risk assessment tool at our hospital, we found out that the predominant workflow often led to misleading risk assessment scores and ultimately incorrectly targeted interventions. The workflow analysis revealed that the main reason for this inaccuracy was, in fact, caused by 'user variability'. Although the assessment tool being converted from a paper document to an electronic version of the instrument had already been found to be superior in both its specificity and sensitivity (Chapman et al. 2011), we detected that there were still inaccurate assessments. The original assessment tool was a paper document and all tabulations were calculated manually by the nurse. If there were any

integrity, decision support, and patient safety.

functionality of the EHR system to create future state workflow.

allow a deeper exploration of workflow. (AHRQ, 2011)

**2.1 Workflow analysis** 

interviews and observations.

calculation errors, this led to inaccurate portrayal of patient's fall risk level (i.e. no risk, moderate and high risk). When the risk assessment tool was converted, we made a special effort during this process to make sure that the tabulations were completed automatically in the electronic version of the tool by the health information system. This new function reduced mathematical errors to zero and advanced the proper identification of the level of risk.

**Interviews.** Literature shows (AHRQ, 2011) that a workflow is not just composed of concrete tasks or a set of sequences, but that it also includes multiple complex mental and cognitive processes (see also Lopez et al. 2010). A workflow analysis based on interviewing can yield more important knowledge of these types of information processing activities. For example, during a risk assessment, nurses look for, and record certain observations and detailed characteristics regarding the assessments. The mental and cognitive processes that occur during this practice may include comparison of the current observation to previous-ones, or to certain standards or reference values. During paper-based risk assessment, mental and cognitive processes (e.g. reasoning) are often necessary and required to reach a conclusion of a certain amount or degree of risk. Analysis of these processes and the content of the related information are important considerations when designing an electronic assessment tool, because these help in identifying processes that may change the representation of the assessment in an electronic medium. During the interview, it is important to ask why a clinician takes some actions over others. Reasoning, calculations and critical thinking are examples of the mental and cognitive processes that are embedded in many risk assessments. A moderator/interviewer can collect the information from the interviews for the work-flow analysis by using a tape recorder or by taking hand-written notes.

**Observations.** Unlike the previous methods, observation can unveil discrete work processes that may not be readily shared during interviews and focus groups. It is possible that a combination of work and information flows are so complex and multi-layered that these are just simply too difficult to describe by a clinician. An observer focuses on watching the practice and the specific activities of the workflow as they are being performed (AHRQ, 2011). As nurses perform a risk assessment, the observer needs to note, most of all, the required activities (i.e. obtaining appropriate tools for measurement), their sequence (i.e. in the Fall Risk Tool we use, risk for falling precedes assignment of injury risk), and other related tasks (AHRQ, 2011).

Considerations of the future work-flow are important, but sometimes difficult to anticipate beforehand. In one instance, we found that we did not design a future workflow for Fall Injury Risk Assessment that included recommendations for interventions. When revisiting the workflow and using the observation method, we noted that once the assessment was completed at the bedside, the nurse then needed to recommend interventions to prevent falls. In addition, nurses needed to perform recommended interventions and document their completion. An integral part of the process for preventing falls was missed. The result of the translation process was indeed an electronic assessment tool, however the observations revealed that the nurse needed to go to a paper document and note the interventions there. This activity made the workflow inefficient and led to dissatisfaction with the tool. Subsequently, the electronic assessment tool was redesigned to allow nurses to select appropriate interventions and to document their completion.

Transforming Risk Assessment Tools from Paper to Electronic 285

Eclipsys product screen shot(s) reprinted with permission from Eclipsys Corporation. Any other use or

Eclipsys product screen shot(s) reprinted with permission from Eclipsys Corporation. Any other use or

Fig. 3. Electronic assessment tool for CIWA (Clinical Institute Withdrawal Assessment for Alcohol). Each numerical value is associated with a description of the assessment option.

After the workflow analysis has been completed, the next step is to build the tool in the electronic environment. Careful consideration of the risk assessment tool's layout design and precise review of the tool's documentation functions are critical. This process is moderately easy for those risk assessment tools that have been developed using scientific methods (e.g.

Fig. 2. In this example, one of the assessment parameters is not entered. The tool was designed to NOT provide a total score unless all of the assessments are documented. This

ensures the instruments' integrity for identifying a pressure ulcer risk.

reproduction is strictly prohibited

reproduction is strictly prohibited

#### **2.2 Design & build of an electronic risk assessment tool**

An analysis of system functionality is essential after the workflow's current state has been documented. Many organizations are using a clinical informaticist for this stage of the process (Hersh, 2009). Clinical Informaticists (informaticians) have a clinical background in addition to training in health informatics. Saba & McCormick (2005) has defined clinical informatics as a specialty that integrates clinical science/practice, computer science, and information science. It is also possible that a clinician with knowledge of information technology would be involved to work with clinicians and system analysts. The goal is to have a clinician and clinical informaticist explore, describe and/or demonstrate how the IT system accommodates data entry and utilization of the other functionalities in the design of the system. Once clinicians have an understanding of the possibilities for data entry and data manipulation, they are able to propose a design of an assessment tool that leverages the system's functionality to optimize the tool's feasibility. In the recent example, scoring for the pressure ulcer risk (Braden Pressure Ulcer Risk Assessment) was designed using dropdown menus and the fields were formed based on the sections on the assessment tool (see Clarke 2007). The defined terms in each field are assigned a value, so that their selection when combined with other fields yielded an overall level of risk. The values of each parameter were necessary to perform calculations of risk. In the Braden scale the sum of these parameters indicated a certain level of risk. The electronic tool requires that *all* fields to be completed before an assignment of risk is generated. This electronic tool has greatly improved the accuracy of the risk assessments (see also Gunningberg et al. 2009) and ultimately the targeted interventions by reducing the mathematical errors made on the paper document. (See Figures 1, 2, 3).

Eclipsys product screen shot(s) reprinted with permission from Eclipsys Corporation. Any other use or reproduction is strictly prohibited

Fig. 1. Dropdown lists for risk assessment items. In this example, the Sensory Perception item presents four selections describing the patient's ability to sense discomfort from pressure. Selecting the option that most closely corresponds with the patient assessment yields a score that will populate the flow-sheet.

An analysis of system functionality is essential after the workflow's current state has been documented. Many organizations are using a clinical informaticist for this stage of the process (Hersh, 2009). Clinical Informaticists (informaticians) have a clinical background in addition to training in health informatics. Saba & McCormick (2005) has defined clinical informatics as a specialty that integrates clinical science/practice, computer science, and information science. It is also possible that a clinician with knowledge of information technology would be involved to work with clinicians and system analysts. The goal is to have a clinician and clinical informaticist explore, describe and/or demonstrate how the IT system accommodates data entry and utilization of the other functionalities in the design of the system. Once clinicians have an understanding of the possibilities for data entry and data manipulation, they are able to propose a design of an assessment tool that leverages the system's functionality to optimize the tool's feasibility. In the recent example, scoring for the pressure ulcer risk (Braden Pressure Ulcer Risk Assessment) was designed using dropdown menus and the fields were formed based on the sections on the assessment tool (see Clarke 2007). The defined terms in each field are assigned a value, so that their selection when combined with other fields yielded an overall level of risk. The values of each parameter were necessary to perform calculations of risk. In the Braden scale the sum of these parameters indicated a certain level of risk. The electronic tool requires that *all* fields to be completed before an assignment of risk is generated. This electronic tool has greatly improved the accuracy of the risk assessments (see also Gunningberg et al. 2009) and ultimately the targeted interventions by reducing the

**2.2 Design & build of an electronic risk assessment tool** 

mathematical errors made on the paper document. (See Figures 1, 2, 3).

reproduction is strictly prohibited

yields a score that will populate the flow-sheet.

Eclipsys product screen shot(s) reprinted with permission from Eclipsys Corporation. Any other use or

Fig. 1. Dropdown lists for risk assessment items. In this example, the Sensory Perception item presents four selections describing the patient's ability to sense discomfort from pressure. Selecting the option that most closely corresponds with the patient assessment


Eclipsys product screen shot(s) reprinted with permission from Eclipsys Corporation. Any other use or reproduction is strictly prohibited

Fig. 2. In this example, one of the assessment parameters is not entered. The tool was designed to NOT provide a total score unless all of the assessments are documented. This ensures the instruments' integrity for identifying a pressure ulcer risk.

Eclipsys product screen shot(s) reprinted with permission from Eclipsys Corporation. Any other use or reproduction is strictly prohibited

Fig. 3. Electronic assessment tool for CIWA (Clinical Institute Withdrawal Assessment for Alcohol). Each numerical value is associated with a description of the assessment option.

After the workflow analysis has been completed, the next step is to build the tool in the electronic environment. Careful consideration of the risk assessment tool's layout design and precise review of the tool's documentation functions are critical. This process is moderately easy for those risk assessment tools that have been developed using scientific methods (e.g.

Transforming Risk Assessment Tools from Paper to Electronic 287

It is important to note that some risk assessment tools may require sequencing of data element entry in order to maintain the accuracy and validity of the assessment. In fact, many tools require that data elements are gathered in a particular sequence for tabulation or there can be several decision points that must be sequenced appropriately to optimize the accuracy of the risk score for a clinician's decision making and to support practice. For example, when we were creating the electronic version of the Fall Risk Injury Tool, the required steps were to determine that the fall risk was first in the sequence of the assessment. Once the risk for falling was identified, the nurse could then proceed to assess the risk for injury. When both assessments were completed, an overall, total fall and injury risk score was possible to assign for a patient. In this example, the fall risk assessment score

Once an electronic version of a risk assessment tool and its scoring scale are completed, the scores indicate a certain level of risk and, depending on the instrument in use, there may be corresponding actions or interventions necessary to minimize or eliminate a risk, or to prevent specific iatrogenic conditions. An important question to consider at this point is how the user/clinician will be alerted to a condition, a risk level or score, because this function is also developed in the tool's electronic format. The most commonly used options that are developed and built in an electronic risk assessment tool may provide either alerts or advisories to clinicians. It is also possible to develop functions in an electronic risk assessment tool that suggest actions, but since the practices may vary quite a lot between organizations, the development of this function requires that the actions are carefully analyzed and instituted by the clinicians. The alerts can be active (i.e. automatically activated by the IT system and triggered by the predetermined risk levels) or passive (i.e. informing a clinician regarding the pre-determined risk levels) in nature. It is very common that built-in alerts, regardless of their nature, also provide clinicians with care recommendations (Berner, 2009). The electronic risk assessment tool can be developed so that the care recommendations can either be added by the clinician, or if evidence-based recommendations are available, these predetermined interventions can be auto-populated by the information system. As with most risk assessments tools, the electronic versions provide some level of decision support for bedside clinicians. It is important to note that today clinical decision support is a common feature in most clinical information systems. What this means is that based on the availability of data in the EHR (electronic health record), the system can be designed to trigger alerts to numerous conditions to provide evidence-based care recommendations. For example, the presence of allergies to certain medications, conditions like pregnancy or lactation are possible to include in the alerts integrated with the best practice recommendations. Depending on the IT system's functionality, highly advanced alerts and decision support can be designed and created

Many IT systems have a feature that allows the user to review the previous assessment, or portions of it, in relation or as a part of the current assessment. This function goes by many names, with "copy forward" being the most common. The function is useful when designing and building an electronic risk assessment tool that requires data collection as a series of assessments at brief intervals. The benefit of this function is that it allows the clinician to copy the previous assessment forward and to edit the parameters that may have

is a dependency for determining the overall risk of injury.

based on complex utilization of the data from the EHR.

theories and concepts) and statistical testing, because the items in these tools have been clearly stated (i.e. based on commonly accepted definitions) and the scales have been numerically expressed. These same characteristics are important in an electronic medium and thus it is crucial to transform and attach the same definitions to each assessment item including the criterion. It is highly important to underscore that during this process no changes or modifications are allowed on the items or the scoring systems once the instrument has been statistically tested. If a clinician has been using a hard copy of the risk assessment instrument, it is advisable that the layout of the electronic version has resemblance, so that the transition phase from paper to electronic is smooth and a clinician can learn quickly to select scoring options that most closely meet the cited definitions or correctly identifies the risk criteria. We have provided examples from our current Fall Risk Assessment tool (see Figure 4).


Eclipsys product screen shot(s) reprinted with permission from Eclipsys Corporation. Any other use or reproduction is strictly prohibited

Fig. 4. Fall Risk Assessment Tool: as clinical questions get answered, numerical scores populate the electronic version of the instrument and yield total fall risk score.

theories and concepts) and statistical testing, because the items in these tools have been clearly stated (i.e. based on commonly accepted definitions) and the scales have been numerically expressed. These same characteristics are important in an electronic medium and thus it is crucial to transform and attach the same definitions to each assessment item including the criterion. It is highly important to underscore that during this process no changes or modifications are allowed on the items or the scoring systems once the instrument has been statistically tested. If a clinician has been using a hard copy of the risk assessment instrument, it is advisable that the layout of the electronic version has resemblance, so that the transition phase from paper to electronic is smooth and a clinician can learn quickly to select scoring options that most closely meet the cited definitions or correctly identifies the risk criteria. We

have provided examples from our current Fall Risk Assessment tool (see Figure 4).

Eclipsys product screen shot(s) reprinted with permission from Eclipsys Corporation. Any other use or

Fig. 4. Fall Risk Assessment Tool: as clinical questions get answered, numerical scores populate the electronic version of the instrument and yield total fall risk score.

reproduction is strictly prohibited

It is important to note that some risk assessment tools may require sequencing of data element entry in order to maintain the accuracy and validity of the assessment. In fact, many tools require that data elements are gathered in a particular sequence for tabulation or there can be several decision points that must be sequenced appropriately to optimize the accuracy of the risk score for a clinician's decision making and to support practice. For example, when we were creating the electronic version of the Fall Risk Injury Tool, the required steps were to determine that the fall risk was first in the sequence of the assessment. Once the risk for falling was identified, the nurse could then proceed to assess the risk for injury. When both assessments were completed, an overall, total fall and injury risk score was possible to assign for a patient. In this example, the fall risk assessment score is a dependency for determining the overall risk of injury.

Once an electronic version of a risk assessment tool and its scoring scale are completed, the scores indicate a certain level of risk and, depending on the instrument in use, there may be corresponding actions or interventions necessary to minimize or eliminate a risk, or to prevent specific iatrogenic conditions. An important question to consider at this point is how the user/clinician will be alerted to a condition, a risk level or score, because this function is also developed in the tool's electronic format. The most commonly used options that are developed and built in an electronic risk assessment tool may provide either alerts or advisories to clinicians. It is also possible to develop functions in an electronic risk assessment tool that suggest actions, but since the practices may vary quite a lot between organizations, the development of this function requires that the actions are carefully analyzed and instituted by the clinicians. The alerts can be active (i.e. automatically activated by the IT system and triggered by the predetermined risk levels) or passive (i.e. informing a clinician regarding the pre-determined risk levels) in nature. It is very common that built-in alerts, regardless of their nature, also provide clinicians with care recommendations (Berner, 2009). The electronic risk assessment tool can be developed so that the care recommendations can either be added by the clinician, or if evidence-based recommendations are available, these predetermined interventions can be auto-populated by the information system. As with most risk assessments tools, the electronic versions provide some level of decision support for bedside clinicians. It is important to note that today clinical decision support is a common feature in most clinical information systems. What this means is that based on the availability of data in the EHR (electronic health record), the system can be designed to trigger alerts to numerous conditions to provide evidence-based care recommendations. For example, the presence of allergies to certain medications, conditions like pregnancy or lactation are possible to include in the alerts integrated with the best practice recommendations. Depending on the IT system's functionality, highly advanced alerts and decision support can be designed and created based on complex utilization of the data from the EHR.

Many IT systems have a feature that allows the user to review the previous assessment, or portions of it, in relation or as a part of the current assessment. This function goes by many names, with "copy forward" being the most common. The function is useful when designing and building an electronic risk assessment tool that requires data collection as a series of assessments at brief intervals. The benefit of this function is that it allows the clinician to copy the previous assessment forward and to edit the parameters that may have

Transforming Risk Assessment Tools from Paper to Electronic 289

Eclipsys product screen shot(s) reprinted with permission from Eclipsys Corporation. Any other use or

Fig. 5. In this figure, the top portion of the flow-sheet contains the assessment performed by

Several testing phases are necessary to ensure accuracy of the electronic version of the risk assessment tool and its impact on decision-making after its configuration is available in the EHR test system. The electronic tool first undergoes **unit testing** (i.e. testing of the different sections and scoring system of the tool) to ensure that the developed assessment instrument and documentation perform as desired. During this phase the IT system analysts and staff nurses perform the testing, preferably using a test system and 'non-clinical' environment. The focus of the team is to enter data into each of the fields in the electronic instrument, and follow the same steps and approach as they would have when using a paper tool. As the team members continue to add data to the tools' fields, the clinicians are also asked to evaluate the function of the tool and determine whether it performs accurate risk-score calculations and yields the correct score based on the entered observations (cf. Weiner et al. 2007). This testing phase is very important, because the goal is that the electronic risk assessment tool accurately assigns the appropriate level of risk to inform the clinician, as it would in a paper format.

the Registered Nurse. The interventions below the assessment allow both RN and

reproduction is strictly prohibited

**2.4 Testing & signoff** 

unlicensed staff to document interventions.

changed. Most of all, this function assists with clinical efficiency and tasks requiring fast paced and short time-interval execution. However, monitoring for improper use of this particular function is recommended, because busy clinicians may continue using the original assessment without editing changed parameters in order to save time. Unlike the 'copy' and 'paste' function, the copy forward function leaves an electronic audit trail which is commonly used to monitor the use of this function. (Mangalmurti et al, 2010)**.** 

#### **2.3 Access and security**

Unlike a paper record, the use and review of clinical information in electronic systems can be protected by controlling users' access, availability, or activity to portions of the electronic health record. However, privacy of patient information balanced with the clinicians' scopes of practice are important to take into consideration when creating appropriate levels of security in electronic health records. Access to the system alone, always requires a password. Once within the system, the type and amount of information that can be seen, entered or manipulated, is determined by a clinician's role in patient care. Most health care organizations use the 'minimum necessary' standard for access. As the phrase implies, access is granted for the least amount of information needed to perform in a clinician's role. For example, the access to enter or edit data related to a risk assessment today is usually limited to the Registered Nurse. A patient's health assessment is generally regarded as one of the most important tasks and core domain of nursing, therefore, most risk assessments are performed by Registered Nurses. The access to view and use the electronic risk assessment tool may be limited to licensed clinicians who are members of the care team. Depending on the defined details of the designed security in the clinical information system, some portions of the risk assessment tool may allow access to unlicensed caregivers. It is important to consider, whether there are health assessment data elements that are within the scope of practice of an unlicensed caregiver and how those fields may be available for data entry with appropriate security settings. The issues of access and security are important to consider early on, so that the electronic risk assessment tools allow for contribution by all members of the care team.

The following example demonstrates the care team's role in the risk assessment, its execution and documentation. In the conversion of the Braden Scale and Fall Risk Injury assessments tools from paper to electronic, the security configuration on the electronic version of the tool allowed only RNs to document the risk assessments. Once the assessment has been completed, the RN would select interventions to prevent either of these outcomes (i.e. pressure ulcers; falls and injuries). However, the interventions (e.g. skin care, repositioning; activation of bed alarms, hourly rounding) are often performed by an unlicensed staff member, therefore, providing limited access and ability to document the intervention is an important part of the care process. In both electronic risk assessment tool examples, the right to enter an assessment was only granted to Registered Nurses, and the portion of the tool with intervention selections was available for all care team members to document. (See: Figure 5) If the clinical information system does not have the functionality to restrict access to unlicensed personnel, another option is a hospital policy that clearly defines and prescribes the rights of each team member. However, this approach does require periodic audits to ensure appropriate use.

changed. Most of all, this function assists with clinical efficiency and tasks requiring fast paced and short time-interval execution. However, monitoring for improper use of this particular function is recommended, because busy clinicians may continue using the original assessment without editing changed parameters in order to save time. Unlike the 'copy' and 'paste' function, the copy forward function leaves an electronic audit trail which

Unlike a paper record, the use and review of clinical information in electronic systems can be protected by controlling users' access, availability, or activity to portions of the electronic health record. However, privacy of patient information balanced with the clinicians' scopes of practice are important to take into consideration when creating appropriate levels of security in electronic health records. Access to the system alone, always requires a password. Once within the system, the type and amount of information that can be seen, entered or manipulated, is determined by a clinician's role in patient care. Most health care organizations use the 'minimum necessary' standard for access. As the phrase implies, access is granted for the least amount of information needed to perform in a clinician's role. For example, the access to enter or edit data related to a risk assessment today is usually limited to the Registered Nurse. A patient's health assessment is generally regarded as one of the most important tasks and core domain of nursing, therefore, most risk assessments are performed by Registered Nurses. The access to view and use the electronic risk assessment tool may be limited to licensed clinicians who are members of the care team. Depending on the defined details of the designed security in the clinical information system, some portions of the risk assessment tool may allow access to unlicensed caregivers. It is important to consider, whether there are health assessment data elements that are within the scope of practice of an unlicensed caregiver and how those fields may be available for data entry with appropriate security settings. The issues of access and security are important to consider early on, so that the electronic risk assessment tools allow for contribution by all

The following example demonstrates the care team's role in the risk assessment, its execution and documentation. In the conversion of the Braden Scale and Fall Risk Injury assessments tools from paper to electronic, the security configuration on the electronic version of the tool allowed only RNs to document the risk assessments. Once the assessment has been completed, the RN would select interventions to prevent either of these outcomes (i.e. pressure ulcers; falls and injuries). However, the interventions (e.g. skin care, repositioning; activation of bed alarms, hourly rounding) are often performed by an unlicensed staff member, therefore, providing limited access and ability to document the intervention is an important part of the care process. In both electronic risk assessment tool examples, the right to enter an assessment was only granted to Registered Nurses, and the portion of the tool with intervention selections was available for all care team members to document. (See: Figure 5) If the clinical information system does not have the functionality to restrict access to unlicensed personnel, another option is a hospital policy that clearly defines and prescribes the rights of each team member. However, this approach does

is commonly used to monitor the use of this function. (Mangalmurti et al, 2010)**.** 

**2.3 Access and security** 

members of the care team.

require periodic audits to ensure appropriate use.


Eclipsys product screen shot(s) reprinted with permission from Eclipsys Corporation. Any other use or reproduction is strictly prohibited

Fig. 5. In this figure, the top portion of the flow-sheet contains the assessment performed by the Registered Nurse. The interventions below the assessment allow both RN and unlicensed staff to document interventions.

#### **2.4 Testing & signoff**

Several testing phases are necessary to ensure accuracy of the electronic version of the risk assessment tool and its impact on decision-making after its configuration is available in the EHR test system. The electronic tool first undergoes **unit testing** (i.e. testing of the different sections and scoring system of the tool) to ensure that the developed assessment instrument and documentation perform as desired. During this phase the IT system analysts and staff nurses perform the testing, preferably using a test system and 'non-clinical' environment. The focus of the team is to enter data into each of the fields in the electronic instrument, and follow the same steps and approach as they would have when using a paper tool. As the team members continue to add data to the tools' fields, the clinicians are also asked to evaluate the function of the tool and determine whether it performs accurate risk-score calculations and yields the correct score based on the entered observations (cf. Weiner et al. 2007). This testing phase is very important, because the goal is that the electronic risk assessment tool accurately assigns the appropriate level of risk to inform the clinician, as it would in a paper format.

Transforming Risk Assessment Tools from Paper to Electronic 291

assessment tool must be selected and opened prior to use. Users will thus need to know how to access the tool and then how to enter the risk assessment scores (i.e. data elements of the assessment). Depending on the system's design, some fields of the tool can be automatically completed based on information (i.e. known data elements) from the patient's record. Such information as the patient's age, and gender are common data elements that are already existing elsewhere in the patient record. These can be pulled into the respective fields of a risk assessment tool. However, the IT system may require manual entry of all information (i.e. data elements) regarding the patient and the risk assessment. Similarly with the instrument's assessment scores and calculations, some systems are able to automatically present the risk level (i.e. outcome) or 'score' without user action. Conversely, there are systems that require a user 'action' after the risk assessment scores have been entered in order to obtain the overall risk level of a patient. These are just some examples of the important objectives necessary to consider and

Eclipsys product screen shot(s) reprinted with permission from Eclipsys Corporation. Any other use or

Fig. 6. In this example, a nursing department displays key patient information for all members of the clinical staff. Three columns display the most recent Fall Risk and Braden

scores. It also highlights all of the patients on Fall Risk precautions.

understand when developing the training curriculum.

reproduction is strictly prohibited

After the unit testing has been completed and the team is satisfied with the tools functions and accuracy, the next phase is to conduct **a system and/or integrated testing**. Different types of risk assessments are, in fact, only a small part of the EHR and patient's record, whether these are paper or electronic versions. Yet in both instances, risk assessments are an integral part of the overall quality care and safe treatment of patients. It is possible to claim that all the risk assessment tools provide critical information regarding assessment and thus most elements/fields of the tool are necessary to disseminate, including the assessment's outcome. It is important to note that the assessment results may be necessary as a resourceplanning report for department and staffing managers. During **the system testing phase**, the goal is to evaluate the tool's integration into available decision-support mechanisms, reports and on-line views. Ideally, the electronic medium allows instantaneous dissemination of any risk assessment information needed by the entire healthcare team. Today many systems provide a whiteboard or tracking board feature that enables display of an overview of a unit or department (See Figure 6). Outcomes of many risk assessments are often tracked and summarized in these departmental views.

Once testing of the electronic tool is complete, many organizations will perform a Failure Mode Effects and Analysis (FMEA). FMEAs are an engineering technique used to define, identify, prioritize and eliminate known or potential failures, problems and errors from the system, design, process or service before they reach the customer. (McDermott et al, 1996)

An FMEA examines a process associated with great risk and identifies ways in which injury or unexpected death among patients might be averted. The FMEA team members identify steps in the process that could fail (termed "failure modes"), agree upon scoring criteria to determine the probability of failure and the expected severity of its effect, determine which failure modes could be eliminated or controlled by an intervention, and determine how the process might be changed, before a patient is injured (Kenney-Weeks et al., 2004).

#### **2.5 Training**

Patient health risk assessments are part of standard care and best practices and many clinicians are accustomed using different types of risk assessment tools in their daily patient care. If the electronic risk assessment tool is one that is already familiar to clinical staff, training need only focus on the use of the tool in a new, electronic environment. However, if a tool and its content (e.g. scoring scale, criteria for scoring, calculations) are new to the clinicians, this will require focused and carefully planned in-service education prior to the computer training. It is important that the clinicians comprehend the tool's use, how to interpret the results and most of all to feel confident and comfortable using the tool for a risk assessment. It is critical to identify any non-computer related content that needs to be included in the training and understood before using the tool in patient care. The content of the training regarding the use of the tool in an electronic environment is taught by the computer training staff, however it is recommended in the literature (Brettle, 2003) that any practice related information should be taught by a Registered Nurse at a separate time. The learning process is quicker and the outcomes (e.g. accuracy of the use of the tool, compliance) are better when the PC instruction is limited to use of the information system and the electronic tools. For example, in many IT systems the risk

After the unit testing has been completed and the team is satisfied with the tools functions and accuracy, the next phase is to conduct **a system and/or integrated testing**. Different types of risk assessments are, in fact, only a small part of the EHR and patient's record, whether these are paper or electronic versions. Yet in both instances, risk assessments are an integral part of the overall quality care and safe treatment of patients. It is possible to claim that all the risk assessment tools provide critical information regarding assessment and thus most elements/fields of the tool are necessary to disseminate, including the assessment's outcome. It is important to note that the assessment results may be necessary as a resourceplanning report for department and staffing managers. During **the system testing phase**, the goal is to evaluate the tool's integration into available decision-support mechanisms, reports and on-line views. Ideally, the electronic medium allows instantaneous dissemination of any risk assessment information needed by the entire healthcare team. Today many systems provide a whiteboard or tracking board feature that enables display of an overview of a unit or department (See Figure 6). Outcomes of many risk assessments are

Once testing of the electronic tool is complete, many organizations will perform a Failure Mode Effects and Analysis (FMEA). FMEAs are an engineering technique used to define, identify, prioritize and eliminate known or potential failures, problems and errors from the system, design, process or service before they reach the customer. (McDermott et al, 1996)

An FMEA examines a process associated with great risk and identifies ways in which injury or unexpected death among patients might be averted. The FMEA team members identify steps in the process that could fail (termed "failure modes"), agree upon scoring criteria to determine the probability of failure and the expected severity of its effect, determine which failure modes could be eliminated or controlled by an intervention, and determine how the

Patient health risk assessments are part of standard care and best practices and many clinicians are accustomed using different types of risk assessment tools in their daily patient care. If the electronic risk assessment tool is one that is already familiar to clinical staff, training need only focus on the use of the tool in a new, electronic environment. However, if a tool and its content (e.g. scoring scale, criteria for scoring, calculations) are new to the clinicians, this will require focused and carefully planned in-service education prior to the computer training. It is important that the clinicians comprehend the tool's use, how to interpret the results and most of all to feel confident and comfortable using the tool for a risk assessment. It is critical to identify any non-computer related content that needs to be included in the training and understood before using the tool in patient care. The content of the training regarding the use of the tool in an electronic environment is taught by the computer training staff, however it is recommended in the literature (Brettle, 2003) that any practice related information should be taught by a Registered Nurse at a separate time. The learning process is quicker and the outcomes (e.g. accuracy of the use of the tool, compliance) are better when the PC instruction is limited to use of the information system and the electronic tools. For example, in many IT systems the risk

process might be changed, before a patient is injured (Kenney-Weeks et al., 2004).

often tracked and summarized in these departmental views.

**2.5 Training** 

assessment tool must be selected and opened prior to use. Users will thus need to know how to access the tool and then how to enter the risk assessment scores (i.e. data elements of the assessment). Depending on the system's design, some fields of the tool can be automatically completed based on information (i.e. known data elements) from the patient's record. Such information as the patient's age, and gender are common data elements that are already existing elsewhere in the patient record. These can be pulled into the respective fields of a risk assessment tool. However, the IT system may require manual entry of all information (i.e. data elements) regarding the patient and the risk assessment. Similarly with the instrument's assessment scores and calculations, some systems are able to automatically present the risk level (i.e. outcome) or 'score' without user action. Conversely, there are systems that require a user 'action' after the risk assessment scores have been entered in order to obtain the overall risk level of a patient. These are just some examples of the important objectives necessary to consider and understand when developing the training curriculum.


Eclipsys product screen shot(s) reprinted with permission from Eclipsys Corporation. Any other use or reproduction is strictly prohibited

Fig. 6. In this example, a nursing department displays key patient information for all members of the clinical staff. Three columns display the most recent Fall Risk and Braden scores. It also highlights all of the patients on Fall Risk precautions.

Transforming Risk Assessment Tools from Paper to Electronic 293

staff needs to be planned beforehand, because the 'go-live' phase may require from 3 to 30 persons or more. The timing of planning of support staff is vital, because the support team members have to receive extensive training on the computerized assessment tool. They need to be familiar with the intricacies of the tool, its use and most of all be able to assist staff/clinicians who are using the tool for the first time. Our experience has demonstrated that one IT support team member can cover 1 or 2 departments at once in these types of 'go-

Once the risk assessment tool is being used by clinicians/staff throughout the organization, the progress of its use can be easily measured. Most clinical IT system databases are developed with the intent of retrieving data about care processes. The processes of Risk Assessment are important to monitor especially when a tool has been transformed from paper to an electronic version. The reports regarding the risk assessment tools can be very helpful, for example, for department (nursing) managers. The focus of the reports (i.e. data elements) for monitoring needs to be determined by the department leadership in collaboration with the report writers. Depending on the focus of monitoring, reports may include aggregate data, allowing managers to assess the resources necessary to mitigate overall risk in the department. Reports may also focus on compliance of use of the risk assessment tool. If the risk assessment is associated with a regular frequency (i.e. daily, every shift), then reporting allows (nurse) managers to check and audit the assessments done by all staff. This helps to identify staff/clinicians who are still reluctant to try the changed process. Publishing the reports of utilization rates of departments and staff members serves as motivation to use the tool regularly. The trends shown by the reports can

also be used as discussion points and reveal barriers to regular use of the tool.

valuable when striving to meet certain patient safety and care goals.

Aggregated reports can provide researchers in clinical settings support to the tool's validity, reliability and feasibility, but also can demonstrate the tool's effectiveness and impact on patient outcomes. For example, comparison of departmental utilization rates of a Fall Injury Risk Tools to the actual number of falls in those same departments may illuminate opportunities for improvement. It is possible to claim that, unlike paper- based tools, electronic risk assessment tools provide vital advantages in reporting: having information readily available allows for more timely data analysis and trending, which are extremely

This chapter introduces the ongoing need to make risk assessment tools available in an electronic medium. Advances in patient care and the clinicians' need for full availability of patient information, require the conversion of risk assessment tools from paper to electronic. This chapter outlines a process for achieving this conversion. The process described is one organization's method for conversion, although most organizations use a similar process for preparing to move any clinical documentation to an electronic environment. The tools that have been converted are those that this organization has chosen to use to assess health risks

live' processes.

**2.7 Reporting** 

**3. Conclusions** 

on hospitalized patients.

The timing of the training is critical in relation to the availability of the tool for its users. Depending on the assessment tool and its complexity (e.g. scoring scale, assessment criteria, number of assessment items, calculations), the staffs training period should be as close to the 'go-live' date as possible. The literature shows (Brettle, 2003) that training for any computer application is best learned with continued reinforced use of the application. Training too early may affect retention of the learned skills and training material. Today, many organizations maintain two copies of the developed IT system for training purposes so that one of these can be used as a practice environment. This identical, safe training environment allows users to practice the use of a new risk assessment tool, its functions and documentation and gain a learning experience that is possible to transfer directly to the clinical setting. Continuous evaluation of the training methods is important, because the goal is to allow nurses to learn, gain and maintain competency in the use of new electronic tools.

The literature shows (Brettle, 2003) that there are numerous teaching methods and approaches to training that should be considered. Most health care organizations have a computer training staff who can assist with curriculum development. The classes and training regarding the use of electronic risk assessment tools often employ lectures combined with 'hands-on' use of the application. These can be supplemented and sometimes replaced by electronic learning or (also known e-learning) options. The literature shows (Brettle, 2003) that e-learning is an efficient way to train staff when the content is not too complex. The benefit of this option is that it allows the user to learn the material at any time, day, or location. However, the downside of using e-learn for computer classes, is that a user does not have an opportunity to interact with an instructor/computer training staff and thus may be unable to receive clarification, on questions related to the tool.

#### **2.6 Go-live & support**

As with any change, planning the implementation and execution of the electronic assessment tool are critical to the success. To prepare the clinicians, units and organizations for a change in documentation, a strategy needs to be carefully developed. The strategy should obviously include, at least the date and time for 'go-live', as well as the approach that will be used during this phase of the transformation process. Depending on the clinical IT system being used, some 'go-lives' can be incremental (i.e. the tool is introduced to a few departments first, and then gradually other departments of an organization are added) or the 'go-live' can be for the entire organization at once (i.e. 'Big Bang' approach). For the 'golive' of most risk assessment tools, a 'Big Bang' is the simplest and quickest approach. The tools are used by clinicians for only small portions of the work day, however, it is critical from a patient safety perspective that the risk assessments are conducted similarly within an organization and that the transition phase does not cause any gaps in the documentation of these assessments. It is also more economical to make the electronic version available organization-wide, all at once because this approach reduces the time period of support required by the users.

When an IT system change is introduced, the availability of on-site support needs to be revisited and provided. Depending on the size of the organization, the number of users, and the complexity of the electronic risk assessment tool, an estimate regarding extra IT support staff needs to be planned beforehand, because the 'go-live' phase may require from 3 to 30 persons or more. The timing of planning of support staff is vital, because the support team members have to receive extensive training on the computerized assessment tool. They need to be familiar with the intricacies of the tool, its use and most of all be able to assist staff/clinicians who are using the tool for the first time. Our experience has demonstrated that one IT support team member can cover 1 or 2 departments at once in these types of 'golive' processes.

#### **2.7 Reporting**

292 Novel Approaches and Their Applications in Risk Assessment

The timing of the training is critical in relation to the availability of the tool for its users. Depending on the assessment tool and its complexity (e.g. scoring scale, assessment criteria, number of assessment items, calculations), the staffs training period should be as close to the 'go-live' date as possible. The literature shows (Brettle, 2003) that training for any computer application is best learned with continued reinforced use of the application. Training too early may affect retention of the learned skills and training material. Today, many organizations maintain two copies of the developed IT system for training purposes so that one of these can be used as a practice environment. This identical, safe training environment allows users to practice the use of a new risk assessment tool, its functions and documentation and gain a learning experience that is possible to transfer directly to the clinical setting. Continuous evaluation of the training methods is important, because the goal is to allow nurses to learn, gain and maintain competency in the use of new electronic

The literature shows (Brettle, 2003) that there are numerous teaching methods and approaches to training that should be considered. Most health care organizations have a computer training staff who can assist with curriculum development. The classes and training regarding the use of electronic risk assessment tools often employ lectures combined with 'hands-on' use of the application. These can be supplemented and sometimes replaced by electronic learning or (also known e-learning) options. The literature shows (Brettle, 2003) that e-learning is an efficient way to train staff when the content is not too complex. The benefit of this option is that it allows the user to learn the material at any time, day, or location. However, the downside of using e-learn for computer classes, is that a user does not have an opportunity to interact with an instructor/computer training staff and

As with any change, planning the implementation and execution of the electronic assessment tool are critical to the success. To prepare the clinicians, units and organizations for a change in documentation, a strategy needs to be carefully developed. The strategy should obviously include, at least the date and time for 'go-live', as well as the approach that will be used during this phase of the transformation process. Depending on the clinical IT system being used, some 'go-lives' can be incremental (i.e. the tool is introduced to a few departments first, and then gradually other departments of an organization are added) or the 'go-live' can be for the entire organization at once (i.e. 'Big Bang' approach). For the 'golive' of most risk assessment tools, a 'Big Bang' is the simplest and quickest approach. The tools are used by clinicians for only small portions of the work day, however, it is critical from a patient safety perspective that the risk assessments are conducted similarly within an organization and that the transition phase does not cause any gaps in the documentation of these assessments. It is also more economical to make the electronic version available organization-wide, all at once because this approach reduces the time period of support

When an IT system change is introduced, the availability of on-site support needs to be revisited and provided. Depending on the size of the organization, the number of users, and the complexity of the electronic risk assessment tool, an estimate regarding extra IT support

thus may be unable to receive clarification, on questions related to the tool.

tools.

**2.6 Go-live & support** 

required by the users.

Once the risk assessment tool is being used by clinicians/staff throughout the organization, the progress of its use can be easily measured. Most clinical IT system databases are developed with the intent of retrieving data about care processes. The processes of Risk Assessment are important to monitor especially when a tool has been transformed from paper to an electronic version. The reports regarding the risk assessment tools can be very helpful, for example, for department (nursing) managers. The focus of the reports (i.e. data elements) for monitoring needs to be determined by the department leadership in collaboration with the report writers. Depending on the focus of monitoring, reports may include aggregate data, allowing managers to assess the resources necessary to mitigate overall risk in the department. Reports may also focus on compliance of use of the risk assessment tool. If the risk assessment is associated with a regular frequency (i.e. daily, every shift), then reporting allows (nurse) managers to check and audit the assessments done by all staff. This helps to identify staff/clinicians who are still reluctant to try the changed process. Publishing the reports of utilization rates of departments and staff members serves as motivation to use the tool regularly. The trends shown by the reports can also be used as discussion points and reveal barriers to regular use of the tool.

Aggregated reports can provide researchers in clinical settings support to the tool's validity, reliability and feasibility, but also can demonstrate the tool's effectiveness and impact on patient outcomes. For example, comparison of departmental utilization rates of a Fall Injury Risk Tools to the actual number of falls in those same departments may illuminate opportunities for improvement. It is possible to claim that, unlike paper- based tools, electronic risk assessment tools provide vital advantages in reporting: having information readily available allows for more timely data analysis and trending, which are extremely valuable when striving to meet certain patient safety and care goals.
