2. Methods

#### 2.1. Medical Office Survey on Patient Safety Culture

The Medical Office Survey on Patient Safety Culture (MOSPSC) is a self-administered tool, which was developed by the Agency for Healthcare Research and Quality (AHRQ) in 2007 [8], and is designed specifically for outpatient medical office providers and other staff and asks for their opinions about the culture of patient safety and health care quality in their medical offices. Although in Portugal the health system is completely different than in the United States, we considered that the primary care environment and culture are similar, which lead us to test its use.

This survey has 38 items grouped into 10 composites and includes questions that ask respondents about problems related to exchange information with other settings and about access to care. Respondents are also asked to rate their medical office in five areas of health care quality


(patient centered, effective, timely, efficient, and equitable) and to provide an overall rating on patient safety (Table 1).

Table 1. MOSPSC composites and items.

1. Introduction

14 Vignettes in Patient Safety - Volume 4

2. Methods

us to test its use.

Health care is vulnerable to error, and so all health care environments and professionals are involved in complex care processes. Since the IOM report [1], almost all countries and health care organizations are attending to Patient Safety issues. In more recent years, the European Council launched a recommendation [2] that shows the importance of establishing patient safety culture in all health care settings. We can read in this recommendation that a poor patient safety represents both a severe public health problem and a high economic burden on limited health resources. A large proportion of adverse events, both in the hospital sector and in primary care,

Before implementing patient safety programs, health care staff must understand their safety culture [3]. Quantitative instruments designed to assess safety culture have been developed, and a few review articles have been published, which allows a more comprehensive way of implementing models of safety culture [4]. Measuring health care safety culture enables us to identify improvements, safety behaviors, and outcomes for both patients and staff. These

Much has been done in hospital environment, and more recently, primary care has also been in the sights. A few review articles were published allowing researchers and primary care staff to

With the publication of the National Patient Safety Plan (2015–2020), the Portuguese Directory of Health along with the Portuguese Hospital Association carried out patient safety culture assessment either in hospitals or in primary care. It was published as a national standard, and every 2 years, patient safety culture is assessed either in primary care or in hospitals nationwide.

The purpose of this study was to translate, adapt, validate, and analyze the reliability and validity of the Portuguese version of the Medical Office Survey on Patient Safety Culture.

The Medical Office Survey on Patient Safety Culture (MOSPSC) is a self-administered tool, which was developed by the Agency for Healthcare Research and Quality (AHRQ) in 2007 [8], and is designed specifically for outpatient medical office providers and other staff and asks for their opinions about the culture of patient safety and health care quality in their medical offices. Although in Portugal the health system is completely different than in the United States, we considered that the primary care environment and culture are similar, which lead

This survey has 38 items grouped into 10 composites and includes questions that ask respondents about problems related to exchange information with other settings and about access to care. Respondents are also asked to rate their medical office in five areas of health care quality

are preventable with systemic factors appearing to account for a majority of them.

instruments should also serve as decision making tools, especially for managers.

take robust decisions on tools to assess patient safety culture [5–7].

2.1. Medical Office Survey on Patient Safety Culture

According to the MOSPSC author's [8], patient safety culture composites and its definitions are:


providers and staff talk openly about office problems and how to prevent errors from happening.

Content validity and semantic analysis were undertaken by six experts chosen from the primary care sector and knowledge on this topic and with research experience (Step 5).

Patient Safety Culture in Portuguese Primary Care: Validation of the Portuguese Version of the Medical Office…

http://dx.doi.org/10.5772/intechopen.80035

17

The pretest was applied (Step 6), which was aimed at assessing whether the MOSPSC was understandable to a larger number of people in the target population. The last version of the MOSPSC was then administered in Web-based format, and we used all recommendations

The Portuguese Directory of Health published a national standard that requires patient safety culture assessment in primary care units (PCUs) nationwide (52 PCUs) every 2 years. A personalized link was sent to all PCUs, where a focal point was in charge of facilitating the administration of the survey. In order to track and maximize response rates, a link was sent to each office PCU. We sent another link so that the focal point could check response rates along

Our goal was to assess the validity and reliability of the Portuguese version of the MOSPSC, by verifying if the 10 patient-safety culture composites were appropriate for the Portuguese population. The R software was used for statistical analysis, and the negatively worded items

Descriptive statistics were used to examine response variability and missing data. To identify and eliminate those items with missing data, an individual descriptive item analysis was performed. A missing-value analysis was performed to verify if it was necessary to remove surveys from the data set. Every survey with missing values was removed, and surveys with more than 1 response in the option "not applicable" were removed. For the remaining surveys with only one answer in option "not applicable," it was replaced by the middle category in a five-point Likert scale. An empirical rule of 10 respondents per patient safety culture item in a

A reliability analysis (internal consistency) was performed using Cronbach's α, where it indicates the extent to which surveys items can be treated as a single latent construct. Values >0.7 reliability is considered adequate for a survey instrument [12], although some authors consider >0.6 adequate [13]. For the entire survey, Cronbach's α should be at least 0.9 [12]. However, the validity of this measure has been questioned, and several authors have suggested alternative measures. In this study, we also used the average inter-item correlation (AIIC), which is independent of the number of items and sample size. This measure evaluates how items within a composite correlate, i.e., there is evidence that the items are measuring the same underlying composite. A rule-of-thumb is that AIIC should be between 0.15 and 0.5 [14].

An exploratory factor analysis (EFA) was performed. EFA is a cluster of common methods used to explore the underlying pattern of relationships among multiple observed variables. EFA is useful for assessing the dimensionality of questionnaire scales that measure underlying latent variables. Researchers use EFA to hypothesize and, later, confirm, through replication or confirmatory factor analysis (CFA), the model that gave rise to the interrelationships among the scale's variables. EFA for ordinal data, a benefit over conventional criteria, where the Pearson correlation

the administration period, which occurred from March 16 till April 30, 2017.

survey with 38 items means that at least 380 completed surveys were needed.

from AHRQ [11] to publicize and promote the survey.

were reverse-scored and they are denoted by R letter.

2.3. Statistical analysis


Since the publication of the National Patient Safety Plan (2015–2020), the Portuguese Directory of Health along with the Portuguese Hospital Association carried out patient safety culture assessment either in hospitals or in primary care. For this purpose, the MOSPSC was the chosen tool because [8]:


The European Society for Quality and Safety in Family Practice (EQuiP) and the World Family Doctors. Caring for People (WONCA Europe) [11] conducted a study to spread the MOSPSC among EQuiP delegates, explore their views and opinions on the MOSPSC, and explore with them the feasibility of the MOSPSC among European countries. Nineteen countries were involved, and 63% of respondents find it would be interesting to use MOSPSC.

#### 2.2. Translation and cultural adaptation process

Immediately after author's permission for MOSPSC use, the survey was translated from English to Portuguese (T1) and backward (T2) by two independent translators, native speakers of Portuguese, and bilingual in English/Portuguese, experienced in this method and knowledgeable about the research objective (Step 1). The two versions (T1 and T2) were compared with the original version of the MOSPSC (Step 2). Back translation by two independent translators (R-T1 and R-T2) was carried out by bilingual native English, who were unfamiliar with the original version of tool and not knowledgeable about the study objectives (Step 3). Discrepancies were assessed, and the cross-cultural adaptations were undertaken (Step 4). Content validity and semantic analysis were undertaken by six experts chosen from the primary care sector and knowledge on this topic and with research experience (Step 5).

The pretest was applied (Step 6), which was aimed at assessing whether the MOSPSC was understandable to a larger number of people in the target population. The last version of the MOSPSC was then administered in Web-based format, and we used all recommendations from AHRQ [11] to publicize and promote the survey.

The Portuguese Directory of Health published a national standard that requires patient safety culture assessment in primary care units (PCUs) nationwide (52 PCUs) every 2 years. A personalized link was sent to all PCUs, where a focal point was in charge of facilitating the administration of the survey. In order to track and maximize response rates, a link was sent to each office PCU. We sent another link so that the focal point could check response rates along the administration period, which occurred from March 16 till April 30, 2017.

#### 2.3. Statistical analysis

providers and staff talk openly about office problems and how to prevent errors from

8. Communication Openness—the extent to which providers in the office are open to staff ideas about how to improve office processes, and staff are encouraged to express alterna-

9. Office Processes and Standardization—the extent to which the office is organized, has an effective workflow, has standardized processes for completing tasks, and has good pro-

10. Work Pressure and Pace—the extent to which there are enough staff and providers to

Since the publication of the National Patient Safety Plan (2015–2020), the Portuguese Directory of Health along with the Portuguese Hospital Association carried out patient safety culture assessment either in hospitals or in primary care. For this purpose, the MOSPSC was the

tive viewpoints and do not find it difficult to voice disagreement.

handle the patient load, and the office work pace is not hectic.

cedures for checking the accuracy of work performed.

• it raises provider and staff awareness about patient safety;

• it examines trends in patient safety culture change over time;

• it conducts comparisons within and across organizations;

2.2. Translation and cultural adaptation process

• it identifies strengths and areas for patient safety culture improvement;

patient safety culture at primary health care services in Europe.

involved, and 63% of respondents find it would be interesting to use MOSPSC.

• it evaluates the cultural impact of patient safety initiatives and interventions;

• it has been used in several countries in Europe (which makes benchmark possible) [9, 10], and the results of the LINEUS study [9] show that it is useful and applicable to assess

The European Society for Quality and Safety in Family Practice (EQuiP) and the World Family Doctors. Caring for People (WONCA Europe) [11] conducted a study to spread the MOSPSC among EQuiP delegates, explore their views and opinions on the MOSPSC, and explore with them the feasibility of the MOSPSC among European countries. Nineteen countries were

Immediately after author's permission for MOSPSC use, the survey was translated from English to Portuguese (T1) and backward (T2) by two independent translators, native speakers of Portuguese, and bilingual in English/Portuguese, experienced in this method and knowledgeable about the research objective (Step 1). The two versions (T1 and T2) were compared with the original version of the MOSPSC (Step 2). Back translation by two independent translators (R-T1 and R-T2) was carried out by bilingual native English, who were unfamiliar with the original version of tool and not knowledgeable about the study objectives (Step 3). Discrepancies were assessed, and the cross-cultural adaptations were undertaken (Step 4).

• it assesses the current status of patient safety culture;

happening.

16 Vignettes in Patient Safety - Volume 4

chosen tool because [8]:

Our goal was to assess the validity and reliability of the Portuguese version of the MOSPSC, by verifying if the 10 patient-safety culture composites were appropriate for the Portuguese population. The R software was used for statistical analysis, and the negatively worded items were reverse-scored and they are denoted by R letter.

Descriptive statistics were used to examine response variability and missing data. To identify and eliminate those items with missing data, an individual descriptive item analysis was performed. A missing-value analysis was performed to verify if it was necessary to remove surveys from the data set. Every survey with missing values was removed, and surveys with more than 1 response in the option "not applicable" were removed. For the remaining surveys with only one answer in option "not applicable," it was replaced by the middle category in a five-point Likert scale. An empirical rule of 10 respondents per patient safety culture item in a survey with 38 items means that at least 380 completed surveys were needed.

A reliability analysis (internal consistency) was performed using Cronbach's α, where it indicates the extent to which surveys items can be treated as a single latent construct. Values >0.7 reliability is considered adequate for a survey instrument [12], although some authors consider >0.6 adequate [13]. For the entire survey, Cronbach's α should be at least 0.9 [12]. However, the validity of this measure has been questioned, and several authors have suggested alternative measures. In this study, we also used the average inter-item correlation (AIIC), which is independent of the number of items and sample size. This measure evaluates how items within a composite correlate, i.e., there is evidence that the items are measuring the same underlying composite. A rule-of-thumb is that AIIC should be between 0.15 and 0.5 [14].

An exploratory factor analysis (EFA) was performed. EFA is a cluster of common methods used to explore the underlying pattern of relationships among multiple observed variables. EFA is useful for assessing the dimensionality of questionnaire scales that measure underlying latent variables. Researchers use EFA to hypothesize and, later, confirm, through replication or confirmatory factor analysis (CFA), the model that gave rise to the interrelationships among the scale's variables. EFA for ordinal data, a benefit over conventional criteria, where the Pearson correlation matrix is used. Pearson correlations assume that data have been measured on, at least, an equal interval scale, and a linear relationship exists between the variables. These assumptions are typically violated in the case of variables measured using ordinal rating scales. Pearson correlations have been found to underestimate the strength of relationships between ordinal items.

EFA is useful for assessing the dimensionality of survey scales that measure underlying latent variables. This factor analysis gives an indication of the number of factors that the survey appears to measure of its intended subject. In this way, through EFA, we can investigate if the Portuguese data will produce different factors from the American structure.

Since the data are ordinal, it was used a polychoric correlation matrix for EFA analysis and a Varimax rotation. To decide on the number of factors, it was used a parallel analysis [15, 16]. Items with a factor loading lower than 0.4 on all factors were excluded. Libraries psych and polycor from R were used [17, 18].

We used confirmatory factor analysis (CFA) for ordinal data to compare the Portuguese sample factor structure to the factor structure reported for the original HSOPSC. CFA for ordinal data will use diagonally weighted least squares (DWLS) to estimate the model parameters, but it will use the full weight matrix to compute robust standard errors and a mean- and variance-adjusted test statistic. We used the goodness-of-fit index (GFI), which accounts for the proportion of observed covariance between the manifest variables (items), explained by the fitted model (a concept similar to the coefficient of determination in linear regression). Generally, GFI values between 0.9 and 0.95 indicate good fit, and GFI values above 0.95 indicate a very good fit. Bentler's comparative fit index (CFI) was used to correct the underestimation that can occur when samples are small. CFI is independent from the sample size. Values between 0.9 and 0.95 indicate good fit, and values equal to or above 0.95 indicate a very good fit. The Tucker-Lewis index (TLI) varies between 0 and 1; values close to 1 indicate a good fit. Parsimony GPI (PGFI) is obtained to compensate for the "artificial" improvement in the model, which is achieved simply by adding more parameters, i.e., a more complex model may have better fit than a simpler model (parsimonious). Values between 0.6 and 0.8 indicate a reasonable fit and values above 0.8 a good fit. The index root mean square error of approximation (RMSEA) was used to adjust the model simply by adding more parameters. Empirical studies suggest that the model fit is considered good for values ranging between 0.05 and 0.08 and very good for values less than 0.05. The lavaan library from R was used [19].
