Assessing a safety climate tool adapted to address respiratory illnesses in Canadian hospitals

Abstract

Background:

Studies have shown an association between workplace safety climate scores and patient outcomes. This study aimed to investigate (1) performance of the hospital safety climate scale that was adapted to assess acute respiratory illness safety climate, (2) factors associated with safety climate scores, and (3) whether the safety scores were associated with following recommended droplet and contact precautions.

Methods:

A survey of Canadian healthcare personnel participating in a cohort study of influenza during the 2010/2011–2013/2014 winter seasons. Factor analysis and structural equation modeling were used for analyses.

Results:

Of the 1359 participants eligible for inclusion, 88% were female and 52% were nurses. The adapted items loaded to the same factors as the original scale. Personnel working on higher risk wards, nurses, and younger staff rated their hospital’s safety climate lower than other staff. Following guidelines for droplet and contact precautions was positively associated with ratings of management support and absence of job hindrances.

Conclusion:

The adapted tool can be used to assess hospital safety climates regarding respiratory pathogens. Management support and the absence of job hindrances are associated with hospital staff’s propensity and ability to follow precautions against the transmission of respiratory illnesses.

Introduction

Healthcare personnel are at risk of being exposed to pathogens from both symptomatic and asymptomatic patients. Routine or standard precautions are implemented to reduce the spread of pathogens between and among patients and their caregivers. The use of additional or transmission-based precautions is recommended in specific circumstances, including the use of contact and droplet precautions for patients with symptoms of a respiratory infection. ^1,2 Vaccination, hand hygiene, the use of protective equipment, and staying home while symptomatic are strategies available to reduce transmission of respiratory pathogens at the individual level yet the uptake of these measures varies by profession, the type of precaution, facility, ward, experience, perceived risk, workload, and other factors. ^3–5 Organization-level factors such as training, sick leave policies and practices, availability of equipment, and environmental controls impact the ability of healthcare personnel to protect themselves and their patients. ^6,7

Workplace safety climate is defined as individuals’ perceptions of safety-related policies, procedures, and practices. ⁸ Workplace safety climate scores have been associated with worker safety including the use of protective equipment and presenteeism. ^5,9–11 Several tools have been developed to evaluate workplace safety climate. Jackson et al., ¹² in their review of four safety climate scales including the Hospital Survey on Patient Safety Culture, Safety Attitudes Questionnaire, Patient Safety Climate in Healthcare Organization, and Hospital Safety Climate Scale (HSCS) concluded that each had acceptable psychometric properties.

Gershon et al. created the 20-item HSCS in which several of the tool’s six organizational dimensions (factors) were significantly associated with healthcare personnel’s safe work practices regarding blood-borne pathogens. ¹³ Since respiratory tract infections are the most frequently reported healthcare-associated infections, ^14,15 Turnberg and Daniell adapted the HSCS to assess practices regarding respiratory pathogen exposures. ¹⁶ The factors extracted from the modified tool were largely consistent with the HSCS, but the authors did not assess the factors’ associations with safe work practices. We likewise adapted several items from the HSCS to determine how Canadian healthcare personnel rated their hospitals’ safety climates with regard to respiratory pathogens.

We aimed to evaluate the performance of the adapted hospital safety climate tool with acute care hospital workers and to investigate participant characteristics associated with the tool’s scores. We also assessed whether the safety scores were associated with observance of recommended droplet and contact precautions.

Methods

This substudy was embedded within a four-season (2010/2011– 2013/2014) prospective cohort study of healthcare providers designed to assess risk factors for influenza. ¹⁷ The study was approved by the research ethics boards of all participating hospitals. Participants were recruited prior to the start of each influenza season from nine acute care hospitals in Toronto and Hamilton, Ontario and Halifax, Nova Scotia through direct emails to former study participants who agreed to be contacted about future studies, posters placed in work areas of hospital units, and short presentations about the study at staff meetings. Participation was voluntary and participants could withdraw at any time. Recruitment was designed to ensure that at least 30% worked in high risk areas (emergency departments, intensive care units, medical inpatient units, and general pediatric inpatient units), 30% cared for patients with acute respiratory illnesses, and 20% performed or assisted with high-risk respiratory procedures (categories are not mutually exclusive). Although participants could enrol for multiple seasons, only their first season of participation was eligible for this analysis. Other eligibility criteria included healthcare personnel who worked ≥20 hours per week, reported having direct patient contact, and completed the baseline questionnaire.

Each season, participants completed one on-line baseline questionnaire that incorporated 16 questions related to four workplace safety climate factors: (1) senior management support for safety programs [management support], (2) absence of workplace barriers to safe work practices [no job hindrances], (3) minimal conflict and good communication among staff [good communication], and (4) frequent safety-related feedback and/or training by supervisors [feedback/training]. To reduce respondent burden and in consultation with a co-author (DLH, an occupational health physician), the 16 items included 11 items from the HSCS ¹³ that were modified to address respiratory virus rather than blood-borne pathogen safety (similar to, but not identical to those the Turnberg and Daniell adaptation ¹⁶ ), 12 items related to availability of PPE, training, cleanliness/orderliness, and patient instructions were excluded, and five new items were added (see Supplementary Figure). Response options were rated on a 5-point Likert scale from strongly disagree (1 point) to strongly agree (5 points) with don’t know responses treated as neutral (3 points). A sensitivity analysis that excluded all questionnaires with a don’t know response was conducted to assess the impact of including don’t know responses as a neutral category.

Occupations, as reported in the baseline questionnaire, were categorized as physician, nurse, and other (ie, other healthcare professionals and administrative, research, and support staff). Adherence to precautions was categorized according to Canadian recommendations for droplet and contact precautions ¹⁸ as (1) fully compliant: usually/always wear a surgical mask or N95 respirator, gloves, and a face shield or goggles and wash hands after patient contact, (2) partially compliant: usually/always wear a surgical mask or N95 respirator and gloves and wash hands after contact but sometimes/rarely/never wear face protection, (3) limited compliance: sometimes/rarely/never wear a surgical mask or N95 respirator, gloves, nor wash hands after contact, or (4) not applicable: not in close contact with patients with febrile respiratory illnesses.

A confirmatory factor analysis was conducted to measure the performance of the models. Goodness-of fit index (GFI), adjusted goodness-of-fit index (AGFI), normed fit index (NFI), and comparative fit index (CFI) values of ≥0.90 and root mean square error of approximation (RMSEA) values between 0.03 and 0.08 were considered a reasonable minimum fit for model acceptance. ¹⁹ Measures of sampling adequacy (Kaiser–Meyer–Olkin and Bartlett’s test of sphericity) were calculated prior to exploratory factor analysis, ²⁰ which was conducted if unreasonable fit indices were obtained during confirmatory factor analysis. An exploratory factor analysis-developed model with four factors was extracted by the principal axis factoring method. Factor loadings (ie, the correlation between the variable [item] and the factor) were analyzed by Varimax orthogonal rotation. ²¹ Variables with factor loadings of ≥0.40 were grouped. ²²

Structural equation modeling was conducted to assess the effect of demographic (ie, hospital site, age group, sex) and occupational variables (occupation, years of work experience, and work area) on the four workplace safety climate factors. Four participant-specific factor scores were generated by calculating the mean score of items within each factor. ²³ The scores for the four factors were compared using non parametric Friedman test, with the Tukey method used for post hoc multiple comparison assessment.

Multinomial logistic regression was used to compare the levels of adherence to precautions (limited, partial, or full compliance) by quantile of the mean value of each of the four workplace safety climate factors. Participants who reported that the use of protective equipment was not applicable were excluded. Potentially confounding variables (season, province, age, sex, years of work experience, occupation, and work area) were included in the model. The model was assessed for influential observations and goodness of fit using two separate logistic regression models. All statistical analysis used R statistical software 3.0.0 (Institute for Statistics and Mathematics, Vienna, Austria). The sample size was deemed sufficiently large based on both the more liberal rule of thumb of ≥500 observations as well as the 5:1 ratio of observations per variable for exploratory factor analysis of the primary research question. ²⁴

Results

Of the 2,093 healthcare personnel enrolled in the study, 1,473 were eligible for this substudy and 1,359 (92.3%) submitted complete baseline questionnaires. As shown in Table 1, nurses, those working in high risk areas, and participants in earlier seasons of the study were more likely to have complete data than other participants. Of those included in the analyses, 1,191 (87.6%) were female, 702 (51.6%) were nurses, and 709 (52.2%) worked in higher risk areas.

Table 1. Characteristics of eligible Canadian healthcare personnel, 2010/2011–2013/2014 influenza seasons

Note. IQR: Interquartile range.

^a Non-physician/non-nurse acute care hospital workers (eg, other healthcare professionals, support and administrative staff).

^b Emergency department, intensive care unit, medical inpatient unit, or pediatric inpatient unit.

Figure 1 shows the distribution of responses to the full 4-factor/16-item questionnaire. More than 80% of participating healthcare personnel agreed or strongly agreed with 12 of the 16 items about their workplace safety climate. The exceptions were the three items in the feedback/training factor and one from the absence of job hindrances factor, for which <80% agreed.

Figure 1. Distribution of item response using adapted hospital safety climate scale (HSCS) questions; acute care hospital healthcare personnel, Canada, 2010/2011–2013/2014 influenza seasons.

Performance of adapted scale

Table 2 demonstrates the results of the exploratory factor analysis of the 16 items loaded to the four factors identified in the HSCS: management support, absence of job hindrances, minimal conflict/good communication, and feedback/training. Management support, comprised of four items from the HSCS, explained 17% of the variance with factor loadings (ie, correlations between item and factor) from 0.63 to 0.73. Absence of job hindrances, with only two items, explained 13% of the variance. The original and adapted items from the HSCS had loadings ranging from 0.50 to 0.79. Four of the five new items loaded to one of the original four factors with loadings of 0.45 to 0.66. Item 9, “in my current work area, my coworkers support me in following safe work practices,” did not correlate highly with any of the four factors. In the sensitivity analysis, with all don’t know responses categorized as missing, the factor loading results (see Supplementary Table) were largely similar to the results in Table 2.

Table 2. Results of exploratory factor analysis of adapted hospital safety climate scale items, acute care hospital healthcare personnel, Canada, 2010/2011–2013/2014

Note. *See Supplementary Figure for changes from HSCS. HSCS: hospital safety climate scale; Good communication: minimal conflict/good communication; N/A: not applicable.

The indices of fit from the confirmatory factor analysis of the full 4-factor/16-item model were not optimal (Table 3). The Kaiser–Meyer–Olkin measure of sampling adequacy was 0.94 and the significance of Bartlett’s test of sphericity was <0.001. Although the exploratory factor analysis generated a 4-factor/15-item model (omitting item 9) explaining 56% of the cumulative variance, the adapted HSCS 4-factor/11-item model explained 60% of the cumulative variance, had slightly better fitness indices, and is used for the remaining analyses.

Table 3. Indices of fit for models of an adapted hospital safety climate scale, acute care hospital healthcare personnel, Canada, 2010/2011–2013/2014

Note. EFA: exploratory factor analysis; HSCS: hospital safety climate score; GFI: goodness-of fit index; AGFI: adjusted goodness-of-fit index; RMSEA: root mean square error of approximation; NFI: normed fit index; CFI: comparative fit index.

Factors associated with the adapted safety climate score

The mean scores of the adapted items within each of the four factors in the 4-factor/11-item model were highest for management support (4.17, standard error [SE] 0.02), followed by minimal conflict/good communication (3.99, SE 0.02), absence of job hindrances (3.86, SE 0.03), and feedback/training (3.56, SE 0.03); P < .001. As shown in Table 4, the mean score of each factor was one-tenth to one-quarter (β: −0.10 to −0.26) of a point (on a scale with a maximum of 5 points) lower for healthcare personnel working in higher risk compared with other hospital areas. Also, nurses scored three of four factors significantly lower than other (non-nurse/non-physician) healthcare personnel, with the rating of lack of job hindrances four tenths (β: −0.40) of a point lower for nurses. Conversely, older participants scored factors higher than younger ones with the lack of job hindrances factor two tenths (β: 0.19) of a point higher for 35–49 compared to 20–34-year-old participants and three tenths of a point (β: 0.31) higher for 50–69 compared to 20–34-year-olds.

Table 4. Association between hospital safety climate scale factors and participant characteristics, structural equation model results ^a ; acute care hospital personnel with direct patient care, Canada, 2010/2011–2013/2014

Note. *P < .05; **P < .001. NS: Nova Scotia; ON: Ontario; SC: standardized coefficient.

^a See supplementary Figure 1 for structural equation model’s path diagram.

^b Non-physician/non-nurse nurse acute care hospital workers (eg, other healthcare professionals, support and administrative staff).

^c Emergency department, intensive care unit, or medical inpatient unit.

Association of adapted safety score with following recommended droplet and contact precautions

Of the 980 healthcare personnel whose job put them in close contact with patients with a febrile respiratory illness, 109 (11%), 629 (64%), and 242 (25%) reported limited, partial, and full compliance with droplet and contact precautions. Participants who worked in higher risk areas were significantly more likely to report partial or full compliance with precautions than participants working in other hospital areas. Nurses were significantly more likely than other (non-physician) healthcare personnel to report partial or full compliance with recommended precautions. Meanwhile, 50–59-year-old participants were less likely to report partial or full compliance compared with 20–34-year-olds.

After adjusting the 4-factor/11-item model for participants’ age, occupation, risk profile of work area, and sex, self-reported compliance with droplet and contact precautions was higher for those who rated management support items higher. For those who rated management support in the highest quartile, the adjusted relative odds ratios (arOR) were similar for those reporting partial (arOR 3.19, 95% CI 1.26, 8.05) and full (arOR 4.08, 95% CI 1.52, 11.0) compliance and significantly higher than those reporting limited compliance (Table 5). Although the use of protective equipment was higher for participants who rated the absence of job hindrances factor higher, the association was not consistently significant across scores.

Table 5. Association between safety climate factors and use of protective equipment, Canadian acute care hospital healthcare personnel with direct patient care, 2010/2011-2013/2014

Note. *P < .05; **P < .001.

^a Compliance when caring for patient with febrile respiratory illness defined as Full: usually/always wear surgical mask OR N95 respirator, gloves, and face shield/goggles and wash hands after patient care; Partial: usually/always wear surgical mask OR N95 respirator, gloves and wash hands after patient care (goggle/face shield used sometimes/rarely/never); Limited: sometimes/rarely/never on any one or more of: surgical mask or N95 respirator, gloves, or wash hands after patient care.

^b Adjusted for other variables in column, season, and province.

^c Emergency department, intensive care unit, medical inpatient unit, or pediatric inpatient unit.

^d Non-physician/non-nurse nurse acute care hospital workers (eg, other healthcare professionals, support and administrative staff).

Discussion

This 4-factor/11-item adaptation of Gershon et al.’s workplace safety climate scale to address healthcare personnel’s perception of safety concerning respiratory pathogens resulted in factor loadings to the same four factors/dimensions as the original scale. ¹³ Turnberg and Daniell reported the same conclusion with their adaptation of the full 25-item scale that was done to address respiratory pathogen exposures. ¹⁶ They reported, however, that the feedback/training factor separated into two factors in their population, with feedback separate from training. In the current study, only two of the original five items in the feedback/training factor were included; both items loaded to the feedback factor (ie, not training) in Turnberg and Daniell’s study.

In the current study, nurses rated items from the management support and absence of job hindrances factors significantly lower than non-physician healthcare providers with direct patient contact. Singer et al. ²⁵ also reported that nurses in 92 US hospitals were more likely than physicians to negatively rate senior managers’ engagement, unit recognition/support for safety efforts, and provision of safe care. Alternatively, Huang et al. ⁴ found no difference in the perception of the safety culture of physicians and nurses working in an intensive care unit at one US hospital. The disparity in findings may reflect the work conducted or work culture in hospital units rather than occupational differences.

In this study, healthcare personnel working in higher risk areas for transmission of respiratory illnesses (intensive care units, emergency departments, adult medical inpatient, and pediatric inpatient units) rated the workplace safety items lower than personnel working in other hospital areas. Singer et al. ²⁵ reported that healthcare personnel in emergency departments rated the safety climate lower than those working in other areas, including intensive care units. Zhou et al. ²⁶ reported lower levels of safety climate scoring by Chinese healthcare personnel working in pediatrics, intensive care, emergency, and outpatient areas than those working in internal medicine, surgery, and obstetrics/gynecology. These differences are not unexpected given that healthcare personnel in these departments often face less predictable patient care requirements and may need additional support from supervisors and infection control staff to facilitate precautions against healthcare-associated infections.

The safety climate factors that were consistently associated with higher levels of protective equipment use in this study of acute care hospital healthcare personnel were management support and absence of job hindrances. This is not unexpected given the fact that the items in both of these factors ask healthcare personnel to rate whether protecting staff from respiratory disease is a priority in their workplace. Gershon et al. ¹³ similarly reported that higher ratings of management support and absence of job hindrances were correlated with strict compliance with universal precautions. DeJoy et al. ¹⁰ also report that protective equipment use among nurses administering liquid antineoplastic drugs to patients was higher for those rating management commitment to safety higher. To increase the use of protective equipment, supervisors and upper management need to be seen as caring for the health of their staff through various means including adequate staffing levels and involvement in health and safety matters.

Of interest, although nurses and healthcare personnel working in higher risk areas rated hospital safety climate factors lower than their peers, they were more likely to follow droplet and contact precautions when working with a patient with a febrile respiratory illness. Equal and opposite, while older healthcare personnel rated their hospital’s safety climate factors higher than younger ones, they were less likely to follow recommended precautions. Although focused more on patient safety rather than healthcare provider safety climate, Hessels et al. also found that the observed adherence to standard precautions by medical, surgical, emergency department, and intensive care unit nurses from five US hospitals was negatively correlated with high ratings of patient safety climate factors, specifically staffing levels and teamwork. ²⁷ It may be that personnel who recognize the need to adhere to guidelines note the lack of support more acutely than those who do not. However, these findings are contrary to the findings of three studies conducted in the USA and Canada that reported higher rates of protective equipment use with higher scores on hospital safety tools. ^28–30 Further study into these contradictory findings are needed to understand the issue.

This large cohort study spanned four influenza seasons, reducing the likelihood that responses to the questionnaires were biased by infectious disease outbreaks. On the other hand, participation in the study was voluntary and limited to healthcare personnel with access to and familiarity with computers and e-mail increasing the likelihood of selection bias. Participation was also voluntary, with oversampling of healthcare personnel in areas considered higher risk for exposure to patients with acute respiratory illnesses. As such, the distribution of participants by discipline and work area does not necessarily represent the staff of each hospital and will limit the generalizability of results. Also, since data were self-reported, there is likely to be social desirability bias with the reporting of protective equipment use. As such, results should be interpreted with some caution.

Unlike Gershon ¹³ and Turnberg ¹⁶ , don’t know responses were categorized as neutral in the current study’s 5-point Likert-like scale rather than being excluded. However, in a sensitivity analysis in which don’t know items were excluded and a 4-point score was used, similar loadings resulted: only one item, one that was new in this study’s version, loaded differently. For researchers who prefer to force non-neutral responses and/or want to shorten the time to complete a survey, the 4-point scale may be considered.

This study demonstrates the adaptability of the HSCS to assess the respiratory illness safety climate within healthcare settings. The tool may be useful for infection prevention and control programs within hospitals to assess the safety climate in their facilities and to use the findings to improve conditions.