If you work in a hospital these days, you’ve probably gotten the invitation: Take a survey about how well you, your team and your hospital do at protecting patients from harm, and how empowered you feel to do the right thing.
In fact, you’ve probably gotten many invitations and reminders to take the survey, as your hospital tries to get employees to respond. It’s all in the name of gauging something called “patient safety culture.” But a new study questions whether such surveys actually measure how well a hospital is doing at keeping patients safe.
A team of researchers based mainly at the University of Michigan Medical School and VA Ann Arbor Healthcare System did the study using data from hundreds of hospitals. They’ve published their findings in the journal BMJ: Quality & Safety.What they found surprised even them. Overall, hospital units’ patient safety culture scores didn’t match up with how well the units did on a key patient safety goal: Reducing two risky infections that patients can catch during their hospital stay.
Study: No connection between the safety culture scores derived from the surveys, and the actual decline in infection rates.
Big Disconnect – Big Implications
This disconnect between safety culture survey results and actual safety improvement happened in hospital units around the country that had signed on to a national patient safety project.
All the units were working to reduce two kinds of infections that patients can get during a hospital stay: One called central line associated blood stream infection (CLABSI) from devices used to deliver medicine into their bloodstream, and another called catheter-associated urinary tract infection (CAUTI), from devices used to collect urine.
The hospitals gave staff technical help — tools, training, new procedures and other support to help reduce these infections. They also fostered cultural changes aimed at improving teamwork and encouraging staff to speak up to stop an unsafe situation. And, they surveyed employees about patient safety at the start, and toward the end.
“We hypothesized that those that did better on survey measures of safety culture would achieve better infection rates, especially given that there had been so much effort put into trying to improve safety culture in these collaboratives,” says lead author and assistant professor Jennifer Meddings, M.D.
But, she explains, “In the data from both collaboratives, there was no connection between the safety culture scores derived from the surveys, and the actual decline in infection rates on the units. We think this indicates it’s much more difficult to detect and measure safety culture than has been thought.” She also hopes that new or revised surveys will get tested for their ability to measure real change on a small scale – a process called validation – before they’re launched on a large scale.
Survey Says…
Many hospitals put a lot of resources behind trying to get employees to take the surveys that form the basis for the safety culture scores.
Even so, less than half of the hospital staff on the units in these projects took them. The length of the survey – 42 questions for each staff member – may have contributed to this. But with survey results from only a portion of employees, the score that comes from those results may be inaccurate.
A previous study that looked at baseline data from the CLABSI-reduction project found that performance on a measure dubbed “safety climate profile” generated by researchers using some combinations of the individual measures did correspond with reductions in infections.
But Meddings and her colleagues wanted to use the safety culture data that hospitals actually receive back once in feedback their employees have taken the survey, as recommended by the survey tool instructions, without creating any new types of composite scores. They looked at how well hospital units’ scores on each one of the 42 questions in the Hospital Survey on Patient Safety Culture (HSOPS) survey corresponded with rates of CLABSI and CAUTI in those units.
Though the hospital units in the study managed to reduce CLABSI rates by 47 percent and CAUTI rates by 23 percent overall, the changes in HSOPS score were minor. For some, hospital safety scores worsened despite improvements in infection rates.
Meddings says the findings should give the patient safety community reason to evaluate how patient safety culture surveys get used as part of larger safety-focused efforts, which are often routine components for participation in collaboratives. “Unexpectedly, the data from these collaboratives showed no association of better infection rates for hospitals with better safety culture scores,” she says. “These data suggest that either the infection rates were improved by improving other aspects of care than safety culture such as improving skills in catheter use, or that these surveys simple do not capture safety culture well.”
She adds, “We hope these analyses inform future collaboratives to be designed in way to better assess which components of multi-component interventions are most important to reduce infection and to reduce the routine use of culture surveys for busy clinicians until there are clearer links between the surveyed measures and culture observed on the units.”
