A new study published by JAMA questions using the rate of post-operative blood clots as a hospital quality measure. The study examined whether surveillance bias (i.e., the greater the intensity of a search for a condition the greater likelihood it will be found) influences the reported rate of venous thromboembolism (VTE; blood clot). Venous thromboembolism, which includes deep vein thrombosis (DVT) and pulmonary embolism (PE), is a common post-operative complication that remains a leading, but potentially preventable cause of post-operative illness and death.
The Agency for Healthcare Research and Quality developed a risk-adjusted post-operative VTE rate measure, Patient Safety Indicator 12 (PSI-12), that has been incorporated into numerous quality improvement programs and public reporting initiatives. “However, measuring VTE rates may be flawed because of surveillance bias, in which variation in outcomes reflects variation in screening and detection, or ‘the more you look, the more you find’ phenomenon. This can occur in a number of ways: hospitals may use screening protocols, in which asymptomatic patients routinely undergo VTE imaging studies on a certain post-operative day, or clinicians have a lower threshold to order a VTE imaging study for patients with minimal or equivocal signs or symptoms. Hospitals that are more vigilant and perform more imaging studies for VTE may identify more VTE events, thus resulting in paradoxically worse performance on the VTE outcome measure,” according to background information in the article.
To examine the effect of surveillance bias on the validity of VTE as a quality measure, Karl Y. Bilimoria, M.D., M.S., of Northwestern University and Northwestern Memorial Hospital, Chicago, and colleagues conducted a study that merged 2010 Hospital Compare and American Hospital Association data from 2,838 hospitals. Next, 2009-2010 Medicare claims data for 954,926 surgical patient discharges from 2,786 hospitals that were undergoing 1 of 11 major operations were used to calculate VTE imaging and VTE event rates.
Instead of finding the expected relationship between adherence to VTE prevention and lower VTE rates, the researchers found that VTE prevention rates were positively correlated with VTE event rates. “A paradoxical relationship was also found between a measure of hospital structural characteristics reflecting quality [hospital characteristics that examine healthcare quality and which reflect a hospital’s resources and focus on programs intended to provide higher-quality care and VTE event rates: hospitals with higher structural quality scores had better VTE [prevention] adherence rates, but they had unexpectedly higher risk-adjusted VTE rates. Most important, hospital VTE rates were associated with the intensity of detecting VTE with imaging studies.”
Hospitals in the lowest imaging rate quartile diagnosed 5.0 VTEs per 1,000 discharges, whereas the highest imaging rate quartile hospitals found 13.5 VTEs per 1,000 discharges. “Our study calls into question the merit of the PSI-12 VTE outcome measure as a quality measure and its use in public reporting and performance-based payments. Hospitals reported to have the highest risk-adjusted VTE rates may in fact be providing vigilant care by ordering imaging studies to ensure that VTE events are not missed. Patients selecting hospitals according to publicly available metrics may be misled by currently reported VTE performance. The measure could be counterproductive if a hospital performs poorly on the VTE outcome metric, expends efforts to improve VTE prophylaxis resulting in increased awareness and vigilance in looking for VTE, and then finds more VTEs and becomes an even worse performer on the VTE measure,” the authors write.
Editor’s Note: This study was supported by AHRQ and a Center Development Award from Northwestern University and Northwestern Memorial Hospital to Dr. Bilimoria. Dr. Ju is supported by a grant from the National Institutes of Health. Dr. Haut is supported by a career development award from AHRQ. Please see the article for additional information, including other authors, author contributions and affiliations, financial disclosures, etc.