Many clinical quality registries have been established during the 2000s, particularly in North America where such registers are used as a criterion for health care funding. The quality of these registers and their ability to describe actual patient-reported outcomes has been largely ignored.
A new study, published in Neurosurgery, examines the reliability of clinical quality registers in reporting treatment complications among brain surgery patients.
“The study compared patient outcome data collected for research purposes at the Department of Neurosurgery of the Hospital District of Helsinki and Uusimaa in Finland with two national neurosurgery quality registers from the U.S. and one from the U.K.,” says one of the researchers, DMedSci Hanna Tuominen-Salo, specialist in anaesthesiology and intensive care medicine.
The study examined the U.K.’s national register, the Neurosurgical National Audit Programme (NNAP), which publishes the surgeon- and unit-specific mortality rates 30 days after surgery on its websites. The other registers were from the U.S.: the National Surgery Quality Improvement Program (NSQIP), which featured 136 adverse events, 20 of which were applicable for reporting treatment outcomes 30 days after a neurosurgical procedure, as well as the National Neurosurgery Quality and Outcomes Database (N2QOD), which collects data on 19 different outcome variables and is a register based on clinical treatment outcomes. The NNAP and the NSQIP primarily use register data collected for administrative purposes.
Mortality during the follow-up period of 30 days after neurosurgery was only 2.4 percent at Hospital District of Helsinki and Uusimaa in Finland. (Image credit: Elina Reponen)
“With their constantly accumulating, large amounts of data, quality registers are understandably an attractive source of information for comparative studies evaluating the safety and quality of treatment,” says one researcher, Docent Miikka Korja, who is a neurosurgeon and Chief Innovation Officer at HUS.
“If these registers are used to compare treatment facilities or even physicians without carefully evaluating the reliability and scope of the data, the results may lead to entirely misleading conclusions.”
The data encompassing 418 patients at the Hospital District of Helsinki and Uusimaa (HUS) Department of Neurosurgery featured significant treatment-related adverse events (e.g., pulmonary embolism or coronary infarction) in 18.7 percent of the patients, and minor adverse events in 38 percent.
Temporary and permanent paralytic symptoms relating to the surgery were recorded for 9.8 percent of the patients. Mortality during the follow-up period of 30 days was only 2.4 percent. Of the deceased patients, most were elderly or had undergone surgery for a malignant brain tumor.
“The HUS Department of Neurosurgery is a global pioneer in the transparent and open reporting of treatment outcomes, and I believe this to be a factor in the quality of the treatment, which has been objectively established to be high,” says Korja.
When the adverse events in the HUS data were classified according to the criteria in the U.K. and U.S. quality registers, most of the events did not even cross the reporting threshold.
The NNAP register only contains data for 30-day mortality, so all other adverse events (94.9 percent) went unrecorded. The criteria for NSQIP covered 96.2 percent of the significant, but only 38.4 percent of the minor adverse events. The most comprehensive register for adverse events was N2QOD, but even its criteria would have excluded 23.4 percent of the (primarily minor) adverse events in HUS.
“As health care becomes more patient-centered, the role of the patient as an active decision-maker has become more prominent. Transparent quality and patient satisfaction reporting for health care institutions, for example on the websites, is a common practice these days, and patients may choose their health care provider based on this information. For this reason, it is crucial to ensure that the published data describe treatment outcomes and adverse events comprehensively,” says DMedSci Elina Reponen, the principal author of the study and a specialist in anaesthesiology and intensive care medicine.
“The surgeon-specific mortality rates reported by the NNAP are misleading, as death is an extremely rare adverse event in contemporary neurosurgery. For example, every neurosurgeon working in the Töölö Hospital would have to perform surgery on 2,400 patients every year for us to be able to reliably detect a statistically significant difference of 50 percent between surgeons.”
The Hospital District of Helsinki and Uusimaa, and its Department of Neurosurgery in particular, intends to report on its surgery outcomes entirely openly, for example on a public website, and to include reporting from the patients themselves. The research group had previously established that patients who had had neurological brain surgery were very reliable in reporting complications themselves. Traditionally, complications are evaluated by the physician or nurse.
