A study led by researchers at the Peter Munk Cardiac Centre (PMCC) at UHN suggests that the model used by the Canadian Institute for Health Information (CIHI) significantly underestimates mortality in specialized heart surgery centers. The findings show that CIHI’s model does not capture all of the medical problems that patients at high risk for surgery have, and therefore does not accurately predict mortality after heart surgery in these patients.
The study, published in CMAJ Open, looked at the outcomes of 1,635 cardiac operations performed at the PMCC between 2013 and 2016. It compared the observed mortality within 30 days of surgery to the predicted mortality rates estimated with either the CIHI administrative data model or the clinical data model used by the Society of Thoracic Surgeons (STS).
Of the 1,635 patients that had heart surgery at the PMCC, 32 died within 30 days of surgery. These results were in line with the 1.96 percent mortality predicted by STS model, which uses comprehensive data to describe how sick patients are before surgery, for this group of patients. In contrast, the CIHI Cardiac Care Quality Indicator estimated that the mortality rate for these 1,635 patients would be 1.03 percent.
PMCC researchers noted that the STS model captures seven medical conditions that predict worse outcomes after heart surgery that are not captured in the CIHI model. These risk factors include whether the patient had heart failure, an abnormal heart rhythm, a recent heart attack, very low blood pressure (shock), needed recent CPR or a mechanical heart pump to live, or had kidney failure. These serious conditions predict a worse outcome after heart surgery, and are not included in the CIHI database.
“The predicted mortality of patients undergoing heart surgery based on the STS model is similar to what we actually observed,” says Dr. Barry Rubin, Medical Director of the PMCC and one of the authors of the study. “The failure to include these seven medical conditions causes the CIHI database to underestimate predicted mortality after heart surgery in high risk patients.”
According to Dr. Douglas Lee, senior scientist at the PMCC and lead author of the study, mortality prediction models — either based on clinical or administrative data — use risk adjustment to account for how sick patients are before surgery at different hospitals. This is necessary because leading academic institutions such as the PMCC may operate on sicker patients, who may not have as good outcomes after surgery.
“The CIHI and STS models aim to predict outcomes based on the medical complexity and acuity of the patient. In general, academic hospitals take on higher risk cases compared to community hospitals, and good risk adjustment models should factor that in when predicting mortality rates,” explains Lee.
CIHI has a legislative mandate to publicly release Cardiac Care Quality Indicator data, and has done so since October 2017. Clinicians and researchers have been concerned that the CIHI model may underestimate how sick cardiovascular surgery patients are at PMCC.
Surgical mortality that is higher than what CIHI data estimates could lead to the incorrect impression that mortality is higher at the PMCC or other academic centres than it should be. This could have the unintended effect of causing the highest risk surgery patients to defer potentially-life saving heart surgeries.
By taking into account the significant medical conditions these high-risk patients have, the STS model accurately predicts how these patients will do after surgery. The CIHI model does not accurately account for complexity of patients, which is recorded in databases specifically designed for the measurement of surgical quality, such as STS.
“Clinical data-based models like the STS collect much more detailed patient information, but are also more costly to maintain,” says Rubin. “Administrative models like CIHI’s continue to play a very important role in assessing quality of care across Canada. We will continue to work in collaboration with CIHI to improve the accuracy of quality report cards that can be used as valid evaluation tools for Canadian hospitals.”
A grant from the Canadian Institutes of Health Research supported this research study. The authors caution that there are limitations to the study, which was done in a single center during a three-year period, observing 1,341 isolated coronary bypass graft, 143 isolated aortic valve replacement and 151 combined procedures. PMCC researchers suggest validation of these findings in other academic centres, over a longer time interval that would include a larger subset of cardiac operations. The authors of this study declared no conflicts of interest.
