A health economics and outcomes research group has concluded that the assessment for high-risk medical devices in the European Union is of low quality and needs improvement. The report appears in the December 2017 issue of Value in Health by ISPOR.

The authors looked at 1,376 health technology assessment (HTA) reports filed between 2010 and 2015. Of those reports, 93 were eligible for analysis. Researchers rated several criteria, including:

• Evidence base — whether the evidence in the HTA reports was primarily based on submissions by the manufacturer, on data identified through an independent systematic literature search, or both.
• Type of evidence — distinguished between “direct” (e.g., head-to-head trials) and “indirect” evidence. Direct evidence from a well-conducted randomized controlled trial or a meta-analysis of such evidence was seen as providing the most valid estimates regarding the effectiveness of competing health care interventions. However, in some cases, interventions were not directly compared in RCTs. If there was insufficient evidence from direct-comparison trials, results of trials with different comparisons can be used to estimate the effects of treatments.
• Level of evidence — Using Cochrane Collaboration level of evidence analysis, researchers classified clinical studies used in HTA reports.
• Further considerations of scientific evidence — Whenever possible, the researchers behind hte report collected the total number of considered studies per HTA report. If HTA reports explicitly evaluated study quality (i.e., the risk of bias) using a specific tool or approach, the information was also considered for analysis.

In more than half the reports identified, evidence for the demonstration of the effectiveness and safety came from clinical studies of a moderate or low level of evidence, mainly case series or reports. Additionally, the findings illustrate that even if systematic reviews and randomized controlled trials were available for assessment, most of these studies showed an unclear or high risk of bias according to the specific tools used in their reports. The study showed that the evidence used in HTAs is systematically low for high-risk devices and the authors called for stricter quality requirements in trial designs and more alternative evidence sources to achieve marketing authorization.

The findings should be of concern for medical device companies that market to European countries. Authors also noted the HTA agencies in the EU face tremendous hurdles in making recommendations to healthcare decision makes. Although medical device investigations must adhere to principles of good clinical practice, there is no legal requirement for valid demonstration of the clinical benefit of a device in order for it to be in compliance with applicable directives. As a consequence, many high-risk devices are granted licensure based on low-quality evidence.

Although the evidence gap is a well-known problem, this is the first attempt to document the issue, according to report author Britta Olberg,MScPH, of the Berlin University of Technology in Germany.

“Our results show that additional changes are necessary, specifically with regard to the marketing authorization process of medical devices, with stricter quality requirements based on methodologically robust trials, possibly in combination with other evidence sources,” Olberg said.

Olberg strongly recommended enforcing a requirement of high-quality studies for demonstrating the clinical efficacy and safety of high-risk medical devices.