[Image courtesy Army Medicine on Flickr]
The 21st Century Cures Act, passed by Congress in late 2016, actually promotes real-world evidence. The practice offers a way for a medtech company to prove safety and efficacy without necessarily having to conduct a clinical trial. But there are also worries that it exposes a device company to accusations that it is promoting off-label use of a device.
Michael Drues, president of Vascular Sciences, is a Boston area regulatory consultant who has worked with both medical device companies and FDA. He recently spoke with MDO about what people in the industry need to consider when it comes to real-word evidence.
MDO: What exactly is real-world evidence?
Drues: It has different meanings to different people. … But here’s what it means to me. Real-world evidence means data collected and associated with the medical device or drug when used in the “real world” that is not necessarily in a clinical trial. This has been a topic of much debate and much controversy for a long time – especially at the FDA because oftentimes real-world evidence or real-world data is synonymous with off-label use. And as you and your audience know, historically FDA has not been keen on considering data from off label use as part of the submission.
That is starting to change a little bit – very, very slowly. As you know, FDA put out a guidance on this last summer, but I can tell you the pill is difficult to swallow for a lot of my FDA reviewer friends. I’ll give you a very quick example. As you know, I do a lot of [presubmissions] at the FDA. We did a presub just a few months ago which was basically the label expansion. That is, this particular medical device was already on the market. … It’s actually got a few 510(k)s on it, and we were doing a label expansion to add a different indication. We went into the FDA with a whole pile of real-world evidence because, after all, this device has been on the market for several years and has been used with several thousand patients already.
FDA was insisting that we do a clinical trial. Historically, the randomized clinical trial or RCT has been considered the gold standard, and we basically said, “With all due respect, we don’t need to do that because we’ve already got a ton of real-world evidence on it.” Which, I would argue is actually much better than the “gold standard” of the randomized clinical trials because it’s data that was collected in real life use as opposed to some very controlled, very artificial clinical trial. Anyway, long story short, FDA was really not swallowing that pill and was really pushing back hard that they wanted clinical trial data.
I anticipated this. Coincidentally, we [were doing the presubmission] shortly after this particular guidance came out. I anticipated this kind of response. I had a hard copy of the guidance with me, and I literally, I was out in front of them, and I said, “Look, here’s a guidance that just came out … basically saying how we should do exactly what we are proposing on doing.” Bottom line, even though the guidance has come out, one of the things that your audience has to understand for a variety of reasons is [that] it’s a difficult pill, to use that metaphor, for a lot of my FDA reviewer friends to swallow.
MDO: When is real-world evidence a good fit?
Drues: Clearly, if you’re working on a brand new medical device – a device that’s not on the market – then, obviously, real world evidence is not applicable because the product is not on the market. Therefore, you have no real-world evidence. Now, where it starts to get interesting is if you have a device on the market, say somewhere else in the world, and you are now going to get the device on the market for the first time here in the United States. Can you use real world data that was collected from some other country as part of a U.S. submission? FDA did put out a guidance, I think maybe about 2 years ago now, specifically addressing clinical trial data from outside the U.S. But as far as I know, no specific guidance has come out on real-world data from outside the US.
My personal opinion is that we should be willing to consider all data regarding a device regardless of what part of the planet you happen to be standing on. The most obvious time that a company should consider using real-world data is in a scenario that I just described a moment ago where you have a device that’s already on the market here in the United States, and you’re going back to the FDA – most likely in the form of a label expansion. There’s something you’re changing about the device, and because the device is already on the market, you probably [have evidence collected by your company] or maybe there has been published studies in the literature that your company had nothing to do with that might support the label expansion. Those are the most common scenarios where, I think, real world data, real world evidence needs to be considered. …
Again, this is just my opinion and I think it’s important to understand different sides of all the issues. But I personally think having a policy like, “You can never consider real world evidence, or you can never consider off-label use data,” … I personally think that might be like an ostrich sticking your head in the sand, because quite frankly if the data is already out there and if in some cases it might be better data that is more realistic data, than the so-called gold standard to randomize clinical trials. Why not be able to use it? It makes no sense to go out and collect additional data for no other reason other than that.
MDO: It really is an expanded label situation in some cases. It basically means you’re giving them real-world evidence on off-label use, right?
Drues: Well, it can be, and that’s one of the reasons why this is such a thorny porcupine, so to speak. …. I have no problem using real world evidence when it’s appropriate, but let me put an important caveat on that. … Whether you’re talking about real world evidence or … the “gold standard” randomized clinical trials, … the quality of the data is only as good as how it was collected.
In other words, you have to know under what circumstances … the data was collected and so on.
MDO: When you are referring to the quality of the data, what makes it quality data?
Drues: Things like sample size and number of patients or statistics, what we would call them, the power of the trial – those are all very important things to consider. Whether you consider that to be the quality of data or not, I’m not sure. That’s sort of a matter of semantics, but really what I refer to as the quality of data means in the traditional randomized clinical trial, one of the things that’s very important is that we make sure that all of these investigators – the surgeons, nurses, whoever they are – are all doing the same thing. They’re all following the protocol, because anybody that knows something about statistics, you can’t do an apples-to-apples comparison if everybody is doing things a little bit differently. One of the ways that we ensure that in a typical clinical trial is via the use of what’s called a clinical monitor – which is basically somebody who is responsible for, in some cases, physically going out and watching how that device is used in the clinical trial in order to make sure that the investigator is doing it the way they’re supposed to do it.
When it comes to real-word evidence, because it’s not in a formal clinical trial, most likely they’re not using a clinical monitor. As a matter of fact, since it’s not in a clinical trial, there is no clinical trial protocol per se because there’s no trial. We have to be very careful about being able to make an apples-to-apples comparison if people are not all doing things the same way. That is, in my opinion, a very legitimate concern, perhaps even the most significant concern about using real-world evidence. That said, I think it is a potential challenge, but … I think that’s a solvable problem.
I don’t think that we should ban the use, as we have in the past, of real-world evidence just because of that particular problem. I think it’s a very legitimate concern . … One of the first questions that a good FDA reviewer could and probably should ask is, “How do you know that since was not a clinical trial, that everybody did this the same way and therefore it makes sense to pool all that data together?” In my opinion, it’s up to the company, meaning us, to be able to have a response to that question, and if we cannot present that, if we cannot show, “Well here’s how we know,” then whether or not we can actually use that data can be very much under debate.
MDO: I could see people saying, “Well, this is just a way for companies do an end run around doing real science around their devices.” What is the best answer to that?
Drues: Yeah, it’s an excellent question, and it can be very useful. It can also be potentially very harmful as well. Once again, let me be crystal clear: This is not an excuse for companies to take short cuts. This is not an excuse to be sloppy in terms of our reporting, our data analysis, our statistical methodology or any of that kind of stuff. Once again, it comes down to the quality or the integrity of that data. Whether it’s collected in a formal clinical trial or outside of a clinical trial, quite frankly, I don’t really care.
Shakespeare said, “A rose by any other name would smell as sweet.” At the end of the day it comes down to the quality of that data. …
Simply put, I view real-world evidence as another important tool which we can and, in my opinion, should be able to use when it makes sense. But like any tool – whether it’s a scale poll or a regulatory strategy – in the hands of … a skilled user, that tool can be very beneficial. In the hands of somebody that’s not a skilled user, you can cause an awful lot of damage.
[Want to stay more on top of MDO content? Subscribe to our weekly e-newsletter.]
