Patients and healthcare providers remain at the core of successful AI implementations in medtech. [piai/Adobe Stock]
But the allure of AI can sometimes overshadow the central goal of addressing tangible clinical problems.
During his talk at DeviceTalks West, Ha Hong, chief AI officer at Medtronic Endoscopy, underscored the importance of putting patients and healthcare providers at the forefront when incorporating AI into medical devices.
With a plethora of AI tools at our disposal — many of which are increasingly user-friendly — the onus is on us to wield them responsibly. Below, you’ll find tangible feedback distilled from Hong’s experience commercializing AI-enabled medical devices at Caption Health (acquired by GE HealthCare) and his current role at Medtronic Endoscopy.
1. Keep your eye on the clinical problem when working with AI in medtech
Medtronic Endoscopy Chief AI Officer Ha Hong [Photo courtesy of Medtronic]
In the medical device space, Hong encouraged focusing on solving real clinical problems rather than looking for places to apply AI.
While it is possible that a technology-first approach could offer a clinical benefit, the approach can be risky in the high-stakes medical device field, where it could take years to commercialize a device.
“At the end of the day, it’s really sad if you find out that you’ve spent 10 years of your life creating a device that doesn’t matter clinically,” Hong said.
2. Data quality is paramount in medtech
A staggering amount of data is available in healthcare, with RBC Capital Markets estimating the industry generates some 30% of global data. But data quality is frequently more important than quantity in medical device development. For example, assume you have two different training datasets: one with 1 million data points and another with 1 billion data points.
“The dataset with 1 billion data points might look more impressive than the one with 1 million, but what if the 1 billion dataset is highly skewed?” Hong said.
In an extreme example, 99.9999% of the larger data set could be confined to a single category, while the 1 million dataset has equal representations of all the data types a device developer cares about.
FDA’s guidance on Good Machine Learning Practice prescribes ensuring that data sets are representative of the intended patient population for a device. The agency also recommends maintaining independence between training and test datasets to minimize bias and promote generalizable performance across the intended patient population. The document also stresses the importance of adhering to good software engineering practices, including prioritizing data quality assurance and robust data management to ensure data authenticity and integrity.
FDA continues to release new guidance for AI-enabled medical devices and to request feedback from the industry on its thinking.
3. Get your hands dirty
While it’s vital to keep the clinical problem in mind, it also takes work to understand best practices of AI tool deployment. In the past, the barrier to entry to using AI tools was considerable.
“Now, many of the tools are becoming very easy to use,” Hong said. “But you still need to study the basics.”
Before embarking on incorporating AI functionality into a medical device application, “it’s useful to just dabble and get your hands dirty,” Hong said. “There are really good tutorials that provide you with a better understanding of the basic fundamental concepts.”
After grasping AI’s basics, refocus on the clinical issue. “Don’t jump straight to coding; take the time to understand the problem and plan accordingly,” Hong said.
4. Be transparent and concrete
Many companies and individuals tend to use the term “AI” broadly without qualifying which technology they are referencing. The lack of precision can obscure the message. Device developers describing an AI product should be specific. A website describing an AI-enabled product might include a clickable asterisk that offers curious individuals with an opportunity to dive deeper.
Transparency is key in helping establish trust. There are a range of strategies available to ensure transparency, such as publishing white papers or articles in academic journals, both clinical and technical.
In addition to being transparent about the type of AI involved in a device, another consideration is to favor explainable models over black box algorithms when possible. By providing additional context and data such as confidence levels, device developers can offer valuable information to physicians and help them contextualize AI predictions.
“Rather than just giving an answer to a clinical question, for example, provide the confidence level as well,” Hong said. “Rather than just saying, ‘I found the polyp in this video frame,’ try to put a bounding box around it.”
Being transparent with the FDA and other regulatory agencies — which Hong said is “the barest minimum” — allows them to inspect a company’s processes and technologies to ensure the AI systems are safe, effective and trustworthy. The ultimate aim is to develop responsible AI medical technologies to improve patient care, enabling more people to realize the utility of AI systems in healthcare.
5. Less can be more when it comes to AI in medical devices
In the AI domain, data scientists and other highly skilled professionals can command sizable influence. But individuals lacking deep technical skills can also play an important role in bridging the gap between technology and real-world applications. Similarly, relatively simpler tools with few parameters can sometimes be more effective and have fewer points of failure than their more complex counterparts.
“If you can solve your problem with simpler tools, it’s actually better,” Hong said. “It’s not always necessary, for instance, to use deep learning technology with thousands of layers and billions of parameters. I would use simpler tools all the time if they can effectively solve the problem at hand.”
6. Human intelligence can complement machine intelligence
The proliferation of large language models such as ChatGPT has reduced the barrier of entry for non-technical people to use AI. Such models can streamline research and development.
“These tools have increased productivity for many people,” Hong said.
For example, using a large language model, you can easily draft certain text types, such as the introductory text on known facts in internal communications, business, marketing and regulatory applications. While humans still need to verify the output’s accuracy, the time saved in generating initial drafts is often considerable.
“But when it comes to creating safe and effective medical devices or tangible products that people are willing to pay for, the tool is only part of the equation,” Hong said. “Your understanding of the technology and your ability to use the tool correctly are crucial,” he added.
Human intelligence still matters. A company with the best technology will unlikely succeed unless paired with a clear strategic vision.
7. Aim for a symbiotic relationship between AI and physicians
On one end of the spectrum, skeptical physicians encountering an AI-enabled radiological device might reject all of its advice. At the opposite pole, an especially enthusiastic group could theoretically accept all of an algorithm’s interpretations without reflection.
“There needs to be a middle ground where AI assists physicians, but physicians are in the driver’s seat,” Hong said. “AI is not 100% perfect, and no system can be.”
But people make mistakes, too. Bringing people and AI together can maximize clinical benefits while minimizing errors. To aid physicians in their decision-making, the AI’s predictions must be presented clearly and unambiguously.
“At the end of the day, the AI system spits out a number, but the visual presentation of it actually matters a lot,” Hong said. “When you present the AI predictions to the user, make sure that it’s not ambiguous and is crystal clear.”
8. Seeing regulatory agencies as AI allies
Instead of viewing regulatory agencies as gatekeepers looking to block innovation, consider them allies who ensure new technologies are safe and effective for public use. Regulatory agencies can provide valuable feedback and even product ideas that can be beneficial in the long run.
“They are very, very happy to give you some really constructive feedback,” Hong said.
Hong reiterated the importance of maintaining transparency with regulators, saying they want to help, not to steal trade secrets or intellectual property.
9. Validate your model to ensure unbiased and accurate performance
Validation is the linchpin that holds together the efficacy of machine learning models in medical applications. While novice ML developers might be tempted to use as much data as possible for training, “It is important to save enough data for validation,” Hong stressed. “Please, please, please make sure that you have enough data for validation.”
Ultimately, a dataset should be meticulously curated to capture the full spectrum of disease types and confounding variables. A multidisciplinary approach is paramount in the medical device creation process. Assembling a team that includes stakeholders from engineering, regulatory, clinical, marketing, product, and human factors design ensures a holistic perspective is applied to the project.
“Quality of data can be improved by increasing the fidelity of ground truth labels and correcting mistakes made by human labelers,” Hong said, noting that another strategy is to increase the data volume.
In addition to refining the dataset, the code must also be optimized. This process includes debugging, optimizing the network architecture, and adjusting various mathematical components such as loss functions, which quantify the difference between predicted and actual values in a machine learning model, as well as solvers, which are algorithms that optimize the parameters of the model.
“As you improve your solution, you should diligently evaluate the performance of your entire system in a trial that reflects the real-world clinical applications,” Hong said.
