The FDA will discuss a new framework for regulating generative-AI-enabled devices during a Digital Health Advisory Committee meeting this week.
The advisory committee is meeting Nov. 20–21 to address the balance between encouraging innovation and ensuring safety for this emerging technology that is reshaping the medical device industry.
According to the FDA, generative artificial intelligence technologies are increasingly integrated into healthcare, offering opportunities while posing unique regulatory challenges. The FDA’s executive summary for the advisory committee meeting outlines a Total Product Lifecycle (TPLC) approach to oversight, including premarket evaluation, risk management and postmarket monitoring.
What is generative artificial intelligence?
Generative artificial intelligence is a type of AI that creates new data by mimicking patterns learned from training datasets. Unlike traditional AI models designed for prediction or classification, generative AI generates text, images, audio, or other outputs that resemble the data it was trained on.
Commonly built on “foundation models,” these systems are versatile but lack the transparency of traditional AI, complicating their use in medical applications, the FDA says.
For medical devices, generative AI could streamline clinical workflows, improve diagnostic accuracy, and assist in medical training. For example, generative-AI-enabled devices might analyze complex medical images or generate synthetic data for research and simulation purposes. However, the adaptability of generative AI models introduces risks such as producing “hallucinations,” where the AI generates incorrect or misleading information.
“The complexity of the models, including model architecture and the large corpus of data typical of GenAI models, can be a factor that leads to such hallucinations. Thus, while a potentially notable benefit of GenAI is that it can generate outputs that are applicable to a specific area of interest from a variety of different data types, or that it can generate outputs that are relevant to a broad number of tasks, GenAI can also present potential risks that may require varying levels of risk controls for different applications, as is true of other technologies,” the FDA said in its executive summary about the Digital Health Advisory Committee meeting this week.
Implications for medtech
Generative AI applications in medtech range from administrative support, such as automating documentation, to advanced clinical tools capable of diagnosing diseases or personalizing treatment plans. Generative AI can potentially improve healthcare efficiency and accessibility, particularly in underserved areas, according to the FDA.
However, the FDA’s summary highlights several concerns. Generative AI models often rely on datasets that are too large or complex for developers to analyze fully, raising questions about data bias, reproducibility and ethical considerations. Because generative AI is constantly changing, it can be hard for traditional regulatory approaches to keep up. Foundation models used in generative AI may not have been designed with medical applications in mind, creating additional hurdles for ensuring their safety and reliability.
The FDA proposes a TPLC regulatory framework tailored to GenAI-enabled devices. This lifecycle approach includes:
- Premarket evaluation: Manufacturers must provide detailed descriptions of device design, training methodologies and data sources. The FDA seeks transparency on the foundation models used, especially when they are “unlocked” or capable of ongoing updates.
- Risk management: New controls may be needed to mitigate risks tied to GenAI’s open-ended outputs, including governance strategies and mechanisms for real-world feedback.
- Postmarket monitoring: Continuous device performance evaluation is critical, especially for models that adapt over time. This may include tracking accuracy, detecting bias and addressing regional variations in data.
