Comment from George Evgrafov
AnonymousSupportIndividual
Summary: The commenter suggests that the FDA should explicitly include guidance on the use of publicly available AI foundation models, such as ChatGPT and Gemini, in the draft guidance. They argue that because these models lack transparency and can change frequently, the FDA should provide specific requirements for monitoring, risk assessment, and documentation to ensure regulatory consistency.
Explicit Mention of Publicly Available AI Foundation Models
The draft omits discussion of publicly available AI foundation models, which represent a growing class of tools used by sponsors and service providers in regulated environments. Common examples include ChatGPT (OpenAI), Google Gemini, Claude (Anthropic), Mistral, and Meta LLaMA. These models are often used to analyze, summarize, or classify regulated content or real-world data.
Recommendation: I recommend that the FDA explicitly address the use of pre-trained, publicly available AI models in regulated contexts. The lack of transparency in these models (e.g., undisclosed training data, unknown model architecture), coupled with the vendors’ ability to make changes to the models at any time, creates significant risk. When sponsors use such models, it is critical to establish processes to maintain control, including:
•Monitoring for changes and updates by the model provider
•Performing periodic risk assessments
•Executing regression testing to ensure continued fitness for purpose
•Keeping detailed documentation of the model’s behavior, limitations, and role in the regulated process
Explicit guidance would ensure consistency and clarity across the industry and help organizations using foundation models adopt appropriate controls to maintain the validated state of their systems.