Comment from George Evgrafov
AnonymousSupportIndividual
Summary: The commenter suggests that the FDA clarify that human-in-the-loop oversight does not exempt AI components from rigorous validation requirements. They argue that models must be validated for accuracy, reliability, and traceability to ensure risks are not underestimated in regulatory decision-making.
Human-in-the-Loop Does Not Remove Need for Validation
It is common for organizations to assume that having a human involved in reviewing or confirming AI outputs eliminates the need for validation. This assumption can be problematic, particularly when the AI influences or filters what a human sees.
Recommendation: The FDA should clarify that the presence of human-in-the-loop oversight does not negate the requirement for validation or credibility assessment of the AI component. If AI outputs are used in any capacity that could influence regulated data or decisions, the model should be validated for:
•Accuracy, reliability, and consistency
•Appropriateness of integration into human workflows
•Traceability of human vs. AI-generated content in audit trails
This clarification will help ensure that risk is not underestimated and that compliance expectations are consistently applied.