A validation team deploys an AI agent to draft IQ protocols for a new MES rollout. The agent produces 34 protocols in under an hour. The validation lead opens the first one. The structure is correct. The test steps reference the right functional specs. The acceptance criteria align with the URS.

The lead clicks “Approve.”

Then clicks “Approve” on the next one. And the next. All 34 protocols are approved in eleven minutes.

An FDA inspector pulls the audit trail six months later. Thirty-four approvals. Eleven minutes. No edits. No comments. No rejections.

The inspector writes the observation: the human review was not independent. The validation team delegated compliance accountability to an AI tool.

This is not a hypothetical. FDA issued observations in 2026 against firms that over-relied on AI-generated cGMP documentation without meaningful Quality Unit review. The guidance is now explicit: compliance accountability cannot be delegated to AI tools.

The fix is not to stop using AI. The fix is to build Human-in-the-Loop correctly.

What HITL Actually Means in CSV

HITL is not “having a person click approve at the end.” It is a structured, documented system where AI performs draft or assistive work, but qualified humans review, challenge, approve, and remain accountable for all GxP-relevant decisions.

The human is not validating the AI. The human is validating the business decision that the AI is recommending.

Under the FDA CSA guidance — finalized September 2025 and updated February 3, 2026 to align with the new QMSR folding ISO 13485:2016 into Part 820 — AI/ML tools used for production and quality management system purposes are explicitly in scope. On-prem, SaaS, IaaS, PaaS, analytics, automation, and AI tools. Not a gray area.

The February 2026 update matters because it confirms that your AI agents doing quality or CSV work are not operating in a regulatory vacuum. They are covered by the same risk-based framework that governs every other computerized system in your GxP environment.

The Five-Level Risk Tier

Not every AI function needs the same level of human scrutiny. CSA’s entire point is that effort should be proportionate to risk. Apply that logic to HITL:

L0 — Informational. AI summarizes an SOP. No approval needed. Log the action for audit trail purposes.

L1 — Low Risk. AI drafts meeting minutes or formats a report. Manager spot-checks on a sampling basis.

L2 — Medium Risk. AI generates an IQ protocol or drafts a traceability matrix. Validation engineer reviews, edits, and signs electronically.

L3 — High Risk. AI recommends test cases or classifies a deviation. CSV Lead reviews. QA approves. Two distinct humans.

L4 — Critical GMP. AI recommends a batch release decision, validation conclusion, risk acceptance, or deviation closure. AI cannot finalize. Human must approve. Often requires dual approval — SME plus QA.

The tier determines three things: how much of the output a human must review, whether a single person or two people must sign, and what the system does when confidence is low.

Confidence-Based Routing

Reviewing everything is wasteful. Reviewing nothing is negligent. The answer is routing based on confidence signals.

Confidence is not just the model’s token probability. It is a composite signal: retrieval quality from the knowledge base, consistency across multiple model runs, policy-rule checks, completeness against templates, and historical evaluation scores.

Here is the routing logic:

  • 95% or higher: Auto-continue. Logged for audit trail but no blocking human review.
  • 80–95%: Route to engineer for review. Single-click verification with optional comment.
  • 60–80%: Route to SME for full review. Mandatory rationale entry for accept or reject.
  • Below 60%: Reject and escalate. The output does not reach a human reviewer in its current form. The system regenerates or routes to a senior specialist.

This means in a well-tuned system, roughly 60–70% of AI outputs flow through with light-touch verification, and the remaining 30–40% get the intensive review they deserve. Review time drops from days to hours without sacrificing rigor.

The Policy Engine: Compliance Gate Before Human Review

Before any AI output reaches a human reviewer, it should pass through an automated policy engine that checks:

  • ALCOA+ compliance (attributable, legible, contemporaneous, original, accurate, complete, consistent, enduring, available)
  • Part 11 signature requirements configured
  • SOP references present and current
  • Traceability links maintained
  • Required document sections complete
  • No hallucinated references or fabricated data points

If the policy check fails, the human never sees the output. The system regenerates or flags for investigation. This dramatically reduces reviewer workload because they are not spending time catching structural defects that a rules engine can catch first.

Where the Human Sits in the Lifecycle

HITL is not a single checkpoint bolted onto the end of a process. It is woven through every phase of the validation lifecycle.

Requirements and Risk Assessment. AI parses URS documents and generates draft risk classifications. SMEs review and approve. The human defines intended use and risk tier — not the AI.

Traceability. AI builds a draft Requirements Traceability Matrix by matching requirements to test cases. Match confidence scores are displayed. High-confidence matches get single-click verification. Low-confidence matches trigger mandatory manual assignment. Unmapped requirements are flagged before the RTM locks.

Test Script Generation. LLMs draft test plans, positive and negative test scripts, and synthetic test datasets. An SME reviews against risk assessments, modifying parameters and adding domain-specific edge cases. Under CSA, unscripted testing — exploratory, scenario-based, error-guessing — is explicitly permitted for non-high-risk systems. This is where human exploratory testers provide unique value that AI cannot replicate.

Test Execution. Automated tools execute scripted or unscripted regression tests, capturing screen state and system logs. The human reviewer verifies pass/fail statuses and signs off on the execution package. The AI cannot “pass” a test case — only a qualified person can.

Deviation Handling. Anomaly detection scans execution logs and flags potential deviations. The system highlights the anomalous entry, displays historical root-cause matches, and suggests a classification. The Quality Specialist must review context, confirm or reclassify, write the impact statement, and approve the remediation plan.

Release. AI never has release authority. Period.

The Dual-Layer Audit Trail

Part 11 and Annex 11 compliance requires a dual-layer audit trail that captures both the machine’s activity and the human’s response.

Layer 1 — Machine Activity. Model parameters, prompt hashes, system inputs, raw AI outputs, confidence scores, model version, and timestamp. This lets you reconstruct exactly what the agent was capable of producing at the time of a specific approval.

Layer 2 — Human Action. Who reviewed, when they reviewed, explicit edit history showing diffs between the automated draft and the finalized text, reason for modification or approval, electronic signature with meaning, and timestamp.

The critical detail: both layers must be retained. You cannot overwrite the AI’s original output with the human’s edited version. The inspector needs to see both — what the machine produced and what the human did with it.

Rationale Cards: Explainability by Design

Do not present human reviewers with opaque recommendations. Every automated output should include a Rationale Card explaining why the system produced what it produced.

Example: “Suggested Test Case #12 because Requirement URS-42 references multi-factor authentication, matching ISO 27001 control criteria. Confidence: 87%. Source span: URS-42, paragraph 3, line 2.”

Rationale cards include the confidence band, highlighted evidence (the specific text spans or data points driving the recommendation), source citations, and alternative interpretations when the system considered them.

This is what allows an SME to meaningfully challenge the output. Without it, the reviewer is staring at a black box and the approval is theater.

Preventing Rubber Stamping

If your audit trail shows 100% approval in two seconds, an FDA inspector will conclude the human review was not independent. This is the single most common HITL failure mode.

The controls:

Forced rationale entry. Mandatory text fields for accepting or overriding low-confidence outputs. “Reviewed” is not a valid reason. The reviewer must state what they verified.

Blind-first assessment. For critical steps, require the human to enter their own assessment before seeing the AI’s suggestion. This prevents anchoring bias.

Blind challenge verification. Periodically inject known test variations — deliberately incorrect outputs — into the review queue. If the human approves them, you have a training problem or a rubber-stamping problem.

Review time tracking. Flag reviews completed in unusually short timeframes. Investigate. A human reviewing 200 test scripts in 3 minutes is not reviewing.

Override rate monitoring. If reviewers override AI recommendations less than 0.5% of the time, they are not reading them. If they override more than 15% of the time, the model is underperforming. Both thresholds trigger investigation.

Batch size limits. Cap the number of items in a single review session to prevent “approve all” behavior.

Structured review interface. Replace open text fields with discrete actions — Accept, Reject, Edit — each requiring a structured reason. This turns every review decision into structured data that can be analyzed for patterns.

Technical Architecture

A production HITL system for CSV requires four architectural layers.

Layer 1: AI Agents. Specialized agents for different tasks — CSV drafting, QA review, risk assessment, traceability mapping, policy compliance. Each agent critiques the others before output reaches a human. A multi-agent pipeline where the CSV agent’s output passes through the Traceability agent, then the Risk agent, then the Policy agent, means the human only sees work that has survived multiple automated reviews.

Layer 2: Review Orchestrator. The HITL gateway that routes outputs to the right reviewer based on risk tier and confidence score. Enforces workflow sequencing, escalation paths, and batch size limits.

Layer 3: Human Review Portal. A structured interface showing the AI output alongside rationale cards, source citations, confidence scores, and the diff from any previous version. Discrete accept/reject/edit actions with mandatory reason fields. Electronic signature integration.

Layer 4: Audit Trail and Evidence Pack. Immutable storage of both machine activity and human actions. Version-controlled prompt and agent configurations stored alongside human decisions. Exportable evidence packs for inspection.

No email approvals. No Slack approvals. A validated workflow engine with hard stops, role-based access, and Part 11-compliant e-signatures. This is the control that itself needs to be validated — test that the system blocks bypass attempts, that signatures lock records, that audit trails cannot be modified.

Validating the HITL System Itself

The HITL controls are part of the validated system. They must be tested.

Installation Qualification: Workflow version, user roles, e-signature configuration, and integration with the eQMS are verified.

Operational Qualification: Test that human gates cannot be bypassed. Test that Part 11 signatures lock records. Test that audit trails capture all required data points. Test the negative — what happens when a human rejects? What happens when the designated reviewer is unavailable? Test that confidence routing sends outputs to the correct queue.

Performance Qualification: Use actual end-users, not QA testers, with realistic scenarios. Inject seeded errors to prove humans actually catch them. Measure time to review, human error detection rate, and override rate. Run PQ with the same volume and complexity the system will face in production.

Document with a traceability matrix: Requirement → Risk → HITL Control → Test Case → Evidence. CSA encourages digital evidence — system logs, audit trails — instead of duplicative screenshots.

The Continuous Monitoring Loop

Validation is not a one-time event. Post-deployment, define and track:

  • Override rate per AI function (target: 2–15%)
  • Mean time to review per item (target: context-dependent, never < 2 seconds)
  • Inter-reviewer agreement (target: κ ≥ 0.8)
  • Low-confidence case volume and trends
  • Model drift indicators
  • Deviation rates linked to HITL effectiveness

Every human edit, approval, rejection, and comment becomes evaluation data. This feedback loop builds a golden dataset that continuously improves the system. Before deploying a new model version, it must perform at least as well as the previous one on these golden test cases.

If override rate exceeds 15% for two consecutive monitoring cycles, trigger automatic rollback and retraining under change control. If override rate drops below 0.5%, investigate whether reviewers are actually reading the outputs.

Model updates — new training data, hyperparameter tuning, architecture changes, prompt modifications, retrieval corpus changes — are changes to validated software. They require formal change control, re-qualification, and updated evidence packs. The locked-model approach remains the safest regulatory posture.

The Cultural Shift

The most underappreciated aspect of HITL is what it demands from the people in the loop.

Validation engineers and quality reviewers shift from manual document authors to risk analysts and decision stewards supported by interpretable AI. Training must cover not just the system but cognitive bias — automation complacency, anchoring, and the tendency to defer to a confident-seeming machine.

SMEs must understand why the system makes certain recommendations and the GxP impact of incorrectly overriding them. They must have the organizational authority and documented qualifications to make the decisions they are being asked to make.

The reported 32% reduction in validation cycle time from properly implemented HITL does not come from removing humans. It comes from redirecting their expertise to higher-value judgment calls while AI handles pattern recognition and draft generation.

The Bottom Line

Under FDA CSA, GAMP 5 2nd Edition, and the joint FDA-EMA AI guidance, the regulatory position is clear: automation and AI are not prohibited, but accountability is not transferable.

Build the risk tier. Place the gates. Route by confidence. Show the rationale. Capture the audit trail. Test the human part. Monitor the override rate. Retrain under change control.

This is what inspection-ready AI-assisted validation looks like. Not a human rubber-stamping machine output. Not a human redoing everything the machine did. A structured partnership where the machine does the heavy lifting and the human makes the decision that matters.