Saram Consulting📦 GitHub Repository: Reference implementations, validation templates, and CSV agent test harnesses discussed in this playbook are open-sourced on GitHub: github.com/saram-io/csv-agents
A QA director at a mid-size biotech recently described her AI strategy in one sentence: “We have six agents running in production. None of them went through change control.” She is not unusual. Shadow AI agents are proliferating across life sciences — built by scientists, deployed on personal API keys, writing to validated systems without a single line of validation documentation.
The regulatory response is no longer theoretical. FDA’s January 2025 draft guidance on AI to support regulatory decision-making introduced a 7-step credibility framework. The FDA-EMA joint publication of 10 Guiding Principles of Good AI Practice landed January 14, 2026. EU GMP Annex 22 (draft, July 2025) targets AI in manufacturing directly. And ISPE published a 290-page GAMP Guide: AI in July 2025 — the most comprehensive industry standard for AI validation to date.
This playbook distills what regulators are actually enforcing, what the validation frameworks demand, and what engineering teams need to build. It is not aspirational. It is operational.
The Core Rule: Agents ≠ Tools
This is the single most important distinction in the entire space, and most teams miss it.
An AI agent that autonomously chains reasoning, tool calls, memory, and action to operate on GxP data is a GAMP Category 4/5 computerized system. You do not validate just the LLM. You validate the entire agentic system: prompts, RAG pipeline, embeddings, tool wrappers, guardrails, and audit trail.
The system boundary for validation includes:
| Component | What You Validate |
|---|---|
| Prompt layer | System prompt + few-shot examples + tool definitions (versioned) |
| RAG pipeline | Chunking strategy, retriever precision/recall, citation fidelity >95% |
| Tool wrappers | Input/output schema, error handling, idempotency |
| Guardrails | Jailbreak resistance, PII leakage = 0, hallucination detection |
| Audit trail | 21 CFR Part 11 — who (user + agent ID), what, when, why, original value |
If your validation scope ends at “the model works,” you will fail inspection. The auditor does not care about the model’s benchmark scores. They care about the system you deployed.
Risk Tiering — Do This Before Anything Else
Risk tiering is the artifact auditors ask for first. It determines how much validation rigor everything downstream requires. Get this wrong and you either over-validate (burning months on low-risk chatbots) or under-validate (leaving GxP-critical agents uncontrolled).
Decision Tree:
Is data GxP?
→ Does agent make autonomous decision or write?
→ Does it create e-record?
→ If YES to any = Tier 1The Four Tiers
| Tier | Name | Examples | Validation | Human Oversight |
|---|---|---|---|---|
| 1 | Critical (Direct GxP) | Batch record review, AE intake, eCTD authoring, manufacturing control | Full GAMP 5/CSA, Part 11, Annex 11 | HITL + Human-in-Command |
| 2 | High (Indirect GxP) | Deviation triage, protocol drafting, signal detection, literature monitoring | Risk-based validation | Human-on-the-Loop |
| 3 | Moderate (Operational) | SOP drafting, meeting notes, competitive intelligence | Functional validation | Periodic audit |
| 4 | Low (Non-GxP) | Internal HR bot, general knowledge Q&A | IT security review | Periodic review |
The risk score formula: Severity × Probability × Detectability Gap. Score each dimension 1-5. A score above 50 means Tier 1 controls. A score below 12 means Tier 3-4 is sufficient.
Most teams start Tier 3 and work up. That is the right sequence.
Architecture Blueprint
A GxP-compliant agent platform has six layers. Every production agent should pass through all of them.
UI Layer (SSO, RBAC, e-signatures)
↓
Orchestration Layer (Supervisor / Planner / Tool Router, version-pinned)
↓
Guardrails Layer (Policy engine, PII/PHI filter, prompt injection defense,
hallucination detector, citation enforcement)
↓
Model Layer (Model Registry, pinned LLM/SLM, Model Cards, embedding versioning)
↓
Tools & Data Layer (ELN, LIMS, Veeva QMS, CTMS via read-only wrappers, validated APIs)
↓
Assurance Layer (ALCOA+ audit logger, immutable logs, traceability)The Six Golden Rules
- No direct write to GxP system without HITL approval. The agent drafts; the human approves. No exceptions for Tier 1.
- Deterministic tool wrappers. The LLM provides reasoning. All output passes through deterministic validation rules — schema checks, fact verification, constraint validation. Critical calculations must be re-verified by code, not by the LLM.
- No open internet RAG for GxP content. Ground agents in curated, version-controlled knowledge bases. Approved SOPs, validated protocols, approved product labels — not Reddit.
- Everything versioned. Models, prompts, tools, embeddings, knowledge bases. If you cannot reproduce the exact configuration that produced a specific output two years ago, you do not have a validated system.
- Separation of reasoning and action. Never allow an agent to directly execute consequential actions without passing through a validated control layer.
- Explicit confidence signaling. Low-confidence outputs automatically route to human review. Define thresholds: >0.85 auto-proceed (Tier 3-4 only); 0.60-0.85 human review recommended; <0.60 human review mandatory.
The HITL Gateway
Human oversight is not a single binary. It has four levels:
| Level | Behavior | When to Use |
|---|---|---|
| Inform | Agent executes; human notified | Tier 4 only |
| Confirm | Agent proposes; human approves | Tier 2-3 |
| Collaborate | Agent and human co-create; dual sign-off | Tier 1-2 |
| Restrict | Human executes; agent provides recommendations only | Tier 1 critical decisions |
The automation bias warning: A review gate that exists procedurally but isn’t measured for rubber-stamping is not a control. If your human reviewer approves 100% of AI outputs, flag them for audit. Track time-on-task. Reviews completed in under 30 seconds are not reviews.
What Regulators Are Actually Enforcing
The regulatory landscape has consolidated around three documents that matter most right now:
FDA Draft Guidance (January 2025) — “Considerations for the Use of AI to Support Regulatory Decision Making for Drug and Biological Products.” Introduces a 7-step credibility framework: define question, define context of use, assess risk, establish credibility evidence, evaluate adequacy, document, monitor. This is not optional guidance. This is what FDA reviewers will reference during inspections.
FDA-EMA Joint: 10 Guiding Principles of Good AI Practice (January 14, 2026) — The first joint FDA-EMA AI document. Covers the full medicines lifecycle. Key principles: human-centric by design, risk-based approach, clear context of use, data governance and documentation, lifecycle management.
ISPE GAMP Guide: AI (July 2025) — 290 pages of AI/ML-specific validation guidance. This is the industry standard that QA teams will use to build their validation programs. Covers model validation, data quality, change control, and monitoring for AI systems in GxP.
Add to these EU GMP Annex 22 (draft, July 2025) targeting AI in manufacturing, and the FDA’s CSA (Computer Software Assurance) final guidance from September 2025 — which explicitly states that CSA principles apply to AI tools in production or quality systems.
The direction is unambiguous. Regulators expect AI agents to be validated, versioned, monitored, and human-supervised.
Validation: What You Actually Test
The old CSV approach — exhaustive documentation of every possible input/output — is dead. CSA replaced it with risk-based critical thinking. But “risk-based” does not mean “skip validation.” It means focus your testing on what matters most.
The GxP SDLC Mapped to FDA’s 7 Steps
| FDA Step | Phase | Key Artifacts |
|---|---|---|
| Define question | Concept | Use Case Charter, Intended Use Statement |
| Define Context of Use | Risk Assessment | Context of Use statement, FMEA |
| Assess Risk | Classification | Tier assignment, Severity × Probability × Detectability |
| Establish Credibility Evidence | Build & Evaluate | URS, Design Spec, Data Curation Spec, Prompt version control |
| Evaluate Adequacy | Verification | Functional testing, data integrity, robustness, red-team |
| Document | Validation | Validation Plan, Trace Matrix, Test Protocols, Summary Report |
| Monitor | Operations | PQ, UAT, Continuous Monitoring Plan, drift detection |
What the Test Suite Must Cover
Prompt testing: Version control every prompt. Regression test after every change. Test prompt injection resistance — quarterly, with updated attack techniques.
RAG testing: Measure retriever precision and recall against a golden dataset. Verify citation fidelity >95%. If the retriever returns the wrong chunk, the agent will confidently hallucinate a correct-sounding answer grounded in the wrong source.
Tool testing: Validate input/output schemas. Test error handling — what happens when the LIMS API is down? Test idempotency — does the agent create duplicate records on retry?
Guardrail testing: Test jailbreak resistance with known attack vectors. Test PII/PHI leakage — the target is zero. Test hallucination detection against fabricated references and unsupported claims.
Audit trail verification: Every action must log: timestamp (UTC, synchronized), user ID + agent ID, action type, input parameters, output, model version, prompt version, knowledge base version, confidence score, human oversight action. This is 21 CFR Part 11.
Revalidation Triggers
You must revalidate when: model version changes, prompt changes materially affect behavior, data sources change, retrieval index changes significantly, performance drifts, a new safety incident occurs, regulatory requirements change, intended use expands, or vendor terms change.
If your team cannot articulate their revalidation triggers, they do not have a validated system.
ALCOA+ for the Agentic Era
ALCOA+ was designed for traditional e-records. It applies to agent actions, with an extension:
| Principle | Agent Interpretation |
|---|---|
| Attributable | To both human AND agent ID — unique agent credentials, no shared accounts |
| Legible | Structured output with clear reasoning chain |
| Contemporaneous | Timestamped reasoning trace at every decision point |
| Original | Raw tool outputs preserved, not just final synthesis |
| Accurate | Grounded in retrieved, approved sources with citations |
| Complete | Full chain-of-thought summary + citations + confidence scores |
| Consistent | Same input → same output range (documented variance) |
| Enduring | Retained per regulatory requirements (minimum 7 years) |
| Available | Accessible for inspection on demand |
The “attributable” requirement is the one most teams violate first. If your agent runs on a shared API key with no per-user attribution, you have a Part 11 violation on day one.
The Anti-Patterns That Get 483s
These are the patterns that show up in warning letters. Every one of them is avoidable.
- Shadow agents. Scientists building agents outside IT/QA oversight, using personal API keys, writing to validated systems. The most common and the most dangerous.
- Autonomous write to QMS/EDMS without HITL. The agent updates a deviation record or batch record without human approval. This is a Part 11 violation.
- Unversioned prompts in production. If you cannot reproduce the exact prompt that generated a specific output, you cannot reconstruct the decision during an inspection.
- ChatGPT web for submission drafting. Uncontrolled environment, data sent to third-party servers, no audit trail. Three violations in one action.
- RAG without source citations. An agent that generates regulatory or scientific content without citing its sources is indistinguishable from a hallucinating model.
- No monitoring plan post go-live. Deployment without continuous monitoring is not a validated system — it is a hope.
- Validating the vendor, not your system. Your vendor’s SOC 2 report does not validate your implementation. You validate the system you deployed, including your prompts, your RAG pipeline, your tool integrations, and your guardrails.
Agent Charter: The Minimum Artifact
Every agent in production needs a charter. Not a slide deck. Not a Confluence page no one reads. A controlled document that answers:
- What is it allowed to do?
- What is it prohibited from doing?
- Who reviews and approves its output?
- What evidence shows it works safely?
- How do we detect and correct failure?
The charter includes: agent name and ID, business/technical/quality owners, risk tier, autonomy level, intended use, prohibited uses, data sources, data classification, systems accessed, tools available, human oversight requirements, approval workflow, logging requirements, performance metrics, safety metrics, failure modes, escalation path, fallback process, vendor dependencies, model version, prompt version, validation status, and review date.
If any field is unclear, the agent is not ready for production.
Implementation Roadmap
Most regulated organizations should not attempt everything at once. The roadmap has four phases:
Phase 0: Foundation (Months 0-3) Establish the AI Governance Committee. Draft the acceptable use policy. Build the risk tiering model. Inventory existing AI use. Select low-risk pilots.
Phase 1: Pilot (Months 4-6) Deploy Tier 3-4 agents: SOP Q&A, literature summarization, meeting notes. Validate lightly. Collect performance data. Build organizational confidence.
Phase 2: Scale (Months 7-12) Deploy Tier 2 agents with full validation: deviation summarization, PV narrative drafting, regulatory intelligence. Implement continuous monitoring. Establish model registry. Conduct first internal AI audit.
Phase 3: Advanced (Months 13-18) Evaluate Tier 1 use cases with rigorous risk-benefit analysis. Implement multi-agent architectures. Conduct external audit simulation. Engage regulators proactively.
The organizations that try to skip to Phase 3 are the ones that get 483s. The organizations that build the foundation first are the ones that scale.
The Bottom Line
AI agents in regulated life sciences are not a technology problem. They are a quality system problem that happens to use AI. The agent must fit the quality system — not the other way around.
The playbook is clear: risk-tier every agent, validate the entire system (not just the model), version everything, enforce human oversight with measurable controls, log every action to ALCOA+ standards, and monitor continuously. The regulatory framework is no longer emerging. It is here.
The companies that master this disciplined approach will gain the real competitive advantage — faster development cycles, higher quality submissions, and inspection-ready evidence — while the ones that skip governance will spend their next audit cycle explaining why their AI was writing to validated systems without change control.
Start with risk tiering. Start today.
Full compiled research with templates, RACI matrices, agent charter templates, and inspection checklists: [[AI-Agent-Playbook-Regulated-Life-Sciences-Compiled-Report-2026]]