A quality director at a mid-size biopharma recently walked me through their AI stack. They had a private LLM endpoint. A Langfuse instance capturing traces. A RAG pipeline pulling from their EDMS. An evaluation suite with 160 golden test cases. A policy document saying “human review required for GxP outputs.”

Every individual component was solid. None of them were connected into a system.

The LLM had no idea what the policy document said. The evaluation suite tested the RAG pipeline in isolation, not end-to-end. The Langfuse traces captured latency but not ALCOA+ compliance attributes. The human review process existed on paper but had no code path that enforced it. When the FDA inspector asked, “Show me exactly what data and prompt version your agent used to draft CAPA-1234 on March 4th,” nobody could answer.

They had components. They did not have an architecture.

This is the most common failure mode in regulated AI today: teams build excellent individual pieces — a harness, an observability layer, an eval suite — but never compose them into a single, integrated system where every request flows through all seven layers from identity to audit commit. The result is gaps that look fine on a slide deck and collapse under inspection.

The Core Paradox

Traditional CSV assumes deterministic systems. Same input, same output, every time. That assumption is the legal foundation of every validation protocol ever written.

AI agents break all of it. They reason probabilistically. They chain tool calls in emergent sequences. They retrieve context that changes between runs. They produce different outputs from identical prompts.

The architectural response is not to make the LLM deterministic — that is impossible. The response is to wrap the non-deterministic reasoning inside a deterministic, validated framework where every state transition is logged, every tool call is gated, every output is checked, and every human decision is signed. The LLM becomes an untrusted sub-component inside a trusted system.

This is the reference architecture for that system.

The 7-Layer Reference Architecture

┌──────────────────────────────────────────────────────────────────┐ 
│  LAYER 7 — EVALS & VALIDATION                                    │ 
│  Golden datasets · Regression suites · LLM-as-judge              │ 
│  CI/CD gates · Continuous monitoring · Drift detection           │
├──────────────────────────────────────────────────────────────────┤ 
│  LAYER 6 — OBSERVABILITY                                         │ 
│  OpenTelemetry traces · Immutable audit trail · WORM logs        │ 
│  Prompt/response capture · Cost/latency metrics · Dashboards     │ 
├──────────────────────────────────────────────────────────────────┤ 
│  LAYER 5 — COMPLIANCE & GOVERNANCE                               │ 
│  21 CFR Part 11 e-signatures · ALCOA+ enforcement                │ 
│  Policy engine · Change control · Version registry               │ 
├──────────────────────────────────────────────────────────────────┤ 
│  LAYER 4 — AGENT ORCHESTRATION (THE HARNESS)                     │ 
│  Deterministic DAG · Supervisor → Planner → Executor             │ 
│  HITL gates · Tool registry · Guardrails · Confidence routing    │ 
├──────────────────────────────────────────────────────────────────┤ 
│  LAYER 3 — MODEL LAYER                                           │ 
│  Private LLM (VPC) · Prompt registry · RAG router                │ 
│  Model versioning · Fallback endpoints · Token budgets           │ 
├──────────────────────────────────────────────────────────────────┤ 
│  LAYER 2 — DATA & KNOWLEDGE                                      │ 
│  Controlled RAG · Vector DB with lineage · WORM storage          │ 
│  Read-only connectors · Document versioning · Hash verification  │ 
├──────────────────────────────────────────────────────────────────┤ 
│  LAYER 1 — IDENTITY & ACCESS                                     │ 
│  SSO (SAML/OIDC) · MFA · RBAC/ABAC · Session management          │  
│  Electronic signatures · Unique user IDs · Least privilege       │ 
└──────────────────────────────────────────────────────────────────┘

Each layer has a specific owner, a specific risk profile, and specific controls. A request enters at Layer 1, passes through every layer, and exits at Layer 7 as an auditable, attributable record. No shortcuts. No bypasses.

Layer 1: Identity & Access — The Entry Gate

Everything starts with 21 CFR Part 11’s requirement for unique user identification and access controls. Before a user even touches the agent, identity must be verified and bound to the session.

The identity layer integrates with enterprise SSO (EntraID, Okta) via SAML or OIDC. It enforces MFA. It applies RBAC with GxP-specific roles — a QA Reviewer sees different data and has different tool permissions than a Manufacturing Operator or a Regulatory Affairs Specialist. Session tokens propagate through the entire execution trace.

This is not optional infrastructure. It is the foundation of Attributable — the first principle of ALCOA+. Every downstream record must carry the identity of who initiated the request.

For electronic signatures, the system requires two-component authentication (password plus a second factor) with printed name, date/time, and signature meaning (review, approval, authorization). The LLM cannot generate or apply signatures. Only a validated e-signature gateway can.

Key design decision: If the agent calls downstream systems (QMS, LIMS, ERP), it should propagate the authenticated user context — not operate through a single privileged service account. Every action must be attributable to a specific human, even when the agent is the intermediary.

Layer 2: Data & Knowledge — The Controlled Context

The agent’s knowledge comes from approved, version-controlled sources. Not the internet. Not an uncontrolled wiki. Not yesterday’s embedding of a document that was revised this morning.

The data layer connects to systems of record — Veeva Vault QMS/EDMS, LIMS, MES, ELN, SAP — through read-only, validated connectors. Every document retrieved carries metadata: source_doc_id, version, effective_date, hash. The agent must be able to reconstruct exactly what it saw and when.

The vector database is not the source of truth. It is an index that references the source of truth. Every chunk stores its lineage. When a source document is updated in the EDMS, the vector index must be updated atomically — if the agent retrieves an outdated SOP, it violates ALCOA+ Original and Accurate.

For agent outputs that become GxP records, WORM (Write Once Read Many) storage applies. S3 Object Lock, Azure Immutable Blob, or equivalent. The agent writes once. Nobody — not even an admin — can modify or delete the record.

Layer 3: Model Layer — The Reasoning Engine

The LLM runs in your VPC. No public API calls for GxP data. Azure OpenAI with Customer Managed Keys or AWS Bedrock with VPC endpoints. The model is a replaceable inference engine — the harness provides the intelligence.

System prompts, RAG templates, and tool definitions are versioned in Git with checksums. They are part of your Design Specification (DS) under GAMP 5. A prompt change is a code change. It goes through PR review, automated testing, and explicit approval before deployment.

The RAG router assembles context from approved sources. Retrieved records keep their source lineage — every claim in the agent’s output should be traceable to a specific document version. Context is never directly trusted; it passes through the compliance layer before reaching the user.

A model registry tracks every deployed model version with its model card: intended use, limitations, validation status, known failure modes. Model upgrades go through formal change control. No auto-updates in production.

Layer 4: Agent Orchestration — The Harness

This is where most teams fail FDA inspection. The harness is the deterministic control plane that wraps the probabilistic LLM. It is not a framework feature. It is a GAMP 5 Category 5 custom application.

Use a deterministic DAG (LangGraph, Semantic Kernel, Temporal) — not an autonomous ReAct loop. The orchestrator controls execution flow:

Supervisor Node: Classifies intent and GxP criticality. Routes to appropriate workflow. Assigns GAMP risk level (1-3).

Planner Node: Generates a validated tool plan from a closed tool registry. No dynamic tool discovery. The agent calls only pre-approved, IQ/OQ’d tools with defined inputs, outputs, and failure modes.

Policy Engine: Hard guardrails before any tool call. Examples: “Agent may never close a Deviation — only draft investigation summary.” “Agent may not provide dosing recommendations.” “All outputs for regulatory submission require HITL.”

HITL Gate: For any GxP-critical action, route to a human with e-signature requirement. The gate is a blocking state transition — the workflow does not proceed until a qualified human approves.

Confidence Routing: If the agent’s confidence falls below threshold, or if the query is novel (outside validated use cases), or if the action is high-risk — escalate to human or fallback to a deterministic path.

Fail-Closed Policy: When the agent cannot determine a safe path, the system fails closed (no automated action) rather than proceeding with uncertain outputs.

Every state transition is an immutable event. Every tool call is logged with input, output, latency, and attribution. The harness is the audit boundary — if the harness cannot be validated, nothing it produces is defensible.

Context Firewalls: Each subagent or workflow step receives only the context it needs. The Evidence Reviewer never sees the CAPA Drafter’s raw context. This is not a compute optimization — it is an audit boundary. Clean context isolation means each step’s output is attributable to a bounded, reviewable input set.

Tool Permissions: Capability-based, not role-based. A tool that reads deviation records and a tool that drafts CAPA records have separate credentials, even if the “role” (quality reviewer) is similar. This keeps your Part 11 access control story clean.

Layer 5: Compliance & Governance — The Enforcement Layer

This is the cross-cutting layer that spans everything. It is the ALCOA+ and Part 11 enforcement mechanism.

ALCOA+ Data Integrity Engine:

Principle Technical Control
Attributable user_id, agent_id, model_version, prompt_hash on every record
Legible Structured JSON logs plus human-readable render of prompt and response
Contemporaneous NTP-synced server timestamps at event creation, not at review
Original Store raw retrieval and raw LLM completion before parsing. No overwrite.
Accurate Grounded RAG with citations, HITL review, eval validation
Complete Full chain: Intent → Plan → Tool Calls → Evidence → Final Answer
Consistent Deterministic harness, version-locked models, prompts, and tools
Enduring WORM storage with defined retention (10+ years for GxP)
Available Redundant storage, inspector-ready export, legal hold support

Change Control: Any change to prompts, model weights, tool schemas, guardrail rules, or data sources triggers formal change control — risk assessment, testing, documentation, and approval before deployment.

Policy Engine: Declarative policies (OPA, Cedar, or custom DSL) that are version-controlled and tested. Policies define what the agent can and cannot do, which data it can access, and which actions require human approval.

Layer 6: Observability — The Witness System

Standard APM is not enough. A hallucination returns HTTP 200. Observability must capture both structural execution steps and semantic language model telemetry.

Four Pillars:

  1. Traces: OpenTelemetry from UI → Harness → LLM → Tools. Every agent run is a trace with spans for each step. Use Langfuse or Arize Phoenix for LLM-specific observability.

  2. Logs: Immutable prompt and completion logging. Log the full rendered prompt, not just the template. PII redaction after archival copy.

  3. Metrics: Tool selection accuracy, hallucination rate, HITL escalation rate, latency, token cost per GxP transaction.

  4. Records: Separate SIEM stream for security plus Data Integrity Monitoring. Someone trying to modify the audit store triggers an immediate alert.

Every trace is an ALCOA+ record. It must contain: user identity, session ID, model version, prompt version, input hash, output hash, timestamp of generation versus timestamp of human approval, and the specific human reviewer identity for the approval gate.

The dashboard must answer one question: “Show me exactly what data and prompt version agent 2026-001 used to draft CAPA-1234 on March 4th.” If your observability system cannot answer that, it is not observability — it is logging.

Drift Detection: Monitor three types of drift. Input drift (embedding distribution shift). Output drift (response quality degradation over time). Knowledge drift (source documents updated but vector index not refreshed). A CAPA severity triage agent that silently shifts its distribution of “major” versus “minor” classifications is a bigger regulatory risk than one that throws an error — errors get caught by the human gate, drift does not.

Separation of Concerns: Operational telemetry (latency, errors, token usage) serves engineering. Audit-grade records (who, what, when, why, with what sources) serve compliance. These should not be conflated in the same dashboard, even though they flow through the same collector.

Layer 7: Evals & Validation — The Continuous Proof

Evals are not optional testing. They are your OQ/PQ. Under CSA (Computer Software Assurance), you test based on risk. Unscripted “vibe checks” fail.

Three Tiers:

Tier 1 — Offline Eval Harness (Pre-deployment): Golden datasets of 100-200 SME-curated test cases. For a Deviation Agent: historical deviations with approved investigations. Data must be versioned. Test for:

  • Tool Selection Accuracy (did it pick the right LIMS query?)
  • Citation Faithfulness (is every claim traceable to a retrieved GxP document?)
  • Instruction Following (did it follow the system prompt?)
  • Schema Conformance (did the output match the expected JSON schema?)
  • ALCOA+ Compliance (can it attribute source? does it hallucinate lot numbers?)
  • Safety (jailbreak resistance, prompt injection from ELN fields, data exfiltration attempts)

Use LLM-as-Judge for semantic quality — but the judge model must itself be validated and its prompts locked.

Tier 2 — Online Monitoring (Post-deployment): Drift detection on retrieval relevance, escalating hallucination rate, human override rate. All HITL corrections flow back as new golden data. Canary testing on live traffic samples.

Tier 3 — Periodic Review: Required by EU Annex 11. Quarterly review of audit trails, eval trends, model and prompt change log. Access recertification. Incident and CAPA review.

Eval as CI/CD Gate: No production deployment without passing the eval suite. Evals run on every change — prompt, model, tool, or data. If scores drop below threshold, the deployment blocks. This is the automated OQ/PQ that CSA enables.

Golden Dataset Governance: The eval dataset must have its own change control record, independent of the prompt version history. Otherwise you end up in a position where you cannot prove the eval set was not tuned to fit the prompt.

The Compliance Map

Every layer maps to specific regulatory requirements:

Regulation Architectural Implementation
21 CFR Part 11 (Audit Trails) Immutable, append-only logs at Layer 6. Every prompt, completion, tool call, seed, and model version captured. WORM storage with cryptographic hashing.
21 CFR Part 11 (E-Signatures) Human approval at Layer 4 HITL gate. Re-authentication via OAuth2/MFA. Printed name, date/time, meaning. Record-bound.
21 CFR Part 11 (Access Controls) SSO, MFA, RBAC at Layer 1. Unique user IDs. Session timeout. Separation of duties.
ALCOA+ Attributable Correlation ID per agent run. user_id, agent_id, tool_id on every record.
ALCOA+ Legible Structured JSON logs plus human-readable render at Layer 6.
ALCOA+ Contemporaneous NTP-synced timestamps at event creation. No backdating.
ALCOA+ Original Raw retrieval and raw LLM output stored before any transformation at Layer 2.
ALCOA+ Accurate Grounded RAG with citations at Layer 3. HITL at Layer 4. Evals at Layer 7.
ALCOA+ Complete Full chain from intent to final answer. No silent filtering.
ALCOA+ Consistent Deterministic harness at Layer 4. Version-locked models, prompts, tools.
ALCOA+ Enduring WORM storage at Layer 2. 10+ year retention. Regular integrity checks.
GAMP 5 Category 5 Full validation lifecycle. URS → FS → DS → RTM → IQ/OQ/PQ. Change control at Layer 5.
CSA Risk-based eval depth at Layer 7. Automate low-risk verification. Focus testing on high-risk judgment.

What to Build First

Not the autonomous batch release agent. Not the deviation investigation agent. Not the pharmacovigilance signal detector.

Start with SOP Q&A. Read-only. Approved documents only. Citations required. No record creation. Full audit trail. Basic eval suite. Human feedback loop.

This is Phase 1. It exercises all 7 layers at low risk:

  1. Identity — authenticate the user, enforce RBAC
  2. Data — retrieve from approved SOP corpus, verify document versions
  3. Model — use a pinned, private LLM with a versioned prompt
  4. Orchestration — deterministic retrieval → generation → citation check pipeline
  5. Compliance — log every step with ALCOA+ attributes
  6. Observability — capture full trace, store in WORM
  7. Evals — run golden test set on every prompt change

Build the compliance sidecar before the agent. If you cannot audit it, you cannot validate it.

Phase 2: Draft assistance — deviation narratives, CAPA summaries, investigation support. Human review required. No autonomous submission.

Phase 3: Workflow assistance — create draft records via validated APIs. Propose workflow steps. Maker-checker controls.

Phase 4: High-risk decision support — only after mature controls, strong evals, drift monitoring, quality oversight, and periodic review.

The Agent Is Only 20% of the Platform

This is the most important architectural insight, and the one most teams resist.

The AI agent — the LLM, the reasoning loop, the tool calls — is the smallest component. Everything around it is what makes it acceptable in a GxP environment:

  • Identity (who is asking)
  • Data (what does the agent know)
  • Compliance (what rules constrain it)
  • Observability (what did it do)
  • Evals (how well did it do it)
  • Governance (who approved it, what changed)

The agent itself is a replaceable reasoning engine. The harness, the observability, and the evals are the validated system. Swap the LLM from GPT-4 to Claude to a fine-tuned Llama — the architecture does not change. The compliance controls do not change. The audit trail does not change.

This separation is what makes revalidation feasible. When the model changes, you re-run the eval suite and update the model registry. You do not re-validate the entire platform.

Anti-Patterns That Kill Audits

These are not theoretical risks. They are the specific patterns that produce FDA 483 observations and warning letters:

Using public OpenAI for GxP data. Your prompts and completions may be logged by the vendor. Your data leaves your network boundary. You cannot guarantee data residency. Use a private endpoint with Customer Managed Keys or a local model.

Agent writes directly to QMS/LIMS without HITL. The agent generates a draft. A human reviews and signs. The record is created through the validated system of record. The agent never has direct write access to GxP databases.

Uncontrolled prompts. Prompts stored in UI code, not version-controlled, not change-controlled, not tested. A prompt is a Design Specification artifact. It belongs in Git with checksums.

No correlation ID. If you cannot trace a single request from user input through every tool call, model invocation, and output check to the final audit record, you do not have an audit trail. You have logs.

Dynamic tool discovery. The agent calls arbitrary Python, discovers tools at runtime, or accesses unapproved APIs. The tool registry is closed. Every tool is a GAMP 5 component with its own spec.

LLM-controlled routing. An autonomous ReAct loop that decides which tools to call and in what order, with no deterministic oversight. Use a state machine or DAG. The LLM suggests; the harness decides.

Treating embeddings as source of truth. The vector database is an index. The EDMS is the source of truth. If they diverge, the EDMS wins. Always.

No kill switch. You must be able to disable a specific agent, a specific tool, or a specific use case in production within minutes, not days.

The Technology Stack

This is illustrative, not prescriptive. The architecture is technology-agnostic. But these are the components that work:

Capability Recommended Alternatives
Orchestration LangGraph Semantic Kernel, Temporal, PydanticAI
Observability Langfuse + OpenTelemetry Arize Phoenix, custom OTel collector
Vector DB pgvector (PostgreSQL) Milvus, Qdrant, Pinecone
LLM Serving Azure OpenAI (CMK) or Bedrock VPC vLLM, Ollama for local models
Policy Engine OPA (Open Policy Agent) Cedar, custom DSL
WORM Storage S3 Object Lock Azure Immutable Blob
Model Gateway LiteLLM Portkey
Eval Framework DeepEval + custom OpenEvals, Ragas
CI/CD GitHub Actions with eval gates GitLab CI, Harness.io

The Version Equation

A production agent version is not just the code version. It is the complete configuration snapshot:

agent_version =
  code_version
  + prompt_template_version
  + model_version
  + knowledge_index_version
  + tool_schema_version
  + guardrail_policy_version
  + eval_threshold_version

Every component is versioned independently. Every combination is traceable. When an auditor asks “what was running on March 4th?”, you produce this exact snapshot.

The Bottom Line

The architecture that survives FDA inspection is not the one with the best LLM or the flashiest UI. It is the one where every request flows through all seven layers — from identity verification to immutable audit commit — with no bypasses, no shortcuts, and no unlogged state transitions.

Build the compliance sidecar before the agent. Lock the stack before building the use case. Start with read-only SOP Q&A. Run evals on every change. Treat prompts as code. Make the harness deterministic. Make the audit trail immutable.

The agent is replaceable. The architecture is not.

Deep technical research notes: [[AI Agent Reference Architecture for Regulated Life Sciences: Comprehensive Compiled Report]]

Related: [[AI Agent Playbook for Regulated Life Sciences]] · [[Engineering the AI Agent Harness]] · [[Evaluating AI Agents in Life Sciences CSV]]