Saram ConsultingA pharmaceutical quality team feeds six months of SOPs, deviations, and CAPAs into an LLM extraction pipeline. A week later the graph looks beautiful — thousands of nodes, clean edges. Then an auditor asks a simple question about a CAPA edge: “Why is this relationship here? Which document says it?”
Nobody can answer. The edge has no source. The pipeline that produced it has no memory of where it came from. That single unanswerable question converts the entire graph from an asset into a liability — because in a GxP environment, knowledge that cannot be traced is knowledge that cannot be defended.
This is the problem hiding behind every “text → knowledge graph” project: not extraction, but trust. And trust is an architectural property, not a prompting problem.
Why naive extraction fails
The naive approach — “extract all entities and relationships from this text into JSON” — fails in predictable, structurally identical ways across every model and every domain. Auditing these failures reveals that true out-of-thin-air fabrication is actually rare. What pollutes graphs is more subtle:
- Schema drift. The LLM invents ad-hoc predicates (
is_partner_withvs.collaborates_with) instead of choosing from a defined ontology. Every invented predicate fragments the graph and breaks traversal logic. - Conflation. The relation is real, but attached to the wrong entity. “Satya Nadella — CEO of — Microsoft” extracted from a chunk about Microsoft gets attached to the node for Microsoft Ireland. This is an entity-resolution failure wearing a hallucination costume — and it is the #1 source of false edges in practice.
- Modality misreads. Negation (“X does not cause Y”), hypotheticals (“if the deviation is critical, an investigation is required”), and reported speech (“analysts believe…”) all get flattened into asserted facts. The model was faithful to the span — the span just should never have become an edge.
- Temporal staleness. “QA approves deviations” extracted from SOP v4 becomes a timeless fact, silently contradicting SOP v5.
- Transitive over-inference. The model infers A→C from A→B and B→C when no document ever states it.
The compounding problem makes this worse. Graph construction is a pipeline: per-stage 95% precision on a three-hop path yields roughly 86% path fidelity. And the error distribution is brutally long-tailed — 90% of triples are trivial and any model nails them; the last 10% (implicit relations, cross-chunk coreference, rare entities) is where 80% of the errors and 95% of the engineering time live.
A graph polluted by even a few percent of hallucinated edges silently degrades every downstream consumer: GraphRAG retrieval, impact analysis, compliance reporting. Precision is the product.
The golden rule: LLMs propose, deterministic infrastructure disposes
Every architecture that actually works in production converges on the same inversion of responsibility:
Never let the LLM write a graph edge. Let it produce evidence-bound claims, then let a deterministic system decide which claims are allowed to become edges.
The LLM is a hypothesis generator — powerful, fluent, and untrustworthy as a final arbiter of truth. The graph’s integrity must come from deterministic, verifiable layers around it. And the single most effective mechanism in the entire architecture is brutally simple:
No span, no edge. Every extracted triple must carry the exact character-offset span of source text it was derived from — not a free-text justification, a verifiable pointer. If the model cannot cite a span, the edge does not exist. This converts confabulation from a silent failure into a mechanical failure: it is much harder to invent a specific offset than a plausible-sounding relation.
The pipeline architecture
A production-grade extraction pipeline treats the LLM as one component in a staged funnel of cheap, deterministic filters feeding into progressively more expensive verification:
Raw Documents (SOPs, deviations, CAPAs, URS/FRS, test scripts)
│
▼
[1] Structure-aware Ingestion ── semantic chunks, span offsets,
│ doc ID + version retained
▼
[2] Ontology Registry ── closed entity/predicate sets, domain/range,
│ cardinality, risk tiers (versioned artifact)
▼
[3] Two-Pass Constrained Extraction
├─ Pass A: entities + typing (strict schema, structured output)
└─ Pass B: relations over verified pairs only (allowed predicates)
│
▼
[4] Grounding Gate
├─ span verification (verbatim substring match)
└─ NLI entailment (premise = chunk, hypothesis = triple)
│
▼
[5] Entity Resolution ── blocking → embeddings → LLM adjudication
│ → canonical IDs (mention→entity map)
▼
[6] Assembly ── contradiction mining, temporal scoping, cardinality checks
│
▼
[7] Risk-Based Human Review ── auto-promote / sample / adjudicate
│
▼
[8] Claim Ledger ── bitemporal, provenance-stamped commits
│
▼
Production Graph (Neo4j / RDF + SHACL)
│
▼
Agents · GraphRAG · Auditors (every edge explainable) 1. Ingestion: do not destroy the document
Before a single token is processed, the document must be parsed without destroying its structure. Chunk along semantic boundaries — sections, tables, markdown headers — not fixed token counts. Every chunk carries doc_id, doc_version, section_path, page, and stable character offsets. A coreference pre-pass (FastCoref or similar) replaces pronouns with explicit nominal mentions so chunks are self-contained.
This matters more than it sounds: an SOP’s regulatory authority lives in its version and effective date. A graph that discards “SOP-00123 v7, effective 2026-06-01” has discarded the ability to answer the only question that matters in a regulated environment — what was authoritative on a given date?
2. The ontology is the actual product
Clients think they are buying extraction. They are buying a schema decision. A slightly wrong ontology makes a 99%-precise extractor useless.
- Define the ontology as code — entity types, a closed predicate set, and domain/range constraints on every relation (
WORKS_FOR: domain Person, range Organization). Version it in git, review it like code. - Use constrained decoding (Outlines, XGrammar, vLLM/SGLang structured output, Instructor/Pydantic) so the model physically cannot emit a predicate that does not exist in the ontology. Schema drift becomes impossible rather than discouraged.
- Encode canonicalization decisions in the schema: “works_at” vs “employed_by” is decided once, here — not adjudicated per-chunk forever.
- Risk-tier every predicate:
low(auto-acceptable),medium(requires verification),high(never LLM-inferred — rules or humans only, e.g.,is_regulated_by). - Treat silent schema expansion as its own hallucination vector. Any new predicate the pipeline “wants” to propose routes to a governance queue, not into the ontology.
# Ontology as code — the LLM can only output what validates
class WorksFor(BaseModel):
source_type: Literal["Person"]
target_type: Literal["Company"]
relation: Literal["WORKS_FOR"]
evidence_span: str # must be a verbatim substring of source
confidence: float3. Two-pass extraction: entities first, relations second
Never extract entities and relations in a single generation step. The cognitive load is what causes the confusion.
- Pass A — Entity mention and typing. Extract typed entities matching the ontology, with surface form and character offsets. Deterministic NER (spaCy, GLiNER) handles high-precision types in parallel; the LLM handles ambiguous mentions and coreference.
- Pass B — Relation extraction over verified entity pairs. Given the entities from Pass A, the model predicts relations only from the ontology’s predicate matrix. If the ontology defines no relation between PERSON and REGULATION, the model cannot evaluate or output that edge.
This ordering prevents the most common failure: relation extraction inventing entities to connect.
4. The grounding gate: the anti-hallucination firewall
Every surviving candidate triple must pass all three layers before it is allowed near the graph:
- Syntactic grounding. The
evidence_spanmust be a case-sensitive, verbatim substring of the source text at the claimed offsets. No paraphrasing. A validator checks the offsets actually match. If not — drop. This single rule kills most fabrication. - Semantic entailment. Run a lightweight NLI model (DeBERTa-v3-large, self-hostable, an order of magnitude cheaper than an LLM judge) where the premise is the source chunk and the hypothesis is the verbalized triple. Reject anything that does not return entailment above threshold (0.85–0.90). This catches the paraphrase drift that span matching misses — including negation and modality, because “QA is considering MasterControl” does not entail “QA uses MasterControl.” The NLI model is a bounded classifier; it cannot hallucinate the way a generative verifier can.
- Ontological validation. Domain/range checks, cardinality (a person has one current employer), business rules (contract date cannot precede termination date). Enforced with Neo4j constraints or SHACL shapes.
Only triples passing all three layers get written. Below threshold, they are preserved as claims but never promoted.
5. Entity resolution: where naive pipelines die
Entity resolution requires global consistency, and LLMs process windows, not corpora. The architecture must compensate:
- Blocking — cheap candidate generation (normalized keys, embedding ANN). Never pairwise-compare everything.
- Embedding similarity for the obvious cases (
e5-mistral-7b,text-embedding-3-large). - LLM adjudication with evidence spans for the ambiguous middle band — never merge across type boundaries.
- Human review band for uncertain merges. The economics of the whole engagement are largely “minimize human review hours per 1,000 documents.”
- Never merge in place. Maintain a
mention → entitymapping layer with canonical IDs (ent_person_123) and alias tables. A bad merge becomes a one-line unmerge, not a re-extraction of the world.
6. Assembly: time, contradiction, and the assertion model
RDF’s bare subject-predicate-object is not rich enough for regulated knowledge. Every extracted statement needs an assertion model:
Assertion
├── subject / predicate / object
├── polarity (positive / negative)
├── modality (asserted / required / permitted / prohibited / hypothetical)
├── temporal_scope (valid_from, valid_to — propagated from document metadata)
├── conditional_scope
├── evidence (span, doc_id, doc_version, page, section)
└── provenance (extractor, prompt version, timestamp) This is what makes the graph represent change over time. SOP v4 says “QA approves deviations”; SOP v5 says “Quality Unit approves deviations.” A temporal graph holds both as versioned assertions with validity windows instead of a silent contradiction. It can answer the question that generic GraphRAG cannot: not just “what is true?” but “what was authoritative on June 15, 2025?”
Contradiction mining is a dedicated pass: conflicting edges trigger resolution by temporal precedence or source weighting — or route to human review. Contradictions are held, not silently resolved: store both statements with timestamps rather than overwriting.
7. Human review: risk-tiered, never exhaustive
Nobody is going to hand-review 5 million edges, and a system that requires it is economically dead. Route by risk:
LOW RISK → auto-promote
MEDIUM RISK → sample / secondary verification
HIGH RISK → human adjudicationEvery claim starts as CANDIDATE and moves through GROUNDED → ONTOLOGY_VALID → ENTITY_RESOLVED → CONTRADICTION_CHECKED → RISK_SCORED → APPROVED → GRAPH. The production graph contains approved knowledge, not whatever the LLM generated.
Active learning compounds the gains. Humans adjudicate the most informative uncertain edges, not random ones. Their rulings become the gold dataset; the gold dataset retrains the extractor and calibrates the verifier. The target is driving human review from ~40% of volume down to <5% over weeks as the loop compounds.
8. The claim ledger and provenance: the actual deliverable
Every edge in the graph stores its full biography: (subject, predicate, object, evidence_span, doc_id, doc_version, extractor_version, confidence, valid_from, valid_to). Think of it as a claim ledger — every edge is a signed entry, not an anonymous assertion.
The killer feature falls out of this naturally: every edge can answer “why is this edge here?” — returning the document, version, section, page, verbatim evidence text, extraction model, verifier result, and human approval state. And the system can also explain missing edges: “No unconditional Deviation → CAPA relationship was created, because SOP-123 v7 Section 8.2 establishes a conditional one.”
Provenance also makes edges retractable. When a source document updates, you can surgically retract and re-derive its edges — critical for long-lived production graphs that ingest continuously.
Evaluation: what you are actually selling
You cannot warranty a graph you cannot measure. The evaluation harness is built in week two, not week ten:
- A gold set of 500–1,000 human-annotated triples. Triple-level precision/recall is the only honest headline metric — and it must be tracked per predicate type, not in aggregate, because some predicates are far easier to hallucinate than others.
- An adversarial test set containing negation, hypotheticals, and reported speech — the cases that break naive extractors.
- Hallucination rate tracked as a distinct metric from recall. A pipeline can look great on F1 while still committing a small number of high-consequence false edges.
- Proxy metrics that should sit at ~100%: span-verifiability rate, schema violation rate, NLI pass rate.
- An error taxonomy (conflation / modality / temporal / schema / fragmentation), where each error class maps to a targeted upstream fix. This turns debugging from vibes into a checklist.
Well-executed pipelines hit 95%+ precision on grounded triples and hallucination rates below 5% — often below 1% on constrained domains. And “perfect, zero hallucinated edges” is asymptotic, not achievable. What you sell is known, tiered, measurable precision — a graph whose error rate is characterized, bounded, and continuously improving.
Fine-tuning: downstream, not upstream
The original instinct is always “fine-tune the model to extract better.” That is usually wrong — or at least premature. Fine-tuning against a moving schema is how projects burn six figures and restart.
The order that works:
- Bootstrap with constrained prompting + span grounding + human review. No fine-tuning. Get the ontology stable first.
- Accumulate adjudicated gold data from the human review loop.
- Then distill the verified extractions into a small, fast model (8B-class, Llama-3-8B or Qwen-2.5-14B) with LoRA. Use DPO with a preference dataset where chosen = strictly grounded extractions (high precision, even if recall dips) and rejected = ungrounded but plausible edges — teaching the model to output
[]when no valid relation exists rather than guessing. - Fine-tune the NLI verifier on domain-specific entailment pairs.
The fine-tuning is not for quality — it is for unit economics on the client’s permanent ingestion firehose, plus full on-prem/VPC deployment for regulated data. The frontier model remains the exception handler for difficult verification, not the whole architecture.
What this means for life sciences quality
This architecture is not generic advice — it maps directly onto the layer stack that regulated quality organizations are already building:
- Layer 1 — Master Data Fabric: canonical systems, suppliers, people, documents, requirements, tests.
- Layer 1.5 — Knowledge Evidence Fabric (the missing piece): mentions, candidate entities, candidate claims, evidence spans, NLI verification, contradiction detection, human review. This layer is the security boundary between probabilistic AI and authoritative knowledge.
- Layer 2 — Semantic Knowledge Graph: ontology, canonical entities, validated relationships, temporal graph, provenance, SHACL constraints.
- Layer 3 — AI Agents: change impact, validation, periodic review, audit trail — consuming trusted context from Layer 2, never writing to it directly. Agents propose mutations; a policy engine validates and approves them before any graph write.
The narrow MVP that proves the whole thing: a Validation Knowledge Extractor taking URS → FRS → risk assessments → test scripts → SOPs → validation summary reports, and outputting the Requirement → Risk → Control → Test → Evidence → System traceability chain — every relationship carrying source, page, section, span, document version, and validation status. Then the product question becomes: “Show me every requirement-to-test relationship and prove the evidence supporting it.”
That is a concrete, sellable capability — not “an AI that reads documents,” but “show me the authoritative relationships between my systems, processes, requirements, risks, controls, documents, tests, deviations, suppliers, and changes — and prove where every relationship came from.”
Where the effort actually goes
The deliverable is not a script; it is a running pipeline with an SLA. A realistic engagement effort split:
| Phase | Share of effort | Notes |
|---|---|---|
| Ontology discovery workshops | 15–25% | SME time, consensus-building, schema versioning |
| Gold dataset creation | 15–20% | Adjudication labor — also the moat |
| Pipeline engineering | 25–35% | Ingestion, extraction, grounding, resolution |
| HITL tuning + threshold calibration | 15–20% | Active learning loop, precision targeting |
| Eval harness + handoff | 10–15% | What makes it production-grade vs. a demo |
The moat is not the model. Anyone can swap GPT → Claude → Qwen in an afternoon. What competitors cannot easily reproduce: the ontology, the canonical entity registry, the evidence model, the claim ledger, the validation rules, the provenance layer, the human review history, the golden extraction dataset, and the regulatory semantics. Human review feeds the golden dataset, which feeds better extraction, which produces fewer human reviews — a flywheel that compounds into a domain-specific evaluation dataset nobody else has.
The bottom line
The naive pipeline — LLM → extract triples → load into Neo4j — produces a demo that impresses for ten minutes and collapses under the first auditor’s question. The production pipeline treats the LLM as what it is: a powerful but untrustworthy proposer, surrounded by deterministic infrastructure that grounds, verifies, resolves, and audits every claim before it becomes knowledge.
Every layer exists to make one question answerable: why is this edge here? If the answer is “the model was confident,” the system is a liability. If the answer is a document, version, section, verbatim evidence, and verification chain — the graph becomes an asset that compounds.
The research literature is converging on exactly this shape: constrained extraction, canonicalization, and post-generation validation rather than trusting raw LLM-generated triples (the Extract-Define-Canonicalize framework from EMNLP 2024; DOPKG-SHACL’s deterministic IRI minting and validation from ICCSC 2026). The winners in enterprise knowledge graphs will be the ones who treat extraction as an engineering discipline — and the losers will be the ones who treated it as a prompt.
Related: GraphRAG: When Vector Search Is Not Enough · Hybrid GraphRAG: Three Databases, One Truth · Graph Databases Deep Dive
Sources: Zhang & Soh, “Extract, Define, Canonicalize: An LLM-based Framework for Knowledge Graph Construction,” EMNLP 2024 (aclanthology.org/2024.emnlp-main.548); Uwasomba et al., “Deterministic and Trustworthy LLM-Driven Semantic Knowledge Graph Construction,” ICCSC 2026 (research.edgehill.ac.uk); W3C SHACL; W3C PROV-O.