Stop RAG Hallucination with Typed Schema Contracts

Retrieval-augmented generation was supposed to ground language models in real documents. It did not. What it did was hand a fluent storyteller a stack of source material and trust it to behave. The model still controls the output channel, which means it decides what to paraphrase, what to invent, and what to quietly omit. By the time the response lands in your product, the only way to catch a hallucination is to read the prose and cross-check every claim against the retrieved chunks by hand.

If you want to stop RAG hallucination, the fix is a typed contract, not a better prompt. Replace free-text generation with a schema that forces the model to commit each claim to a named, typed field, and you convert hallucination detection from a guessing game into a verification routine. The conversion has a cost. Strict typing degrades extraction recall on sparse documents, breaks on missing fields, and forces you to build explicit fallback behavior. This article is the engineering playbook for that trade.

Why Free-Text RAG Fails Programmatic Verification

Consider a concrete failure. A user asks an earnings assistant, "What was Acme's Q3 revenue and forward guidance?" The pipeline retrieves two chunks: one stating "Acme reported Q3 revenue of $4.2 billion" and another noting "the company declined to issue forward guidance this quarter." The model returns:

Acme's Q3 revenue was $4.2 billion, and the company projects next-quarter revenue of $4.6 billion.

The revenue figure is grounded in the retrieved context. The guidance figure is invented. Both sit in the same fluent paragraph, and nothing in the output distinguishes verified fact from fabrication. No unit test can catch this. You would need an LLM-as-judge pass, and those systems are themselves unreliable, slow, and expensive to run at scale.

The root problem is structural. Prose has no programmatic surface area. A unit test cannot assert against free text without brittle string matching. A downstream consumer cannot trust any specific claim, because there are no named fields to bind a contract to. The model's confidence is invisible. And when a hallucinated claim is plausible enough, no reviewer under load will catch it.

Structured outputs attack this directly, and they are the foundation of any credible effort to reduce RAG hallucination in production. When the model must return a JSON object with named keys, every claim becomes a queryable value. You can assert types, check nullability, compare against retrieved context field by field, and reject the response at the boundary instead of serving a fabrication. The hallucination surface area does not vanish, but it shrinks from "anything the model could write" to "any value the schema permits in a named field."

Three problems remain, and the rest of this article walks through each:

1. Hallucinated fields. Models fabricate values inside structured outputs too, especially when a required field has no document support.
2. Extraction failures on sparse documents. Strict schema enforcement can cause outright failures when the source document does not contain a value for every required field.
3. Unverifiable payloads. Validation only works if each field links back to an explicit source span, not just the JSON payload as a whole.

Defining a Typed Answer Contract

A typed answer contract is a schema that declares exactly what the pipeline is allowed to extract, what type each value must be, and whether each field is allowed to be null. Treat the schema as a binding specification. Each field pins the model to one fact it must either return or leave null, and every returned value is something you can verify against the source text. Pydantic is the dominant tool in Python because it gives you runtime validation, optional fields, and a JSON Schema export the model can be asked to follow.

A minimal contract for an earnings extraction pipeline looks like this:

from pydantic import BaseModel, Field
from typing import Optional, List

class Span(BaseModel):
    text: str = Field(description="Verbatim source span")
    chunk_id: str

class EarningsFact(BaseModel):
    company: str
    fiscal_quarter: str
    revenue_usd_millions: float
    revenue_source: Span
    guidance_next_quarter: Optional[float] = Field(
        default=None,
        description="Guidance figure if disclosed; null if absent"
    )
    guidance_source: Optional[Span] = None

Two things to notice. First, every factual field carries a parallel _source field typed as a Span. The model is not allowed to return a revenue number without also returning the verbatim source span it claims to be quoting. This is the hook for the citation validation step we build later. Second, guidance_next_quarter is Optional[float] with an explicit None default. The schema tells the model that omission is a valid answer. That single decision prevents the most common structured-output hallucination, which we cover later.

The schema is where the architecture lives. Everything downstream is plumbing. Get this wrong and the rest of the pipeline inherits the defect.

Enforcing Structured Outputs in Practice

Defining the contract is half the work. The other half is making the model follow it. Three enforcement mechanisms dominate production, and the right choice depends less on which is "best" than on which failure modes your pipeline can absorb.

Function Calling and Structured Outputs APIs

OpenAI's structured outputs API and Anthropic's tool use interface apply the schema constraint during decoding on the provider side. That makes them the most reliable option you can buy without running your own model. But "most reliable" is relative, because provider-side enforcement carries its own failure surface.

The risks that bite in production are subtle. Providers can silently coerce types when the model returns a value that almost fits, rounding a float or truncating a string to satisfy the schema without surfacing the mismatch. Schema support is uneven across providers, and some enforce a restricted subset of JSON Schema that rejects patterns you assumed were legal. Retry loops, which fire when the first attempt fails validation, can mask the root cause while quietly inflating latency. The cost and reliability tradeoffs across these modes are well documented, and raw JSON mode has shifted toward being the weakest option in most comparisons for hallucination-sensitive pipelines, because it guarantees only syntactic validity, not schema validity.

Constrained Decoding with Local Runtimes

If you control the inference stack, constrained decoding removes the provider failure class entirely. The schema grammar compiles directly into the sampler, so the model is mathematically unable to emit a token sequence that violates it. Research on structured outputs and constrained decoding validates the approach for production use, and libraries like Outlines make it accessible for self-hosted models.

The trade-off is operational, not reliability. Constrained decoding imposes a throughput penalty because the sampler masks invalid tokens at every generation step, and on large vocabularies that overhead adds up. Token-level restrictions can also produce outputs that are grammatically valid but semantically awkward, forced into strange phrasings to satisfy the grammar. For high-throughput extraction the trade is usually worth it. For low-volume or ad-hoc pipelines, the burden of running your own inference may exceed the reliability gain.

Orchestration with Instructor

The Instructor library wraps Pydantic, the provider APIs, and a retry loop into a single call that returns a validated instance or raises. It patches across OpenAI, Anthropic, and several local backends, so you can swap providers without rewriting your contracts.

Instructor is the right default when you cannot run your own inference, but it inherits every provider-side failure mode described above. Silent type coercion, schema-quirk rejections, and retry-loop latency all pass through untouched. What Instructor gives you is a clean abstraction over the retry loop and a consistent interface across providers, not immunity from their constraints. For pipelines that start from messy PDFs and office documents, LlamaParse handles the upstream parsing that feeds these schemas, because getting clean text into the contract is half the battle in real document intelligence work.

Mechanism	Reliability	Control	Operational cost
Provider APIs (structured outputs, tool use)	High, but silent coercion and schema-quirk rejections persist	Low, you depend on provider behavior	Lowest
Constrained decoding (Outlines, local)	Highest, schema valid by construction	Full, you own the sampler	Highest, throughput penalty and self-hosting
Instructor orchestration	Inherits the underlying provider's reliability	Low, abstraction over provider calls	Low, wraps retry logic for you

The decision is not which tool to pick. It is which failure class your pipeline can absorb. If silent coercion is tolerable, provider APIs plus Instructor get you to production fastest. If you need guarantees by construction and can absorb the throughput cost, constrained decoding is the only mode that removes the provider failure class entirely.

Stop RAG Hallucination with Field-Level Citation Checks

This is where most teams stop too early. They verify the JSON payload parses against the schema and call it done. That is necessary but insufficient. A payload can be schema-valid and still hallucinated. The model can confidently return revenue_usd_millions: 142.3 with a revenue_source span that does not exist anywhere in the retrieved chunks.

The verification step that matters is field-level citation matching. For every factual field, take the source span the model returned and confirm it appears verbatim, or with high fuzzy-match similarity, inside the retrieved context window. If the span is absent, the field is hallucinated, no matter how plausible the value looks.

A minimal verifier looks like this:

def verify_fact(fact: EarningsFact, retrieved_chunks: List[str]) -> bool:
    chunks_blob = "\n".join(retrieved_chunks)
    if fact.revenue_source.text not in chunks_blob:
        return False
    if fact.guidance_source and fact.guidance_source.text not in chunks_blob:
        return False
    return True

In production you want fuzzy matching, embedding similarity, and per-field policy. Some fields tolerate paraphrase, others demand verbatim spans. The literature on evaluating factuality in RAG covers the harder cases, and the RAGAS metrics include faithfulness and answer-relevance scores you can wire in as regression gates.

The principle is non-negotiable. Do not trust the JSON payload. Trust only field-level spans you can prove came from the retrieved context. Any field without a matching span is treated as unverified and routed to a fallback or human review queue. This is what makes LLM validation tractable in production. Without this step, you have not actually learned how to stop RAG hallucination. You have just relocated the fabrication problem into a slightly more structured box.

Handling Missing Data and Strict Mode Failures

Most structured-output guides skip the precision-recall cost entirely, but it is the part that breaks real pipelines. Strict schemas trade extraction recall for programmatic verifiability, and on documents where half the fields are genuinely absent, that trade becomes the dominant engineering problem.

The failure mode is mechanical. When a field is marked required and the document does not contain a value for it, the model has two options under strict enforcement. It can fail to return a valid payload, which surfaces as an extraction error. Or it can invent a value that satisfies the schema, which surfaces as a confident hallucination. Neither is acceptable in a production RAG pipeline.

The fix is to make every field that the document might not contain Optional with an explicit None default, and to instruct the model in the system prompt that null is a valid answer when the source does not support the field. The Instructor structured output guide walks through this pattern in detail. The pattern is the same across providers because the failure mode is the same.

Engineering Around Strict Mode

Three additional workarounds belong in any production playbook for handling missing fields in structured LLM outputs.

Tiered schemas. Define a strict schema for high-confidence extraction, and a looser fallback schema with all-optional fields. If the strict extraction fails after n retries, fall through to the loose schema and flag the result as low-confidence downstream. You keep precision where the document supports it without sacrificing recall on sparse inputs.

Retry with field-specific feedback. When Pydantic raises a validation error, route the error message back into the next prompt as context. Instructor does this natively. Field-level error feedback typically resolves single-field failures in one or two retries.

Explicit null policy. Decide per field whether a null means "document does not disclose" or "extraction failed." These are different conditions and they need different downstream handling. A disclosure-absent null is a valid answer. An extraction-failure null is a retryable error. Conflating the two is how pipelines silently lose data.

The precision-recall trade-off is the real engineering cost of structured outputs, and it is the part most concept-level articles omit. Strict schemas buy you programmatic verifiability. They cost you extraction completeness on documents that do not fit the schema's assumptions. The honest architecture decision is not "structured or free text," it is "which fields do I make strict, which do I make optional, and how do I route failures."

When the Overhead Pays Off

Typed schemas are not free. You pay in schema design, in provider-side schema constraints, in retry loops, in citation verifiers, and in the operational cost of handling null fields at every downstream consumer. For a chatbot where users tolerate occasional fabrication, that overhead is rarely justified. For a pipeline that feeds a database, a contract, a compliance review, or any system where a hallucinated value has real cost, the overhead pays for itself the first time it blocks a fabrication.

The architectural blueprint is consistent across the use cases where it matters. Define a typed contract with parallel source-span fields. Enforce it with structured outputs or constrained decoding. Validate each field's source span against retrieved context at runtime. Make every document-optional field nullable with explicit null semantics. Route verification failures and extraction failures to different downstream paths. Measure factuality regressions with RAGAS or an equivalent framework on every schema change.

Structured outputs will not prevent every RAG hallucination. What they do is convert hallucinations from an undetectable failure into a catchable one, and that is the only durable foundation for a RAG pipeline that has to be trustworthy.