Two-Stage RAG: Standards First, Lab Facts Second

A manufacturer audits a fortified plant-based barista milk recipe for European V-Label vegan certification.

In a conventional RAG pipeline, the system takes the user’s broad query—“Does our beta-carotene colorant meet vegan certification rules?”—and searches the entire corporate archive at once. Fifty thousand documents compete for relevance in a single vector space: ISO standards, marketing brochures, factory sanitization logs, supplier delivery notes, and laboratory test reports.

The single-stage search engine returns five plausible chunks:

1. A marketing slide from the ingredient supplier describing the beta-carotene as “nature-identical and vegetarian-friendly.”
2. A generic excerpt from a food additive dictionary defining beta-carotene as provitamin A.
3. A factory intake receipt showing fifty kilograms delivered to the warehouse.
4. An allergen declaration stating “Manufactured in a facility handling milk derivatives.”
5. A fragment of ISO 23662:2021 mentioning general definitions of plant-derived foods.

The LLM receives this batch and generates an answer: “The beta-carotene is vegetarian-friendly and plant-derived; therefore it appears compliant, though shared-facility warnings apply.”

In production compliance, this answer is a catastrophic false positive.

The pipeline completely missed the actual legal bottleneck. Under V-Label and ISO 23662 standards, beta-carotene itself is rarely the point of failure. The failure almost always lies in its carrier matrix: oil-soluble beta-carotene crystals must be suspended in an emulsion, which frequently uses porcine gelatin or animal-derived glycerol as a stabilizing carrier. Furthermore, certification requires an explicit third-party Certificate of Analysis (CoA) proving the absence of animal DNA below 0.1 mg/kg.

The single-stage search engine failed because it tried to find the answer before establishing the criteria.

The Roller Coaster Inspector Analogy

Imagine a safety inspector dispatched to certify a massive amusement park roller coaster.

A single-stage inspector walks straight into a warehouse containing ten thousand unorganized maintenance binders. He pulls five random binders off the shelf, flips through pages about track paint, carriage grease, and ticket booth electrical wiring, and tries to guess whether the roller coaster is safe to open tomorrow morning.

A two-stage inspector behaves completely differently:

1. Stage 1 (The Rulebook): Before touching a single maintenance log, the inspector opens the municipal safety code book. He extracts the mandatory statutory requirements for high-speed steel coasters: “Coupling Pin #8 must have ultrasound stress-test certification dated within 90 days; braking pneumatic pressure must hold 120 PSI without dropping more than 2% over six hours.”
2. Stage 2 (The Targeted Evidence): Armed with that precise two-item checklist, the inspector walks into the maintenance room and pulls only the ultrasound test report for Coupling Pin #8 and the pneumatic pressure log from yesterday’s maintenance shift.

Where the analogy breaks down: a human engineer can memorize municipal safety codes over a twenty-year career. An automated compliance pipeline cannot rely on internal weights alone, because international standards update annexes, permissible carrier thresholds, and laboratory testing methodologies every year. The pipeline must retrieve the governing law explicitly before searching for evidence.

The Architecture: Decoupling Standards from Evidence

Industrial compliance RAG splits retrieval into two distinct, sequential operations:

User Query: "Audit Beta-Carotene Formulation #204"
                      │
                      ▼
┌───────────────────────────────────────────────┐
│ STAGE 1: Normative Standards Retrieval        │
│ Target: ISO 23662, V-Label Guidelines, Vegan  │
│ Society Standard                              │
│                                               │
│ Extraction LLM Output (Structured Schema):    │
│ {                                             │
│   "target_substance": "Beta-Carotene",        │
│   "carrier_restrictions": [                   │
│     "Gelatin prohibited",                     │
│     "Glycerol must be plant/synthetic"        │
│   ],                                          │
│   "mandatory_evidence": ["CoA", "HPLC"],      │
│   "max_animal_dna_ppm": 0.1                   │
│ }                                             │
└───────────────────────────────────────────────┘
                      │
                      ▼ (Dynamic Query & Filter Generation)
┌───────────────────────────────────────────────┐
│ STAGE 2: Forensic Supplier Evidence Retrieval │
│ Target: 50,000 Supplier CoAs, TDS, Lab Sheets │
│ Metadata Filters:                             │
│   batch_id == "LOT-9921"                      │
│   document_type IN ["CoA", "TDS_Addendum"]    │
│ Targeted Query:                               │
│   "carrier formulation gelatin glycerol HPLC" │
└───────────────────────────────────────────────┘
                      │
                      ▼
┌───────────────────────────────────────────────┐
│ STAGE 3: Deterministic Compliance Synthesis   │
│ Compare Stage 2 lab facts against Stage 1     │
│ legal constraints. Flag unverified criteria.  │
└───────────────────────────────────────────────┘

Stage 1: Formulating the Legal Checklist

The primary retrieval target is not the supplier database; it is the Normative Standards Store.

This store contains the authoritative regulatory texts: ISO 23662:2021 (definitions and technical criteria for vegan food), European V-Label Raw Material Guidelines, and specific retail compliance annexes.

A specialized extraction prompt evaluates the queried ingredient against the standards store, generating a typed schema:

from pydantic import BaseModel, Field

class VerificationCriterion(BaseModel):
    ingredient_name: str
    prohibited_processing_aids: list[str] = Field(
        description="Auxiliary substances forbidden during synthesis, e.g. bone char, gelatin"
    )
    carrier_restrictions: list[str] = Field(
        description="Mandatory origin specifications for suspension solvents"
    )
    required_lab_tests: list[str] = Field(
        description="Mandatory third-party analytical certifications, e.g. PCR, HPLC"
    )
    max_threshold_ppm: float | None = Field(
        default=None,
        description="Permissible unintentional cross-contact limit in parts per million"
    )

For beta-carotene, Stage 1 extracts:

• prohibited_processing_aids: ["fish gelatin", "porcine gelatin", "bone char"]
• carrier_restrictions: ["glycerol plant origin", "sunflower lecithin"]
• required_lab_tests: ["CoA", "ELISA-milk-protein"]

Stage 2: Targeted Forensic Retrieval

With the structured criteria in hand, the system no longer needs to run vague queries like “Is beta-carotene vegan?”

Instead, the pipeline constructs precise, targeted queries directed specifically at the Supplier Evidence Store:

1. Metadata Hard Filtering: Filter the document corpus by ingredient_code == "BC-204" and document_tier IN ["CoA", "TDS_Specification"].
2. Lexical/Semantic Hybrid Queries: Query specifically for the prohibited carriers and test types identified in Stage 1: "Beta-Carotene carrier matrix glycerol solvent source HPLC fatty acid profile"

Because the candidate pool is filtered by metadata and queried using the exact legal criteria, the retrieval engine bypasses marketing sheets and surfaces the exact two-page laboratory test sheet stating the botanical origin of the carrier matrix.

Production Implementation: Sequential Retrieval Pipeline

class TwoStageCompliancePipeline:
    def __init__(self, standards_retriever, evidence_retriever, criteria_extractor, auditor_llm):
        self.standards_retriever = standards_retriever
        self.evidence_retriever = evidence_retriever
        self.criteria_extractor = criteria_extractor
        self.auditor_llm = auditor_llm

    def audit_ingredient(self, ingredient_name: str, batch_id: str) -> dict:
        # Step 1: Retrieve relevant regulatory clauses
        standards_chunks = self.standards_retriever.search(
            query=f"Requirements and restrictions for {ingredient_name}",
            limit=5
        )

        # Step 2: Extract structured statutory criteria
        criteria = self.criteria_extractor.extract(
            ingredient=ingredient_name,
            standards_context=standards_chunks
        )

        # Step 3: Formulate targeted forensic search for supplier proof
        evidence_query = (
            f"{ingredient_name} carrier matrix "
            f"{' '.join(criteria.prohibited_processing_aids)} "
            f"{' '.join(criteria.carrier_restrictions)}"
        )

        evidence_chunks = self.evidence_retriever.search(
            query=evidence_query,
            metadata_filter={"batch_id": batch_id, "doc_type": "CoA"},
            limit=5
        )

        # Step 4: Synthesize compliance determination
        audit_result = self.auditor_llm.evaluate(
            criteria=criteria,
            evidence=evidence_chunks
        )

        return audit_result

Why Sequential Retrieval Prevents Hallucinations

When an LLM attempts to perform legal analysis and factual discovery in a single step, attention weights get smeared across irrelevant details. The model latches onto confident language (“100% pure,” “natural”) and misses the absence of an essential document.

Decoupling standards from evidence changes the dynamics:

• Missing Evidence Becomes Obvious: If Stage 1 demands an ELISA test for casein contamination and Stage 2 retrieves zero documents matching doc_type == "CoA", the system does not hallucinate a pass. It reports an explicit gap: “Missing mandatory lab certificate.”
• Explainability: Auditors can inspect the exact statutory clause pulled in Stage 1, alongside the specific supplier clause pulled in Stage 2.
• Latency Efficiency: Instead of passing 50 noisy documents through a reranker, the pipeline retrieves 3 authoritative rules, generates a laser-focused query, and pulls 3 verifiable evidence records.

Next Friday, in Chapter 8, we explore Hierarchical Chunking & Matrix Structure: why naive text slicing destroys complex ingredient tables (Ingredient → Supplier → Sub-ingredient → Carrier), and how parent-child chunking preserves table context.

---

Have you tested multi-stage retrieval pipelines where the output of an initial search shapes the query filters for a secondary search?