The Relentless Inspector: Deterministic Validators & Ready Score
After eight chapters of building an enterprise retrieval pipeline—curating embeddings, training linear projection adapters, fusing BM25 with dense vectors via RRF, reranking through cross-encoders, and parsing hierarchical tables—the system arrives at the final production milestone: generating the official compliance dossier for European V-Label certification.
At this stage, most engineering teams make a critical architectural mistake: they hand the final synthesis prompt to an LLM and treat its generated text as the source of truth.
The LLM receives thirty retrieved chunks from supplier specifications, Certificates of Analysis (CoA), and ISO standards. It writes a fluent, professional five-page summary:
“Conclusion: The formulation for Barista Oat Drink #402 meets all criteria outlined in ISO 23662:2021. All emulsifiers, stabilizers, and flavorings have verified plant or mineral origins, and processing aids comply with non-animal thresholds.”
The audit director reviews the dossier, signs the formal submission, and sends it to the European certification authority.
Three weeks later, the submission is rejected with a formal violation notice.
Buried inside Table 4 of the supplier addendum was a clarifying agent: Bentonite Clay Solution (Stabilized with 0.02% Lysozyme from egg whites). The LLM read the phrase “Bentonite Clay (Natural Volcanic Ash)”, recognized volcanic ash as a mineral, and breezed past the secondary enzyme carrier without flagging the egg derivative.
The generative model hallucinated compliance because large language models are probabilistic text predictors, not statutory auditors.
The Bank Vault Time-Lock Analogy
Imagine an armored transport truck carrying currency to a federal reserve depository.
The driver of the truck is skilled, alert, and experienced. The driver navigates heavy downtown traffic, swerves around construction hazards, and coordinates with dispatch.
Once the truck reaches the depot, the driver does not decide when the steel vault doors swing open. The vault is secured by a mechanical, spring-loaded time-lock. The time-lock has zero intelligence. It cannot drive, it cannot read a map, and it does not listen to friendly conversation. If the mechanical dial does not strike exactly 12:00:00, the door remains shut.
Where the analogy breaks down: a driver might find a mechanical time-lock inconvenient. In regulated software engineering, placing a deterministic mechanical lock after a generative model is the only reason the system can be legally trusted.
Inverting the Architecture: The Model Drafts, Deterministic Code Decides
To achieve zero-defect compliance dossiers, the system must invert the traditional AI workflow:
Retrieved Context (Stage 1 Standards + Stage 2 Lab Facts)
│
▼
┌────────────────────────────────────────────────────────┐
│ LLM SYNTHESIS STEP (Constrained Structured Extraction) │
│ - Outputs Pydantic typed schema │
│ - Maps findings to exact document quote anchors │
│ - Strictly forbidden from outputting freeform verdicts │
└────────────────────────────────────────────────────────┘
│
▼
┌────────────────────────────────────────────────────────┐
│ DETERMINISTIC VALIDATION GATES (Pure Code, Zero LLM) │
│ Gate 1: Banned Additive Hard Lookup (E-numbers / CAS) │
│ Gate 2: Regex Analytical Threshold Assertions │
│ Gate 3: Cryptographic Lineage & Timestamp Verification │
└────────────────────────────────────────────────────────┘
│
▼
┌────────────────────────────────────────────────────────┐
│ SECTION READY SCORE EVALUATION │
│ Score < 95% ──► Automated Targeted Retrieval Retry │
│ Score >= 95% ──► Dossier Signed & Audit-Ready │
└────────────────────────────────────────────────────────┘
The LLM is treated as an extraction worker. Its sole responsibility is parsing unstructured supplier prose into a strict, strongly-typed data schema:
from pydantic import BaseModel, Field
class IngredientFinding(BaseModel):
ingredient_name: str
cas_number: str | None = None
e_number: str | None = None
carrier_substance: str | None = None
stated_origin: str
evidence_quote: str = Field(
description="Exact verbatim sentence from supplier CoA"
)
document_id: str
Once the model populates this schema, the generative pipeline terminates. Execution shifts to deterministic software that enforces hard rules.
The Three Deterministic Guardrails
1. Banned Additive Hard Lookups
The system maintains an in-memory set of prohibited European food additives and CAS numbers (e.g. E120 Carmine, E904 Shellac, E913 Lanolin, E966 Lactitol, CAS 1390-65-4).
A deterministic Python validator checks every extracted finding against the blacklist:
BANNED_ADDITIVES = {
"E120": "Carmine / Cochineal (Insect origin)",
"E904": "Shellac (Lac insect secretion)",
"E913": "Lanolin (Sheep wool grease)",
"E966": "Lactitol (Milk whey derivative)",
}
def validate_prohibited_substances(findings: list[IngredientFinding]) -> list[str]:
violations = []
for item in findings:
code = item.e_number.upper() if item.e_number else ""
if code in BANNED_ADDITIVES:
violations.append(
f"FATAL: Prohibited additive {code} ({BANNED_ADDITIVES[code]}) "
f"detected in {item.ingredient_name} [Doc: {item.document_id}]"
)
return violations
If a banned code appears anywhere in the record, the pipeline halts with an unrecoverable rejection. The model cannot argue, negotiate, or rationalize the substance as “trace amounts.”
2. Regex Threshold Verification
When standards require laboratory proof—such as verifying that unintentional milk cross-contact remains below 0.1 mg/kg under ELISA testing—the pipeline uses strict regular expressions against the verbatim quote:
import re
THRESHOLD_PATTERN = re.compile(
r"(?:ELISA|PCR|HPLC)\s*:\s*<\s*(0\.\d+|[0-9]+)\s*(?:mg/kg|ppm)",
re.IGNORECASE
)
def verify_lab_threshold(quote: str, max_allowed_ppm: float = 0.1) -> bool:
match = THRESHOLD_PATTERN.search(quote)
if not match:
return False
value = float(match.group(1))
return value <= max_allowed_ppm
If the supplier text says “negligible traces” instead of quoting a numeric test value matching the pattern, the test fails deterministically.
The Section Ready Score
To quantify dossier completeness before submitting to auditors, the pipeline calculates a Section Ready Score ($R$) for each ingredient formulation.
The score weights four verifiable criteria:
ReadyScore = (w1 * C_CoA + w2 * P_lab + w3 * O_carrier + w4 * A_sanit) / Total_Weight * 100
Where:
C_CoA: Binary check confirming a valid Certificate of Analysis is linked (Weight: 30)P_lab: Binary check confirming analytical laboratory test passed regex threshold (Weight: 30)O_carrier: Binary check confirming solvent carriers have verified botanical origin (Weight: 25)A_sanit: Binary check confirming shared-line sanitization protocol is signed (Weight: 15)
def calculate_ready_score(ingredient_data: dict) -> float:
weights = {"has_coa": 30, "lab_passed": 30, "carrier_verified": 25, "sanitization_signed": 15}
total_weight = sum(weights.values())
earned = (
(weights["has_coa"] if ingredient_data["has_coa"] else 0) +
(weights["lab_passed"] if ingredient_data["lab_passed"] else 0) +
(weights["carrier_verified"] if ingredient_data["carrier_verified"] else 0) +
(weights["sanitization_signed"] if ingredient_data["sanitization_signed"] else 0)
)
return (earned / total_weight) * 100
The Threshold Rules
- Score = 100%: Section is certified audit-ready.
- Score between 85% and 99%: Triggers an automated targeted re-retrieval query focused strictly on the missing component (e.g. querying supplier archives specifically for “ELISA casein test addendum”).
- Score below 85%: Flags the ingredient in an auditor escalation dashboard with an exact actionable ticket: “Supplier spec missing carrier provenance.”
The Complete Engineering Architecture
Across nine chapters, we built an end-to-end industrial RAG system designed for zero-defect compliance:
- Chapter 1: Why base LLMs crash-test on multi-ingredient recipes.
- Chapter 2: The geometry of ingredients and cosine negation blindness.
- Chapter 3: Benchmarking 5 embedding models on chemical food specs.
- Chapter 4: Training a 56 MB MLP projection adapter for domain alignment.
- Chapter 5: Hybrid search fusing BM25 exact codes with vectors via RRF.
- Chapter 6: Cross-encoder reranking to catch fine-print carrier contradictions.
- Chapter 7: Two-stage retrieval separating normative standards from supplier facts.
- Chapter 8: Hierarchical tree chunking preserving nested formulation tables.
- Chapter 9: Deterministic regex guardrails and Section Ready Scores.
By placing deterministic code at the exit of the pipeline, the system transforms AI from an unpredictable generator into an accountable compliance instrument.
This concludes our 9-part series on Engineering RAG for Humans. Next, we turn to the European regulatory horizon with our upcoming series on the EU Digital Omnibus & AI Act—connecting new legal mandates, sovereign local models, and audit lineage to practical software engineering.
What deterministic checks do you place downstream of your LLM outputs to catch hallucinations before they reach production users?