A model writes a one-page briefing on a company in thirty seconds. It cites a URL that doesn't resolve. It conflates two firms with similar names. It lifts a financial figure from a press release that was retracted three months ago. The customer has no way to know any of this without doing the research themselves — at which point what was the model for?
The fix isn't a smarter model. It's a QC layer that treats every claim as a hypothesis to be verified before delivery — not a sentence to be admired. What follows is the pipeline we run end to end: the agent that writes, the QC stage that checks, the codes that travel with every claim, and the three gates a report has to pass before a customer sees it.
The two-stage shape — Agent and QC
Two distinct stages, each doing one job. The Agent ingests sources, normalizes them into a structured entity record, and writes the report from that record. QC is a separate stage — a different prompt, often a different model — that re-reads the agent's output against the same entity record, runs a battery of mechanical checks, and produces an annotated diff: which claims hold, which don't, and which need to be re-run, escalated, or struck.
The agent never grades itself. QC never writes the report. Each does one job; both jobs get done. The combined output is an auditable record where every assertion has a verification trail attached.
QC stage one — source verification
QC starts before the agent has written anything. Every source extract clears six checks before the agent is allowed to use it — the layer most pipelines skip, and the one that catches schema drift and license issues before they propagate into reports.
- Schema check. Does the source data have the columns and types we expect? A renamed field is a silent break that quietly poisons every downstream report.
- Row-count band check. Did we get back roughly the volume we expected? A query that returns 12 rows when last week returned 1,800 is not a finding — it's a flag.
- License confirmation. Are we entitled to use this source for this engagement? Per-engagement licensing is normal in research; running a report on an unlicensed source is reputational risk that compounds.
- PII redaction. Sensitive fields that shouldn't appear in a report — and shouldn't even reach the model — are masked before the agent ever sees them.
- Freshness check. When was this data last updated? A claim built on a 14-month-old source is not the same claim as one built on a same-day source. Freshness gets attached to every record and travels with it.
- Cross-source corroboration. For each entity, does at least one independent source agree on the basics (name, location, identifier)? A source that disagrees with every other source on a primary identifier doesn't get used.
QC stage two — per-claim verification
Once the agent has written, QC re-reads each claim and runs five checks. This is the layer that catches the model-specific failure modes: fabricated URLs, summaries that subtly misrepresent the source, internal inconsistencies between the executive summary and the findings table.
- Citation existence. Does the URL or document the claim cites resolve at all? QC issues a live HTTP request at run time. A 200 means the citation is real; anything else is a flag.
- Citation faithfulness. Does the source actually say what the claim says it says? QC re-reads the source against the claim and checks for paraphrase drift, omitted qualifiers, and outright invention.
- Internal consistency. The number in the executive summary has to match the number in the findings table. Numbers that disagree across sections are the most common form of confident-but-wrong output.
- Recency relative to claim. A claim about a current state of affairs supported only by a four-year-old source is downgraded automatically.
- Out-of-scope drift. Did the report start drifting toward an entity or topic the brief didn't include? A report that quietly expands its scope is a report the customer didn't pay for.
The codes that travel with every claim
Every claim, every finding, and every source record carries three independent code-sets through the pipeline. Together they make every assertion in a report machine-auditable.
Severity
One of Critical, High, Medium, or Low. Visible in the deliverable; governs reader attention. Critical findings appear at the top of the report and require claim-by-claim sign-off from a named analyst before delivery. Low findings get spot-checked.
Confidence
A score on [0.00, 1.00], computed per claim from four factors:
- Number of corroborating sources that independently support the claim.
- Source reliability weight — a per-source multiplier that reflects how authoritative the source is for this kind of claim. (Government filings carry more weight on ownership than a press release does.)
- Freshness — how current the source is relative to the claim's assertion.
- Live citation pass — whether QC's HTTP check on the citation URL returned a 200 at run time.
The threshold for “citable in the report” is configurable per engagement — typically 0.70. Anything below the threshold is either re-run with elevated source weighting or struck with a methodology note that explains the exclusion.
Status
The lifecycle of a claim through the pipeline. Every claim ends in one of these states; the audit log shows when each transition happened and which check forced it.
pending— written by the agent, not yet reviewed by QC.supported— passed all per-claim checks against its primary source.corroborated— supported, plus at least one independent source agrees.unsupported— primary source doesn't back the claim as written. Routed for resolution.unverifiable— the source exists but QC cannot confirm or deny the claim against it. Escalated to a human.expired-source— citation no longer resolves, or content has changed in a way that no longer supports the claim.out-of-scope— the claim is true but outside the engagement's brief. Removed.struck-with-note— the claim does not appear in the report; the methodology section explains why.
The anatomy of one claim
The codes aren't metadata loosely associated with a report. They're a structured record attached to every claim, and they're what the audit log preserves so the same report can be re-verified months later. One claim, fully annotated, looks like this:
{
"claim_id": "clm_2026_04_24_0193",
"text": "The association closed 2025 with a $185,528 net operating loss on $2.73M revenue.",
"severity": "High",
"status": "corroborated",
"confidence": {
"score": 0.91,
"factors": {
"corroborating_sources": 2,
"source_reliability": 0.95,
"freshness_days": 38,
"citation_live": true
}
},
"primary_source": {
"id": "src_fin_pkg_2025_12",
"type": "financial_package",
"url": "https://docs.example.org/0L-excellence-dec-2025-redacted.pdf",
"checked": "2026-04-24T11:18:42Z",
"http": 200
},
"corroboration": [
{ "source_id": "src_akam_finance_memo", "agrees_on": "operating_deficit_174507" }
],
"qc_log": [
{ "check": "citation_existence", "result": "pass" },
{ "check": "citation_faithfulness","result": "pass", "note": "summary numbers reconcile to source page 4" },
{ "check": "internal_consistency", "result": "pass", "note": "matches exec summary figure" },
{ "check": "recency_relative", "result": "pass" },
{ "check": "out_of_scope_drift", "result": "pass" }
]
}The claim is supported, corroborated by a second source, and carries a 0.91 confidence score with the components broken out. The QC log lists every check that ran and the result. If the citation goes dark next quarter, the same claim re-run will transition to expired-source and the audit log will record exactly when the source disappeared.
What happens when a check fails — resolution codes
QC doesn't just flag, it routes. A failed check produces a resolution code that determines what happens next:
re-run— the same query against the same source again, often after a transient failure (HTTP 429, source 500).elevate-source-weighting— re-run with a higher-reliability source given more weight in the confidence calculation.escalate-to-analyst— QC can't decide; a named human reviews.strike-with-methodology-note— the claim doesn't appear in the report; the methodology section explains the exclusion.
The rule we apply across batches: if QC flags more than 30% of an entity's claims as unsupported, the report does not ship. The agent's prompt is iterated and the entity is re-run before the customer sees anything. A high failure rate on a single entity is information about the prompt, not about the entity.
“The model that writes the report should not be the model that grades it. Confidence is a code, not a feeling — and the report only ships when the codes line up.”
Two paragraphs, one pipeline
Easiest seen on a single passage. Same agent draft, before and after QC.
Agent draft
“The association closed 2025 with a sizeable operating loss and is currently in financial distress, with legal fees nearly tripling the budgeted amount. The current manager has been borrowing from reserve funds.”
No citations. “Sizeable” is qualitative. “Nearly tripling” is hand-wavy. The reserve-funds claim is unsourced and could be a model fabrication. None of this is verifiable.
After QC
“The association closed 2025 with a $185,528 net operating loss on $2.73M revenue [1]. Legal fees reached $281,001 against a $100,000 budget — a $177,025 overage [1]. The manager has borrowed $334,034 from restricted funds: $242,487 from the 2023 Special Assessment fund and $90,833 from reserves [1, 2].”
Severity tagged High; confidence 0.91; status corroborated across two independent sources. Each bracketed reference resolves to a citation in the report's source index, with a live URL check at run time.
Three gates before delivery
QC alone doesn't ship a report. The pipeline ships through three gates, in order:
- Gate one — sample review with the customer. A handful of representative reports go to the customer for markup before the batch ever runs. The customer's annotations become input to a versioned prompt revision; the batch runs only against an approved sample.
- Gate two — peer-reviewer stranger test. A second analyst with no authorship stake reads the sample cold. They catch what the lead analyst's eyes have already adjusted to.
- Gate three — named-analyst review. Before delivery, a named analyst reads every report cover to cover. Sign-off is claim-by-claim on Critical and High findings; spot-check on Medium and Low. The analyst's name appears on the cover page and in the methodology section. They are who the customer calls if a claim is challenged.
Each gate is rejection-capable. A sample that doesn't pass gate one doesn't advance to gate two. A batch that surfaces a systemic issue at gate three goes back for a targeted re-run. The customer doesn't see a report that hasn't passed all three.
How we think about it at RevenuePoint
Agent + QC + named-analyst review is not the shortest path to a finished report. It is the path that produces a report you can hand to a board, a procurement team, or a regulator without flinching. The agent ingests, structures, and writes. QC re-reads with mechanical discipline — six checks at the source, five at the claim, three code-sets attached to every assertion, a resolution router on every failed check. Three gates govern delivery. A named human signs the cover page. The report ships when, and only when, the codes line up.
