# One Search Call for Five Claims Is Not a Fact-Check

> Batching all claims into one search call shares ten low-context results across every claim. A per-claim design with a cited URL and today's date fixed a gate that was stripping real facts.

- Author: Dil Ahdan (https://dil.dev/)
- Published: 2026-11-07
- Updated: 2026-11-07
- Tags: n8n, fact-checking, llm, content-pipeline
- Canonical: https://dil.dev/notes/batched-fact-check-shared-search-results/

## Key takeaways

- Batching claims into one search call means all claims are judged against the same results, which may be dominated by a single vendor.
- Each claim needs its own request, its own cited URL and today's date injected, and search_context_size high rather than low.
- Fail-closed merging means an unparseable verdict counts as unverifiable, not verified.
- Parallel per-claim calls hit rate limits; 1.5 seconds between requests keeps the pipeline stable.
- GitHub commit PUTs are never retried because a retry can land a duplicate commit or race into a 409.

Batching all your claims into one search call and calling it a fact-check is not a fact-check. It is a shared pool of ten results that may have nothing to do with half the claims in the batch. Execution 17546 made that concrete: five claims, one call, ten results, eight of them from a single vendor's own site, and the gate returned two verified, zero contradicted, three unverifiable. The revision stripped every external fact from the article.

## What the Old Design Actually Did

The original Fact-check node, ported from the Lab Overflow engine, sent every new or changed claim in a single sonar-pro call. The call used `search_context_size low`, which returns ten results. Those ten results were shared across all claims in the batch.

There was no `TODAY` in the request. There was no cited URL. The model had no date anchor and no source to check against, just an open-ended search and whatever happened to rank.

The same design ran the Lab Overflow engine for eleven weekday articles before it was caught and fixed there on 23 October. CargoMode-web inherited the bug when the node was ported.

## What Execution 17546 Showed

Five claims went in. Ten results came back. Eight of those results were from one vendor's own site. The other vendor in the article had zero results in the pool.

The gate verdict: two verified, zero contradicted, three unverifiable. Because the unverifiable count exceeded the threshold, the revision pass stripped every external fact from the article rather than publishing claims it could not support.

The facts were real. The search pool was just wrong.

## The Per-Claim Design

The fix is in commit `fd954be4` (2026-10-24, CargoMode-web). Three things changed together.

**Extract Claims now returns a `cited_url` with each claim.** The node's `maxTokens` was raised from 4000 to 6000 to give it room to include the source alongside the claim text. The cited URL travels with the claim into every downstream node.

**One request per claim, sent one at a time.** Each request gets:

- `TODAY` injected so the model has a date anchor
- the claim's own `cited_url` so the model has a concrete source to check against
- `search_context_size high` instead of low

Requests are spaced 1.5 seconds apart. Parallel calls hit 429 rate-limit errors. The spacing is a small cost for a clean run.

**Merge Verdicts is fail-closed.** If the response for any claim cannot be parsed, that claim is counted as unverifiable, not verified. A broken response never silently passes a claim through the gate.

The threshold itself is unchanged: zero contradicted, at most one unverifiable.

## The A/B on the Same Five Claims

Running the old and new designs against the same five claims from execution 17546:

| Design | Calls | Results | Verified | Contradicted | Unverifiable |
|---|---|---|---|---|---|
| Batched | 1 | 10 | 1 | 0 | 4 |
| Per-claim | 5 | 50 | 3 | 1 | 1 |

The per-claim run found a contradiction the batched run missed entirely. It also verified three claims the batched run could not support. The one remaining unverifiable is within the threshold, so the article publishes with the contradiction flagged for review.

Fifty results across five targeted searches beat ten results shared across a batch. The math is not subtle.

## Why the Cited URL Matters as Much as the Count

More results help, but the cited URL is what makes each search meaningful. Without it, the model searches broadly and returns whatever ranks. With it, the model can check the claim against the actual source and then look for corroboration or contradiction elsewhere.

A claim about a specific product version checked against the vendor's changelog page is a different operation from the same claim checked against whatever ten pages happen to rank for a loose query. The cited URL makes the request specific.

This connects to a broader pattern in the pipeline. The [LLM capture job that invented dates](/notes/llm-capture-job-invented-dates/) showed what happens when a model has no date anchor. The same principle applies to source anchors: without one, the model fills the gap with whatever is available, and whatever is available may be wrong.

## The GitHub PUT Rule

Commit `fd954be4` also adds a rule that GitHub commit PUT nodes are never retried.

The reason is straightforward. A PUT that succeeds on the first attempt but whose response is lost will look like a failure. Retrying it sends the same content again. That either lands a second commit or races the first into a 409 conflict.

Neither outcome is recoverable cleanly. A failed PUT is a signal to investigate. It is not a signal to try again automatically.

This is different from most HTTP failures in the pipeline, where a retry is safe because the operation is idempotent or the failure is clearly transient. A commit PUT is neither. The content is the same but the state of the repository has already changed.

## What I'd do differently

The batched design was not obviously wrong when it was written. One call is cheaper and simpler than five. The problem only became visible when a real execution happened to pull a skewed result pool.

I'd add a result-distribution check to the gate output earlier. If eight of ten results come from one domain, that is a signal the search was not representative, and it should surface before the gate makes a verdict. A simple domain-count log on the search response would have flagged execution 17546 immediately.

I'd also have caught the missing `TODAY` in review. A fact-check request with no date anchor is incomplete by definition, and that is the kind of thing a checklist catches before it reaches production. The Lab Overflow fix on 23 October should have triggered an immediate audit of every node ported from that engine. It did not, and CargoMode-web ran the same bug for another day.

## Quick answers

**Why does batching claims into one search call cause problems?**

The search model returns one pool of results shared by every claim in the batch. If those results happen to cover only one vendor, every claim is judged against that vendor's pages, and claims about anything else come back unverifiable even when the underlying facts are solid.

**What does fail-closed merging mean in a fact-check pipeline?**

If the verdict response for a claim cannot be parsed, the system treats that claim as unverifiable rather than verified. It is the conservative default: a broken response never silently passes a claim through the gate.

**Why send per-claim requests one at a time instead of in parallel?**

Parallel calls to the search model hit 429 rate-limit errors. Spacing requests 1.5 seconds apart keeps all five calls clean without meaningfully slowing the pipeline.

**Why is today's date important in a fact-check request?**

Without a date, the search model has no anchor for recency. A claim about a current product version or a recent event can be matched against outdated pages and still pass, or fail against pages that have since been updated.

**Why are GitHub commit PUTs never retried on failure?**

A PUT that succeeds on the first attempt but whose response is lost will appear to have failed. Retrying it creates a second commit with the same content, or races the first into a 409 conflict. The safer rule is to treat a failed PUT as a signal to investigate, not to retry automatically.

**What changed in the Extract Claims node alongside the per-claim requests?**

The node now returns a cited_url with each claim, and its maxTokens was raised from 4000 to 6000. The cited URL travels with the claim into the search request, giving the model a concrete source to check against rather than an open-ended web search.
