Encyclopedia of contested claims
HomeHow it worksMethodology
Methodology

The full standard, written down.

How claims are phrased, how models are selected, how evidence is curated, and how panel divergence is reported. Versioned and public - like the conclusions it produces.

1. Standard of review

Adjudged is not a fact-checker in the traditional sense. We do not certify true / false. We surface what nine independent reasoning systems conclude when given the same evidence corpus, and we report the shape of their agreement and disagreement.

A claim with 9-of-9 panel agreement at high confidence is treated as well-supported by reasoning under the available evidence. A claim with 5-of-9 agreement at modest confidence is treated as genuinely contested. The distinction is reported in every article header.

2. Claim phrasing

Editors rewrite each submission to meet three tests:

  • Neutral framing: the question does not assume the answer in its construction
  • Falsifiability: there exists evidence that, if found, would change the verdict
  • Scope discipline: the claim describes one testable thing, not a bundle
Example

The submission "chemtrails are real" is rewritten as "Are persistent aircraft contrails evidence of a coordinated covert spraying program?" - same topic, neutral construction, testable.

3. Panel selection

The current standing panel includes nine model families chosen for geographic, institutional, and training-lineage diversity. Selection criteria include:

  • The model must be available via a unified inference layer (currently OpenRouter)
  • The model must come from a different institution and training lineage than the others
  • At least one model must originate outside the global north
  • At least one model with an explicitly non-default political training posture (currently xAI Grok)

The standing roster, version history, and rationale for each member is documented on the nine models page.

4. Evidence corpus

For each claim, editors compile a curated source list before the panel runs. Sources are ranked by primacy:

  • Primary: government data, peer-reviewed studies, declassified records, court filings
  • Secondary: academic surveys, professional society statements, official agency explainers
  • Tertiary: investigative journalism with named sources, credible reference works

Editorial sourcing notes are published with each article. We do not hide which sources we chose, and we accept that another editor could have made different choices.

5. Independent reading

Each model receives the same prompt template and corpus. Models do not see one another's outputs. We do not use a "judge" model to score the others - aggregation is deterministic, based on the structured fields each model returns.

Required fields per model: verdict, confidence, reasoning, cited_evidence, what_would_change_mind. Free-text rationales are preserved in full alongside the structured data.

6. Aggregation rules

Panel score = arithmetic mean of model confidence scores, weighted equally. We do not weight models by reputation, accuracy on prior claims, or any other signal - equal weighting is the only defensible position absent a ground truth.

Refusals and incomplete responses are reported but not averaged in. A 9-model panel with two refusals is reported as 7 of 9 valid · 2 incomplete, not silently averaged across seven.

7. Reporting divergence

Every article includes a "Where the panel diverged" section, even when divergence is small. Models that gave the lowest or highest confidence, refused, or used materially different reasoning are named explicitly. We do not collapse outliers into the headline number.

8. Revisions and corrections

Articles are versioned. Each revision shows what changed and why. Corrections - for errors of fact, source attribution, or aggregation - are dated and preserved in the visible history of the page. Substantive verdict changes require a re-run of the panel.

9. Known limitations

This method has failure modes we accept and try to mitigate, not deny:

  • Models trained on overlapping data can converge to the same wrong answer
  • Editorial source selection introduces upstream bias we try to make visible
  • State-aligned models systematically refuse or aligned-frame certain topics - captured as data, not noise
  • Panel agreement is not truth - it is convergent reasoning under a chosen corpus

Version 1.3 · Last updated 21 May 2026 · Full revision history