The codebase behind OpenSubstance is private. The evidence layer — where every number comes from, how confident we are in it, and how it is checked — is meant to be completely transparent. This page describes that process end to end: how facts are gathered, how they are rated, and how their citations are audited by several independent AI models before anything reaches you.
Every quantitative claim on this site — harm scores, addiction rates, safety margins, purity percentages, detection windows — is traceable to a specific source. Wherever possible that source is primary: a peer-reviewed study, a government dataset, or a forensic lab report, not a secondhand summary.
When a new source is cited, it is recorded in three places at once: the citation table that powers the inline references, the public Sources page, and the project README. A citation that isn't traceable in all three doesn't ship. When sources disagree, we use the more conservative estimate and say so.
Not every number carries the same weight, and pretending otherwise would be dishonest. Each quantitative field and each brain-mechanism explainer is tagged with a confidence tier and a plain-language note on how the value was obtained. The four tiers:
Tiers are visible in the product, not buried in code: the ? buttons in the Rankings charts reveal the tier and source for individual numbers, and each substance's brain section carries its source badge.
A fact being tagged with a citation is not the same as that citation actually supporting the fact. To catch mismatches, an automated audit re-reads every sourced claim and checks it against the source it points to. The audit is read-only: it can flag a claim for human review, but it never edits the underlying data. Editorial ground truth is never overwritten by a machine.
The audit runs in two modes. In search mode, each model is given a live web-search tool and asked to look the source up. In citation-only mode, it evaluates the claim against the reference from its own training knowledge — cheaper and reproducible, but weaker evidence. We are candid about the ceiling here: most cited journals are paywalled, so even search mode usually surfaces abstracts and secondary discussion rather than full paper text. A "supported" verdict means a model found public evidence consistent with the claim — not that the paywalled paper was opened and confirmed.
No single AI model gets to be the arbiter. Each claim is checked independently by several models from different labs and lineages, so that one vendor's blind spot or bias doesn't quietly become ours:
Including an open-weight model (Qwen), run locally rather than through a vendor's API, is deliberate: it keeps at least one voice in the panel that isn't a large US frontier lab.
Each model returns one of five verdicts, from most to least supportive of the stored claim:
The individual verdicts are combined into a single consensus: the plurality verdict wins, and ties break toward the more conservative result. A claim only two models call "supported" while a third calls "contradicted" is not quietly rounded up to "supported" — the disagreement is preserved and the claim is flagged. When models diverge, that divergence is itself the signal: it usually means the claim sits at the edge of the literature and deserves a human read.
This system is a safety net, not an oracle. It is worth being precise about what it can and can't tell you.
Several AI models agreeing that a claim is supported is weak positive evidence on its own — models can share the same misconception, or all latch onto the same wrong paper. Several models flagging a claim is a much stronger signal, because it takes only one to raise a hand. So the audit earns its keep mostly by surfacing problems, not by blessing claims.
The audit does not replace editorial review, and it does not replace reading the actual paper when a source is first cited. Every flag — a contradiction, a claim that isn't in its source, a split decision — is reviewed by a person, who decides what to change. The machines narrow down what to look at. Humans make the call.
Found something that looks wrong? That's exactly the kind of report this whole process is built to act on. Email corrections@opensubstance.org.