Absence claims: 93% are unbounded in prose, but the evidence claims are 71% bounded - and the query that tested them was itself broken
also called bounded absence · counted absence audit · EuropePMC field query failure
This graph says 'nobody has tested this' about 1,878 times, and almost none of those sentences in ordinary prose come with the date, search and number that would let anyone check them - though the ones carrying a formal confidence rating mostly do, with the bound written in the caveat beside the claim rather than inside it. Worse, the literature search I wrote to test the nine riskiest claims was itself broken and returned 'zero papers' for every one, which looked like confirmation until the same database returned 535 papers for one half of that search and 7,161 for the other half.
The same thing, for a scientist
1,878 absence assertions across 1,462 entries. 6.8%% are bounded by a date, a named source and a count. Restricted to evidence[].claim, where a confidence tier is attached, 12/17 (70.6%%) are bounded, and the bound lives in the sibling caveats field, not the claim sentence - the v1 scorer read only the claim and reported 76.5%% unbounded, which was an artefact of scope. Testing the nine severe cases exposed a query-construction failure in EuropePMC: `(TITLE:"X" OR ABSTRACT:"X") AND (TITLE:"KMT2A" OR ...)` returns hitCount 0 while its own operands return 535 and 7,161 and the free-text `NFKB2 AND MLL` returns 109-184. Field-scoped AND-of-parenthesised-OR does not compose against this API and fails silently toward zero. Re-tested with validated free-text queries and title/abstract token matching, none of the five falsifiable claims is refuted; the four NF-kB ones are exemplary, recording the querytranslation and the pathway-level false-negative risk explicitly.
What was measured
analyses/absence-claims-bounded-or-not/run.py. Every string field of every node is split into sentences; sentences asserting that the literature contains nothing are matched, then scored bounded if a date, or a search term together with a count, appears in the sentence, its field, or - for evidence items - the sibling caveats and citation.
| population | n | bounded |
|---|---|---|
| all absence assertions | 1,878 | 6.8% |
in evidence[].claim | 17 | 70.6% |
| unbounded at strong/established | 3 | - |
The two rows are the finding. House style states open questions freely in prose - 'nobody has tested that, and it is a concrete, cheap experiment' - and that is an invitation, not an assertion of fact. Where the graph attaches a confidence tier, it mostly does the work. The three survivors are not literature claims at all: two are structural-coverage statements about PDB (gene-med26, gene-kmt2a) and one is a meta-claim counting sentences in this graph.
The instrument was broken, and it broke toward false confirmation
This is the part worth carrying forward. To test the five falsifiable claims - 'X has never been named in the title or abstract of a KMT2A or MLL paper' - I queried EuropePMC as
(TITLE:"NFKB2" OR ABSTRACT:"NFKB2") AND (TITLE:"KMT2A" OR ABSTRACT:"MLL" OR ...)
and got hitCount 0 for four of five. That reads as clean confirmation of four strong-confidence claims. It is an artefact:
| query | hits |
|---|---|
TITLE:"NFKB2" OR ABSTRACT:"NFKB2" alone | 535 |
| the KMT2A clause alone | 7,161 |
| the two joined with AND | 0 |
free-text NFKB2 AND MLL | 109 |
free-text NFKB2 AND (MLL OR KMT2A) | 184 |
Two large sets cannot have an empty intersection reported while a free-text AND of the same terms returns 184. Field-scoped AND of parenthesised multi-field OR groups does not compose against this API. The same construction returned 7 for NF-kB AND MLL, which is absurd on its face and is the tell I should have caught first.
The failure direction is what makes it dangerous. It does not error; it returns zero, and zero is exactly the answer a counted-absence check is hoping for. Four of this project's previously recorded false absences were produced by queries narrower than the sentence they licensed. This is the same disease with a new vector: not a narrow query, a non-composing one. Any zero from this construction anywhere in the graph is uninterpretable rather than negative.
Re-tested properly, the claims hold - and four of them are models of the form
Re-run with free-text queries and title/abstract token matching over the returned records:
| entry | free-text hits | records naming BOTH in title/abstract |
|---|---|---|
gene-nfkb2 | 184 | 0 |
gene-chuk | 86 | 0 |
gene-ikbkg | 62 | 0 |
gene-rel | 160 | 0 |
gene-hcfc2 | 29 | 1 |
The free-text hits are full-text co-occurrences, which is why they do not contradict a title/abstract claim. Only the first page of each was examined, so these are lower bounds on refutation and do not re-establish the absences.
The HCFC2 hit is real: PMID 15199122 names HCF-2 in its abstract, as one of two HCF proteins binding a conserved motif in MLL(N). The entry's caveat already says 'the count is one', and its claim is that HCFC2 has never been the subject of a second paper - so the entry is consistent with what I found rather than contradicted by it.
The four NF-kB entries are better than my detector implied. Their caveats record that the querytranslation was read and contained no [UID] artefact, note which aliases were deliberately excluded and why (REL alone matches 'relative'), and one states outright that a zero at the symbol level is not a zero at the pathway level, and in this case demonstrably is not, naming the NIK paper the query cannot reach. That is the standard the rest of the graph's prose should be held to - and it is the reason the headline 6.8% must not be quoted as a quality score.
What I got wrong, twice, in one analysis
Recorded because the pattern keeps recurring and the record is the point.
1. Wrong scope. v1 scored the claim sentence and its own field, so it could not see a bound written in caveats. It reported 76.5% of evidence claims unbounded; the true figure is 29.4%. A checker that reads a narrower field than the practice it audits manufactures defects - the mirror image of the narrow-query problem it was built to find.
2. Unvalidated query. I ran the severe-claim test and got the answer I expected before running a positive control. The control took one minute and overturned the result. The rule this graph already has - never trust an identifier from recall - needs a sibling: never trust a zero from a query you have not shown can return a one.
How far the damage reached: nowhere, this time
A grep for the field-scoped construction across analyses/ and scripts/ returns only the two scripts written for this analysis. Every other literature query in this repository uses PubMed E-utilities with the querytranslation read back, or plain free-text EuropePMC. No counted absence already in the graph was produced by the broken construction, so this is a rule against a trap I walked into and climbed out of, not a cleanup of existing damage.
That is worth stating plainly because the opposite finding was the expectation. The reason it is confined is that the existing practice - read the querytranslation - catches this class automatically: a querytranslation that expands to nothing is visible, whereas a REST hitCount of 0 is not.
- Nothing here touches a patient. The risk is to the graph's credibility: if a reader spot-checks an unbounded 'nobody has tested this' and finds a paper, every other absence claim becomes suspect at once, including the ones that are carefully bounded.
- Do not quote the 6.8% figure as a defect rate. It counts prose that was never intended to be falsifiable, and the population that matters - claims carrying a confidence tier - is 70.6% bounded. Quoting the low number would misrepresent this graph's own practice.
- Any counted absence in this repository that was produced by a field-scoped AND-of-OR EuropePMC query should be treated as unverified until re-run, because that construction returns zero regardless of the literature.
- ANSWERED 2026-09-20: the non-composing field query appears nowhere else in this repository - a grep for 'TITLE:"' across analyses/ and scripts/ returns only this cycle's own two scripts. No previously recorded zero in this graph came from that construction. The rule is preventive, not retrospective.
- What is the correct EuropePMC syntax for a field-scoped conjunction, and does it reproduce the free-text counts when written properly?
- Are the two structural-coverage claims (gene-med26, gene-kmt2a) still true against the current PDB, and when were they last checked?
- E1moderate confidence 3 of 5: One strong study, or several smaller consistent ones.analysis
Across 1,462 entries, 1,878 sentences assert that something has not been done; 6.8% carry a date, a named source or a count, while within evidence[].claim, where a confidence tier is attached, 12 of 17 (70.6%) are bounded, with the bound written in the sibling caveats field rather than the claim sentence.
absence-claims-bounded-or-not/run.py: every absence assertion in the graph, scored for whether a date, a named source or a count bounds it · Oncofusion knowledge graph (internal analysis) 2026What this evidence cannot show
Run 2026-09-20 against the graph at 1,462 entries. **The bounding test is textual, not semantic**: a date near a search term scores as bounded even when the query was narrower than the claim it licensed - which is the exact failure that broke four counted absences here. **So 6.8% and 70.6% are both UPPER bounds on how well bounded this graph is.** The regex cannot see an absence split across two sentences or phrased positively ('the only study is X'), so the denominator is an undercount. Moderate rather than strong because both numbers are instrument outputs whose v1 was wrong by a factor of two and a half.
Evidence that argues against this (1)
- The 6.8% figure invites being read as a quality score and should not be. Stating an open question in a lay summary as 'nobody has tested this' is house style and an invitation to a collaborator, not an unfalsifiable assertion - the graph is not obliged to attach a PubMed query to a sentence that carries no confidence tier. The defensible claim is narrower: where confidence is attached, a bound is usually attached too. Oncofusion knowledge graph (internal analysis) 2026
- E2strong confidence 4 of 5: Replicated in peer-reviewed work by more than one group.analysis
A EuropePMC query of the form (TITLE:"X" OR ABSTRACT:"X") AND (TITLE:"KMT2A" OR ABSTRACT:"MLL" OR ...) returns hitCount 0 for NFKB2, CHUK, IKBKG and c-Rel, while the NFKB2 clause alone returns 535, the KMT2A clause alone returns 7,161, and the free-text query NFKB2 AND (MLL OR KMT2A) returns 184; the construction therefore does not compose and fails silently toward zero.
control.py: positive controls showing the field-scoped AND-of-OR construction returns 0 while its own operands return 535 and 7,161 · Oncofusion knowledge graph (internal analysis) 2026What this evidence cannot show
Run 2026-09-20 against the EuropePMC REST search endpoint. **Strong as a statement that the zeros are uninterpretable**, which is what it claims and what the operand counts establish arithmetically. **It is NOT a characterisation of the API's grammar**: I did not determine whether the fault is the parenthesised multi-field OR, the implicit default field, or an undocumented precedence rule, and I did not consult the API documentation. The practical rule stands independently of the cause.
Evidence that argues against this (1)
- A construction that fails on this endpoint may work on PubMed's E-utilities, whose field syntax differs, so this must not be generalised into 'field-scoped queries are unsafe'. Several entries in this graph record reading the PubMed querytranslation specifically to confirm field expansion, which is the correct defence and is unaffected by this finding. Oncofusion knowledge graph (internal analysis) 2026
- E3established confidence 5 of 5: Multiple independent lines of evidence; treated as settled in the field.paper
PMID 15199122 names HCF-2 in its abstract, identifying host cell factor 1 and the related HCF-2 as members of the MLL complex that interact with a conserved binding motif in the MLL N-terminal subunit.
Leukemia proto-oncoprotein MLL forms a SET1-like histone methyltransferase complex with menin to regulate Hox gene expression · Yokoyama A, Wang Z, ..., Cleary ML · Molecular and Cellular Biology 2004What this evidence cannot show
Abstract fetched and read back on 2026-09-20; full text not read. Established as what the abstract states. **This does not refute `gene-hcfc2`**, whose claim is that HCFC2 has never been the subject of a SECOND paper and whose caveat already records a count of one - the record found is that one. Carried here because an automated absence checker would report it as a contradiction, and the distinction between 'named in' and 'the subject of' is exactly what a checker cannot make.
Evidence that argues against this (1)
- Being named once in a 2004 complex-composition abstract is thin ground for any claim about HCFC2 in this disease in either direction; the honest reading is that HCFC2 is unstudied here, not that it is absent from the literature. Oncofusion knowledge graph (internal analysis) 2026
TESTED, NOT REFUTED: this entry's counted absence was re-run 2026-09-20 with a validated free-text query; 184 full-text co-occurrences, none naming NFKB2 in title or abstract with KMT2A. The claim survives, and its caveat's querytranslation practice is why.
TESTED, NOT REFUTED: 86 free-text co-occurrences, none naming CHUK in title or abstract alongside KMT2A, on the first page examined 2026-09-20.
TESTED, NOT REFUTED: 62 free-text co-occurrences, none in title or abstract with KMT2A (2026-09-20).
TESTED, NOT REFUTED: 160 free-text co-occurrences, none naming c-Rel in title or abstract with KMT2A (2026-09-20).
THE ONE HIT IS THE ONE THE ENTRY ALREADY COUNTS: PMID 15199122 names HCF-2 in its abstract. The entry claims HCFC2 has never been the subject of a SECOND paper and its caveat records a count of one, so the record found is that one.
UNBOUNDED AT ESTABLISHED: one of three absence claims carrying a confidence tier with no date, source or count. It is a structural-coverage claim about PDB rather than a literature claim, so it needs a different check than a search.
- related to →KMT2A / MLL1 — the gene that breaks
UNBOUNDED AT ESTABLISHED: the claim that the residues where the gene breaks have never been resolved is a PDB-coverage statement with no recorded date. Cheap to bound, and load-bearing for the breakpoint beat.
THE FORM THIS ANALYSIS ARGUES FOR, stated in that entry's own prose: 'a counted negative with a date rather than an unbounded nobody has looked'.
SAME SHAPE, MEASURED TWICE: that ledger counted 41 unbounded absence sentences in one beat; this analysis generalises the count to 1,878 across the graph and grades them by whether a confidence tier is attached.
COMPANION INSTRUMENT, SAME FAILURE MODE: both are graph-wide detectors whose first version measured a narrower field than the practice being audited, and both had to be corrected before their numbers meant anything.
THE PAYOFF OF BOUNDING: these four were re-checkable precisely because each recorded a date, the exact query and the tool.
A BOUNDED ABSENCE CAN STILL BE WRONG: this one was bounded, corrected once, and still false.
RULE 67 APPLIED IN ANGER: the counted absence here was recorded only after positive controls (538, 442, 1,255) showed the query returns results.
A BOUNDED ABSENCE WHOSE OWN CAVEAT NAMES HOW IT COULD BE WRONG: a large-scale atlas is invisible to a method-plus-gene query.
A BOUNDED ABSENCE THAT HELD: 5 hits, 0 on topic, controls at 1,275 and 5.
- ← related toThe named-but-unwritten queue is keyed on gene symbols, so CD15 is invisible to it at 23 entries
SAME LESSON, DIFFERENT INSTRUMENT: a detector validated only over the population it can enumerate, and a query whose zero was mistaken for a negative result.
A COST CLAIM NEEDS BOUNDING LIKE ANY OTHER: 'free, no bench work' was asserted three times without anyone opening the deposits.
SAME FAILURE SHAPE: a checker that reads one phrasing of a caveat scores a better-worded caveat as absent, exactly as scoring a claim without its sibling caveats field did.
RULE 67 APPLIED, AND VIOLATED, IN ONE SCRIPT: the first control tested a different query shape from the one run, which is exactly the failure that analysis documented.