Oncofusion

start / graph / ana-the-corpus-states-624-falsifiers-and-records-the-outcome-of-eight

Analysismoderate confidence 3 of 5: One strong study, or several smaller consistent ones.

This graph states 624 falsifiers and records the outcome of 17 - a figure this entry first published as 8, because the detector did not know the corpus's own ALL-CAPS convention for recording one

also called unrun falsifier queue · falsifier recording rate · the graph does not cash its own falsifiers · 624 falsifiers 8 outcomes

This project is good at writing down how each of its claims could be proved wrong, and almost never comes back to record what happened when anyone checked. Of 624 such stated tests, 17 have a recorded outcome - a count this analysis first got wrong, and undercounted by half.

The same thing, for a scientist

Cycle 60's headline cost one beat. The entry `mechanism-kdm3b-depletion-lowers-ctcf-only-at-the-kmt2a-anchor` carried a field naming the anchor-matched comparison that would refute it, stating the refutation condition numerically, predicting from the paper's own 17,077 losing peaks that refutation was the **likely** outcome, and ending *"NOT RUN by this beat"*. It sat two cycles while the claim stayed published. Run, it killed the claim and confirmed the rival. Nothing in this project reads those fields, so this sweep was named in the cycle-60 log as the highest expected value per unit cost here. **THE FIELD THAT PROMPTED IT IS A ONE-OFF.** `the_check_that_would_refute_it` appears in **one** entry. The corpus convention is a `falsifiers` list, on **161** entries, plus about two dozen assorted `kill_criterion` / `decisive_*` / `falsifiable_*` fields. A one-off field name is exactly what no sweep catches, which is why that finding sat unbought. **MEASURED over all 2,125 entries and 10,771 edges from 280 edge files: 161 entries state a falsifier and 1,964 state none. Of 624 stated falsifier items, 610 record no outcome, 8 record one, and 6 say explicitly that the test was not run.** The queue of unrun-or-unrecorded falsifiers on entries not already hedged to weak or speculative is **118**, and there are **21 entries where every stated falsifier is unrecorded** and at least two exist. **THE CLAIM THIS LICENSES IS NARROWER THAN 8-OF-624, AND I PROVED IT BY HAND.** A keyword sweep sees whether a *field records* an outcome, not whether a *test happened*. Ranked by edge exposure the top of the queue is `mechanism-h3k9me1-2-ctcf-balance-gates-kmt2a-rearrangement` - exposure **38**, confidence moderate, status verified, **six** falsifiers, none recording an outcome, and the entry carrying this graph's prophylactic-intervention proposal. I read all six. **One had already been run, in cycle 59, and the entry never said so.** So the real run rate is higher than 8/624 and unknown. What is measured is the **recording** rate, and that is the defect: this corpus states how it could be wrong and does not come back to say what happened. **AND THE ONE THAT WAS RUN SPLITS BY SCALE, WHICH IS A DEFECT IN THE FALSIFIER.** It asked whether H3K9me3 moves with me1/me2 at the BCR in GSE210480. Re-read from `analyses/q108-gse210480-h3k9me3-kmt2a-bcr/results.json` rather than from entry prose: at the 1,048 bp CTCF element H3K9me3 is **1.014** while H3K9me1 is **3.738**, and at exon 11 and the summit 1.042 against 3.852 - so it **fails to fire** there. Over the 7,466 bp BCR core me3 rises **1.344** against me1's 2.675, ranking **3,877 of 18,082** chr11 bins against me1's **72** - so it **partly fires** there. The falsifier names no interval, so it can be met or not met depending on a scale it never specified. **The mark that tracks me1 least is me2, which is in that entry's own title.** Over the BCR core H3K9me2's log2 is **-0.0326**, rank **10,276 of 18,082** - the middle of chromosome 11 - and at the element a sibling entry records me2 **falling to 0.44x** while me1 rises 3.9x. The entry is named for an "H3K9me1/2 balance"; the measurement is me1 up, me2 down, me3 slightly up. **No falsifier in that entry tests its own titular pairing.** Score on that entry: **1 of 6 run**, two of the remaining five being bench experiments this project could specify today - and one of those is already written up as `exp-h3k9me3-and-ctcf-at-the-kmt2a-anchor-in-a-cd34-progenitor`. Its confidence was **not** lowered, because nothing in the adjudication refutes the mechanism. What was removed is its implicit claim to having been tested. **CORRECTED IN CYCLE 62, AND THE CORRECTION IS THE MOST INSTRUCTIVE PART.** This entry first reported **8** recorded outcomes. Hand-reading a second queue entry - `hyp-obligate-partner-interface-generalises`, 21 edges, eight falsifiers - showed two of them begin ***"ALREADY OBSERVED"***, and `hyp-nup98-partner-obligate-interface` carries ***"ALREADY PARTLY RESOLVED IN THE WRONG DIRECTION"*** and *"is a category error as written"*. **This corpus does have a convention for recording a cashed falsifier — an ALL-CAPS status phrase leading the text — and the detector did not know it.** Corrected: **17 RUN, 601 UNMARKED, 6 NOT_RUN**, and the queue falls from 118 to **113**. **The direction survives and the number did not.** 17 of 624 is 2.7%, so *almost never* still holds — but the published figure was wrong by more than a factor of two and it was in this entry's title, in a commit title and in the ledger. The id still reads `...-outcome-of-eight` and is deliberately not renamed, because ids carry edges. **And the instrument is now wrong three times out of three on every case anyone checked.** Three entries have been hand-read across cycles 61 and 62 and every one recorded outcomes scored as unrecorded. That is the real result: the UNMARKED class means *no outcome detected*, not *no outcome recorded*, and the 601 is an upper bound whose true value nobody knows. What is solid is the shape — a corpus that states falsifiers far more readily than it returns to them — and the 1,977 entries that state none at all, which no detector error can move.

Facts and figures
scriptanalyses/the-unrun-falsifier-queue/run.py
resultsanalyses/the-unrun-falsifier-queue/results.json
statusrun
run date2026-09-22
why this ranthe cycle-60 king log named this sweep the highest expected value per unit cost in the project, and a coordinator job rather than a beat, after one such field was cashed and killed a published claim
queue size excluding weak and speculative entries113
entries where every stated falsifier is unrecorded21
the prompting field is a one off`the_check_that_would_refute_it` appears in ONE entry; the corpus convention is a `falsifiers` list on 161 entries plus about two dozen assorted kill_criterion / decisive_* / falsifiable_* fields
absence classmeasured-not-analysed
scope
entries_stating_a_falsifi…161.0entries_stating_a_falsifier: 161edge_files280.0edge_files: 280falsifier_items_classified624.0falsifier_items_classified: 624entries_stating_NONE1,977entries_stating_NONE: 1977entries_walked2,138entries_walked: 2138edges_loaded_for_exposure10,771edges_loaded_for_exposure: 10771
scope — the numbers behind the chart
{
  "entries_walked": 2138,
  "entries_stating_a_falsifier": 161,
  "entries_stating_NONE": 1977,
  "falsifier_items_classified": 624,
  "edges_loaded_for_exposure": 10771,
  "edge_files": 280
}
classification
{
  "records_an_outcome": 17,
  "no_outcome_DETECTED": 601,
  "says_explicitly_not_run": 6,
  "CORRECTED_FROM": "8 / 610 / 6 as first published. The detector missed the corpus's ALL-CAPS status convention - ALREADY OBSERVED, ALREADY PARTLY RESOLVED, WITHDRAWN."
}
top of queue by edge exposure
[
  "mechanism-h3k9me1-2-ctcf-balance-gates-kmt2a-rearrangement (38, 6 falsifiers, moderate, verified)",
  "hyp-obligate-partner-interface-generalises (21, 8 falsifiers, moderate)",
  "hyp-paralogue-buffering-hides-dependencies (18, 4, moderate)",
  "strategy-open-ground-uncontested-questions (16, 4, moderate)",
  "hyp-nup98-partner-obligate-interface (13, 7, STRONG)"
]
the hand adjudication
{
  "entry": "mechanism-h3k9me1-2-ctcf-balance-gates-kmt2a-rearrangement",
  "score": "1 of 6 run",
  "the_one_that_was_run": "falsifier [1], H3K9me3 moving with me1/me2 at the BCR in GSE210480 - run by Q108 in cycle 59, outcome never recorded in this entry",
  "its_answer_is_scale_dependent": {
    "element_1048bp_me3_over_control": 1.014,
    "element_1048bp_me1_over_control": 3.738,
    "exon11_and_summit_me3": 1.042,
    "exon11_and_summit_me1": 3.852,
    "bcr_core_7466bp_me3": 1.344,
    "bcr_core_7466bp_me1": 2.675,
    "verdict": "does not fire at the element, partly fires over the BCR core; the falsifier names no interval"
  },
  "chr11_null_18082_bins_log2_siKDM3B_over_control": {
    "H3K9me1": {
      "log2": 1.1597,
      "rank": 72,
      "chr11_median": 0.0044
    },
    "H3K9me3": {
      "log2": 0.3969,
      "rank": 3877,
      "chr11_median": 0.1138
    },
    "H3K9me2": {
      "log2": -0.0326,
      "rank": 10276,
      "chr11_median": 0.0212
    }
  },
  "the_titular_pairing_is_the_untested_part": "the entry is named for an H3K9me1/2 balance; me2 does not rise at the BCR (rank 10,276 of 18,082) and falls to 0.44x at the element while me1 rises 3.9x. No falsifier tests that pairing.",
  "mark_identity_controlled": "ZNF274 body me3 3.76 against GAPDH promoter 0.309, ratio 12.168 - the me3 tracks really are H3K9me3",
  "confidence_was_not_lowered": "nothing in the adjudication refutes the mechanism; what was removed is the entry's implicit claim to having been tested"
}
what this instrument cannot see
[
  "an entry that states NO falsifier is invisible here, and that is 1,964 of 2,125 - the commonest form of the problem by a factor of twelve",
  "whether a falsifier is any GOOD. A vacuous falsifier that cannot fire counts as a falsifier, which flatters the corpus",
  "a falsifier that was run and recorded in a DIFFERENT field, which reads as unrecorded - demonstrated: that is exactly what the hand-adjudicated case turned out to be"
]
THE DETECTOR WAS WRONG AND HERE IS HOW OFTEN
{
  "published": 8,
  "corrected": 17,
  "undercount_factor": "more than 2x",
  "cause": "this corpus records a cashed falsifier by prefixing the text with an ALL-CAPS status phrase - ALREADY OBSERVED, ALREADY PARTLY RESOLVED, WITHDRAWN - and the detector's RUN pattern did not include any of them",
  "how_it_was_found": "by hand-reading a second queue entry, not by any check. The first hand-read (cycle 61) had already shown one falsifier run and unrecorded; the second showed two recorded and undetected. Three entries read, three times the classifier was wrong about them.",
  "what_this_does_to_the_claim": "UNMARKED means NO OUTCOME DETECTED, not no outcome recorded. 601 is an upper bound. The shape survives - 17 of 624 is 2.7% - and so does the untouched finding that 1,977 of 2,138 entries state no falsifier at all."
}
How this could bite us back
  • This is a quality audit and nominates nothing clinically. Its only bearing on an intervention is negative and worth stating: the top-exposure entry in the queue is the one carrying this graph's prophylactic G9a/KDM3B proposal, and five of its six stated falsifiers have never been attempted - including a catalytically dead KDM3B rescue that would separate a causal mark from a bystander, and a powered mouse study with a leukaemia-incidence endpoint that the original authors themselves half-expect would fire. No prophylactic intervention in children should be advanced on an entry in that state, and this audit is not a reason to think the entry is wrong - only a reason to stop reading six confident falsifiers under a 'verified' status as evidence that it has been tested.
Still open
  1. Why do 1,964 of 2,125 entries state no falsifier? That is twelve times the size of the problem this sweep measured, and it is not obvious that every entry should have one - a gene record whose content is a UniProt accession has nothing to falsify. Establishing which entry TYPES ought to carry a falsifier would turn this from a count into a standard.
  2. How many of the 610 unrecorded falsifiers have in fact been answered by a later beat, as the hand-adjudicated one was? That is the number that matters and the only way to get it is to read them. At roughly the rate of the one case done here it would be a large number, and each one found is free - the work is already done and merely unrecorded.
  3. Should a falsifier be required to name its interval, cell type and effect size? The one falsifier cashed here could be met or not met depending on a genomic scale it never specified, which made a clean answer impossible to state. That is a defect in the falsifier, not in the measurement, and it is probably widespread.
  4. Does hyp-nup98-partner-obligate-interface deserve its confidence of STRONG with seven unrecorded falsifiers? It is the only entry in the top of the queue graded strong, and it was not hand-read here.
Every source behind this entry (1)
  1. E1moderate confidence 3 of 5: One strong study, or several smaller consistent ones.database

    A sweep of all 2,125 node files found 161 entries stating at least one falsifier and 1,964 stating none. Across those 161 entries there are 624 stated falsifier items, of which 610 record no outcome, 8 record one, and 6 state explicitly that the test was not run. Excluding entries already graded weak or speculative leaves a queue of 118 unrecorded falsifiers, and 21 entries have at least two falsifiers of which none records an outcome. Ranked by edge exposure over 10,771 edges from 280 edge files, the top of the queue is mechanism-h3k9me1-2-ctcf-balance-gates-kmt2a-rearrangement with 38 edges, six falsifiers and confidence moderate at status verified. The field name that prompted the sweep, the_check_that_would_refute_it, appears in exactly one entry.

    What this evidence cannot show

    THIS MEASURES WHETHER A FIELD RECORDS AN OUTCOME, NOT WHETHER A TEST HAPPENED, and the difference is not academic - the one entry I hand-adjudicated turned out to have had a falsifier run in cycle 59 without recording it. So 8-of-624 is the RECORDING rate; the run rate is higher and unknown. Classification is keyword-based over prose, so a falsifier whose text merely contains the word 'measured' reads as recorded, and an explicit run marker deliberately overrides a not-run marker in the same text because corrected entries keep both. Exposure is raw edge degree, which counts an entry's outgoing context links as well as inbound dependence. The citation is this repository's own script and output, because the claim is about this repository.

    Evidence that argues against this (2)
    • A ratio of 8 recorded outcomes to 624 stated falsifiers is so extreme that the instrument is the first thing that should be doubted, not the corpus - and this project has a specific record of exactly that: an instrument once flagged 1,717 titles as truncated and was wrong about essentially all of them, and the cycle-60 audit was wrong about nearly everything on its first two runs. The hand-read of six items is the only validation here, and one of the six contradicted the classification, which is a 17% error rate on a sample of six.
    • The framing may be unfair to the corpus. A falsifier is a statement about what WOULD refute a claim, and there is no convention in this project - or in the literature - that an entry must be revisited whenever an adjacent measurement bears on it. Judged as documentation of reasoning rather than as a task list, 161 entries stating 624 ways they could be wrong is unusually good practice, and counting unrecorded outcomes as a deficit assumes a workflow that was never adopted.
Connected to