Two-thirds of the load-bearing evidence names no lineage, which is why this graph cannot audit its own commonest failure
also called system of record · species lineage audit · generalisation failure
The mistake this project makes most often is treating a result from a mouse, or from a different leukaemia, as though it were about the infant disease this graph is about. Two-thirds of evidence items name no lineage and nearly two-fifths name no experimental system, so the graph cannot check itself for that mistake.
The same thing, for a scientist
The spine beat could not audit the causal chain for species, lineage or model because **no field records them**, and proposed adding one. Before building that, this measures whether the information exists in the prose. Across **5,300 load-bearing evidence items**: **66% carry no lineage marker and 39% carry no system marker at all.** Where markers do appear, patients 29%, mouse 21%, human cell line 13%, in-vitro biochemistry 10%, and **myeloid 21% against B-lineage 18%** - in a graph whose index disease is infant B-lineage KMT2A::AFF1. **The per-item classification is NOT reliable and is not offered as one.** Three successive versions over-flagged; the distribution is the result.
What is measurable, and what is not
| marker, load-bearing items (n=5,300) | share |
|---|---|
| patients / cohort / trial | 29% |
| mouse | 21% |
| human cell line | 13% |
| in-vitro biochemistry | 10% |
| human primary cells | 2% |
| Drosophila / yeast / zebrafish | 1% |
| no system marker at all | 39% |
| lineage marker | share |
|---|---|
| myeloid | 21% |
| B-lineage | 18% |
| T-lineage | 1% |
| no lineage marker at all | 66% |
The two blanks are the finding. This graph's index disease is infant B-lineage KMT2A::AFF1, and more myeloid-marked load-bearing evidence than B-lineage-marked sits underneath it - which is expected, because the murine MLL-AF9 literature is far larger, and is exactly why the project keeps writing 'not the index fusion' into caveats. But two-thirds of items name no lineage, so the comparison rests on the third that does.
Why I am not reporting a defect count, having built the instrument to produce one
The intended output was a list: load-bearing items resting on myeloid or murine systems whose caveats fail to say so. Three versions produced numbers I could not stand behind.
v1 reported a mouse marker in 55% of load-bearing items. The pattern listed Mll-Af9 and Kmt2a as mouse markers - those are fusion names, used in human and murine systems alike, and they appear throughout this graph. It was measuring the fusion's name and calling it a species. Corrected to 21%.
v3 still flags an art-restoration analogy entry as myeloid. The test asks whether an item mentions AML without mentioning the index fusion - and an analogy entry that reaches for AML as a comparison is doing something legitimate. A corpus that discusses systems comparatively cannot be classified by whether a system is mentioned.
So the distribution stands and the per-item verdict does not. That is the same conclusion the cross-entry contradiction screen reached, for the same reason, and it is the seventh instrument of mine this month to over-flag before it measured anything. The working rule has earned its place: an instrument built quickly against a heterogeneous corpus finds its own construction before it finds a defect.
What this says about the proposed field
It strengthens the case and changes the scope. If the information were reliably in the prose, a system field would be redundant - a parser could extract it. It is not there in two-thirds of items, so the field would be recording something the graph genuinely does not hold.
But it cannot be backfilled by inference, which is what this analysis set out to test and answered in the negative. A system object on evidence items has to be written by whoever reads the paper, at the time they read it, and the honest scope is new evidence from here on plus deliberate retrofitting of whichever beats a future cycle decides matter - not an automated pass over 7,060 items.
One cheap thing would help immediately. The brief could require that any claim whose system is not the index disease name that system in the claim, not only in the caveat. This graph's caveats already do it well; the claims are where the information goes missing, and a claim is what gets quoted.
- Do not read '66% carry no lineage marker' as 66% of the graph being unsound. It says the graph cannot mechanically check its own commonest failure mode, not that the failure has occurred. The cases where it HAS occurred were found by reading, every time.
- And do not read the myeloid-over-B-lineage skew as a bias introduced here. The murine MLL-AF9 literature is genuinely larger than the infant B-lineage one, which is a fact about the field this graph has to work within rather than a choice it made.
- Should the brief require a claim whose system is not the index disease to name that system IN THE CLAIM rather than only in the caveat? The caveats do this well; the claims do not, and a claim is what gets quoted onward.
- Which beats deserve a retrofitted `system` field? Backfilling by inference is now shown not to work, so the choice has to be made deliberately and paid for by reading.
- Is the myeloid-over-B-lineage skew a real property of the available literature, or of which beats this project has run?
- E1moderate confidence 3 of 5: One strong study, or several smaller consistent ones.analysis
Across 5,300 load-bearing evidence items in this graph, 66% carry no lineage marker and 39% carry no experimental-system marker in their claim or caveat text; among those that do, 29% reference patients or trials, 21% mouse systems, 13% human cell lines and 10% in-vitro biochemistry, with myeloid markers at 21% against B-lineage at 18%.
system-of-record: species, lineage and model markers inferred from claim and caveat text across all evidence items · Oncofusion knowledge graph (internal analysis) 2026What this evidence cannot show
Run 2026-09-20. **Markers are inferred from prose, not read from a field**, so every figure is a floor: an item whose system is obvious to a human reader but unnamed in the text counts as unmarked. **Markers are deliberately non-exclusive** - an item comparing myeloid and B-lineage carries both, correctly. **Moderate, and the per-item classification is explicitly not offered**: three versions of the instrument over-flagged, the first by treating fusion names as species markers, and the distribution is the only output that survived.
Evidence that argues against this (1)
- A missing lineage marker is not evidence of a missing lineage. Much of this graph's evidence concerns biochemistry, structure or database records where lineage is not a meaningful property, and counting those as 'unmarked' inflates the 66% substantially. The figure measures what the text says, and a large fraction of the corpus has nothing to say on the point. Shi J et al. 2013
RULE 73 AT WRITING TIME: the system - adult KMT2A::AFDN myeloid OCI-AML2 - is in the first clause of both claims.
THE SAME FAILURE IN A DIFFERENT LAYER: interaction rows carry no indication of whose experiment or which cells, and rule 73 applies to them too.
THE SYSTEM PROBLEM IN A NEW PLACE: partner rows carrying no indication that the cells were embryonic kidney.
SAME SHAPE, DIFFERENT FIELD: a dimension the brief asks for, absent in aggregate, invisible entry by entry, and unenforced by any checker.
MEASURES WHAT THAT ENTRY REPORTED: the spine beat found no field records species or lineage; this finds the prose does not carry it either in two-thirds of load-bearing items, so the proposed field cannot be backfilled by inference.
SAME CONCLUSION, SAME REASON: a corpus that discusses systems comparatively cannot be classified by whether a system is mentioned.
SEVENTH INSTRUMENT THIS MONTH TO OVER-FLAG. Here the first version treated fusion NAMES as species markers and reported a mouse marker in 55% of items; the real figure is 21%.
RULE 73 APPLIED AT WRITING TIME: the system - adult KMT2A::AFDN myeloid OCI-AML2, verified as CVCL_1619 - is named in the first clause of the claim, not only in the caveat.