Six times the null pool, and the criterion I had been using turns out to be the problem (run)
also called deep-pool variance-matched null · 2,743-gene null
Two earlier versions of the same test disagreed about three genes. Running it again against six times as many random comparison genes settles the disagreement, and shows the disagreement was caused by how we were scoring the test rather than by the genes themselves.
The same thing, for a scientist
Repeats the variance-matched null with a 3,000-gene draw (2,743 usable) instead of 500 (459 usable), answering the open question ana-depmap-null-variance-matched left. The top four are unchanged. MEF2C survives; KAT7 and MEIS1 do not beat their matched null maxima, partly reversing cycle 17. More importantly, the pass/fail criterion both earlier analyses used — 'exceeds the maximum of the null' — is shown to be unstable by construction, because the maximum rises mechanically as the pool grows. Under Holm-corrected empirical p, the surviving set is KMT2A, PSIP1, DOT1L, FLT3, MEF2C and KAT7.
The criterion was the bug, not the genes
Cycle 10 drew 377 random genes and asked whether each candidate exceeded the maximum of the null. Cycle 17 matched the null on variance and asked the same question. They disagreed about MEF2C, KAT7 and MEIS1, and cycle 17's entry concluded the three were 'unresolved' and that a bigger pool might settle them.
A bigger pool does settle them, but not the way that entry expected. The maximum of a null is an order statistic: it rises as the sample grows, with no change in the underlying distribution. Requiring a candidate to beat it therefore makes the test mechanically stricter the more comparison genes you draw. Two analyses using that criterion on pools of different sizes are not answering the same question, so their disagreement was never evidence about MEF2C, KAT7 or MEIS1 at all.
KAT7 makes this concrete. Its matched null now holds 314 genes and its empirical p is 0.00318, which is exactly 1/314: precisely one random gene out of 314 exceeded it. Under the old criterion that single gene flips KAT7 from pass to fail. Under any reasonable statistic it is a strong result 1.
The result2 deeper
2,743 usable random genes, matched to each candidate within 20% of its own SD of gene effect across the same 54 leukemia lines; tolerance widened only where fewer than 25 genes matched. Holm correction over the ten candidates.
| Gene | r | matched null n | empirical p | Holm | verdict |
|---|---|---|---|---|---|
| KMT2A | 0.783 | 128 | <0.008 | <0.001 | survives |
| PSIP1 (LEDGF) | 0.686 | 158 | <0.006 | <0.001 | survives |
| DOT1L | 0.681 | 138 | <0.007 | <0.001 | survives |
| FLT3 | 0.645 | 71 | <0.014 | <0.001 | survives |
| MEF2C | 0.526 | 31 | <0.032 | <0.001 | survives, weakly |
| KAT7 | 0.517 | 314 | 0.0032 | 0.016 | survives |
| MEIS1 | 0.468 | 298 | 0.0168 | 0.067 | does not survive |
| DPY30 | 0.446 | 420 | 0.0286 | 0.086 | does not survive |
| MLLT1 | 0.426 | 258 | 0.0465 | 0.093 | does not survive |
| BCL2 | 0.429 | 186 | 0.0484 | 0.093 | does not survive |
| ACTB (control) | 0.154 | 199 | 0.520 | 1.0 | correctly null |
| GAPDH (control) | 0.150 | 158 | 0.496 | 1.0 | correctly null |
The two housekeeping controls land at p 0.52 and 0.50, almost exactly where a null result should sit. That is the check that the whole procedure is calibrated, and it passes.
MEF2C is the weakest of the survivors and the entry should not hide it
MEF2C's gene effect varies more across lines than almost anything else tested (SD 0.63). Variance matching therefore finds it very few comparators: even from 2,743 genes, only 31 match, and the tolerance had to be widened to 30%. Its empirical p of zero means below 1/31, i.e. below 0.032, not below 0.001.
So MEF2C passes every test put to it and is still the least resolved of the six. Deepening the pool did not help it, because its problem is that it is unusual, not that the pool was small. Nothing short of a different design will fix that.
What changed versus cycle 17
Cycle 17 reported that MEF2C, KAT7 and MEIS1 'now exceed their matched nulls, reversing the earlier deflation'. With a six-times-deeper pool none of the three exceeds its null maximum except MEF2C, so on cycle 17's own criterion that conclusion does not replicate.
On the better criterion, MEF2C and KAT7 survive Holm correction and MEIS1 does not (Holm 0.067 — close, and it would be wrong to call it refuted rather than unsupported). The direction of cycle 17's claim about MEF2C and KAT7 was right; its reasoning and its claim about MEIS1 were not.
What this still does not fix
Lineage restriction inside leukemia remains unaddressed by every null this project has run. All 54 lines are leukemia lines, and genes co-essential within a lineage will correlate for reasons that have nothing to do with a physical complex. Variance matching does not touch this, and neither does pool depth.
The empirical-p floor is set by the matched set, not the pool, so deepening the pool from 459 to 2,743 usable genes moved the floors from 1/25–1/80 to 1/31–1/420. Better, but MEF2C shows the floor can stay high for an unusual gene no matter how much is drawn.
The DepMap release version is still not stated by the Open Targets API, so this is not reproducible to a specific data freeze.
| script | analyses/depmap-null-deep-pool/run.py |
|---|---|
| status | run |
| run date | 2026-09-18 |
| answers open question from | ana-depmap-null-variance-matched |
| method | variance-matched nulls within +/-20% of each candidate's SD of gene effect (widened if fewer than 25 matched) drawn from a 3,000-gene random sample; Holm correction over the ten candidates |
| seed | 20260918 |
| n random requested | 3000 |
| n random usable | 2743 |
| n leukemia lines | 54 |
| methodological finding | 'exceeds the null maximum' is an order statistic that rises with pool size, so it is not comparable across analyses and should not be used as a pass criterion; cycles 10 and 17 both used it |
| resolution limit | empirical-p floor is set by the matched set (31 to 420 genes here), not by the 2,743-gene pool |
| figure | analyses/depmap-null-deep-pool/results.md |
survives holm
[ "KMT2A", "PSIP1", "DOT1L", "FLT3", "MEF2C", "KAT7" ]
does not survive holm
[ "MEIS1", "DPY30", "MLLT1", "BCL2" ]
controls
[
{
"gene": "ACTB",
"r": 0.1539,
"empirical_p": 0.52027
},
{
"gene": "GAPDH",
"r": 0.1498,
"empirical_p": 0.49593
}
]- The temptation is to report that the deeper pool 'confirmed' the co-dependencies. It did for four of them, weakly for MEF2C, newly for KAT7, and it removed MEIS1. Reporting the confirmations without the removal would be the selective reading this analysis exists to prevent.
- KAT7 surviving correction is the kind of result that starts a chemistry programme. It rests on one correlation in 54 leukemia lines with an uncontrolled lineage confounder, and nothing here says KAT7 binds the fusion.
- A method error found in my own two previous analyses is a reason to distrust the third, not to trust it more because it is newer.
- Lineage restriction within leukemia is still uncontrolled by every null run here. What design actually removes it — matching on lineage-restricted expression, or conditioning on a lineage factor rather than on variance?
- MEF2C cannot be resolved by pool depth because its variance is extreme and few genes match it. Is variance matching simply the wrong control for a gene like that, and what replaces it?
- KAT7 survives Holm correction and already has chemical matter. Does that move it ahead of LEDGF in the target queue, or does the lineage confounder apply to it more strongly than to LEDGF?
- Both earlier nulls used a criterion now shown to be unstable. Which other conclusions in this graph rest on 'exceeds the maximum of a null' and need rechecking?
- E1moderate confidence 3 of 5: One strong study, or several smaller consistent ones.dataset
Against variance-matched nulls drawn from 2,743 usable random protein-coding genes across the same 54 leukemia lines, correlation with MEN1 gene effect gives Holm-corrected empirical p below 0.05 for KMT2A (r=0.783), PSIP1 (0.686), DOT1L (0.681), FLT3 (0.645), MEF2C (0.526) and KAT7 (0.517), and above 0.05 for MEIS1 (0.468, Holm 0.067), DPY30, MLLT1 and BCL2; the controls ACTB and GAPDH give p 0.52 and 0.50. KAT7's empirical p of 0.00318 is exactly 1/314, i.e. a single random gene in its matched set exceeded it.
DepMap CRISPR (Chronos) gene effect via the Open Targets Platform GraphQL API, with variance-matched nulls drawn from a 3,000-gene sample of the HGNC protein-coding catalogue; analysis in analyses/depmap-null-deep-pool/ · Broad Institute DepMap; HGNC; Open Targets Platform; analysis run by this project · Databases and derived analysis 2026What this evidence cannot show
Our own computation, not peer reviewed. Correlation across cell lines is not evidence of physical interaction. All 54 lines are leukemia lines and no null in this project has addressed co-essentiality within a lineage, which remains the largest uncontrolled confounder. MEF2C's matched set is only 31 genes even from this pool, so its empirical p floor is 0.032 and its apparent significance is the weakest of the six. DepMap release version is not stated by the API, so this is not pinned to a data freeze.
Evidence that argues against this (2)
- This analysis contradicts the entry that requested it. ana-depmap-null-variance-matched concluded that MEF2C, KAT7 and MEIS1 'now exceed their matched nulls'; at six times the pool depth only MEF2C does, and MEIS1 fails Holm correction. Where two runs of the same test on the same data disagree, the disagreement is evidence about the method, and here it is: 'exceeds the null maximum' is an order statistic that rises with sample size and should not have been used as a pass criterion in either earlier analysis. analysis run by this project 2026
- A co-dependency that survives every null is still only a correlation across cell lines. The project's own lineage-conditioned analysis previously overturned the MYB claim on exactly this ground, so a surviving gene here should be treated as a hypothesis to test, not a validated partner. KAT7 in particular has chemical matter available, which makes over-reading it costly. analysis run by this project 2026
The deep-pool null withdraws the DPY30 co-dependency on which this hypothesis rests: DPY30 does not survive Holm correction against a 2,743-gene variance-matched pool
Superseded on the key number: the deep-pool re-run places DPY30 among the genes that fail Holm correction
MEF2C is one of six genes surviving Holm correction against the 2,743-gene variance-matched null (r 0.526)
at six times the pool depth, cycle 17's reversal for KAT7 and MEIS1 does not replicate on its own criterion; the criterion itself is unstable
same methodological objection: 'exceeds the null maximum' rises with pool size and is not comparable across analyses
both apply Holm correction to the same candidate set; this one supplies the empirical null the bootstrap could not
the lineage confounder this analysis cannot remove is the one that analysis addressed for a different question
corrects the co-dependency column for MEF2C and KAT7; the LEDGF conclusion survives but KAT7 is now held back by constraint, not by absence of signal
A different control on the same ranking: the deep null pool addresses sampling, this addresses a named biological confound.
- ← related toThe assumption the whole project rests on, tested at last: menin dependency is fusion-specific (run)
different statistic, partly different ordering: DOT1L ranks third by correlation and ninth by group difference
removes the lineage confounder that entry named as its largest remaining weakness; the top four survive nearly untouched
the deep-pool Holm survivors are the co-dependency criterion used for the CCNT1, CTR9 and MEIS1 verdicts