The 1 kb bin-overlap retention measure this project uses for eviction FAILS a degron positive control: it reports the fusion as still occupying 40-93% of its own sites in cells where the fusion has been destroyed, because a collapsing ChIP signal makes MACS call FEWER but 1.9x WIDER peaks - so it fails hardest exactly where the protein is most completely gone
also called the bin measure is broken · 40.1% versus 2.5% · wider peaks not fewer bins · the metric failed its positive control
This project measures whether a drug knocks a protein off DNA by asking how many one-kilobase chunks of the genome still overlap a detected signal, so we tested that yardstick on a case where the answer is certain - the protein had been deliberately destroyed - and the yardstick said it was still there. The reason is that when a signal collapses the peak-detection software returns a few very broad, blurry regions that blanket the chunks without actually sitting on the original spots.
The same thing, for a scientist
**The test.** A degron gives a positive control with a known answer: destroy the HA-tagged fusion, and its own HA ChIP must vanish. GSE173599 supplies five such arms. | HA arm | **1 kb bin overlap** (the measure in use) | **summit containment** (strict) | |---|---:|---:| | 3 h, batch 1909x2 | **40.1%** | 2.5% | | 3 h, batch 190814 | **74.5%** | 7.1% | | 24 h | **93.2%** | 15.6% | | 4 d, batch 1912x2 | **90.2%** | 10.3% | | 4 d, batch 200908 | 0.9% | 1.0% | **Four of five arms read above the 33% 'cleared' threshold under the bin measure and three read above 67%** — i.e. the measure says the oncoprotein is still on chromatin in cells where it was degraded. The strict measure clears **5 of 5**. **THE MECHANISM, and it is the part that generalises.** MACS2 was run here with `--slocal 1500` on paired-end libraries whose fragment-size estimates reach **710-866 bp**, so calls are broad. When a signal collapses, the surviving calls do not merely become fewer — **they become much wider**. Median peak width in the dVHL arm is **1.92x** the DMSO arm's across the HA arms (range 1.17-2.79x), against **1.09x** across every non-HA arm. A few hundred broad residual calls blanket the 1 kb reference bins **without covering a single summit**. **So bin overlap measures how WIDE the surviving calls are, not whether the protein is still at the site — and it degrades exactly in proportion to how completely the protein has left.** For a measure whose entire purpose is to quantify eviction, that is the worst possible failure direction: it is least trustworthy at the strongest effects. **A third, independent demonstration in the same data.** Under the bin measure the two independent 4 d experiments read 0.9% and 90.2% — an apparent qualitative between-batch contradiction, which I initially recorded as a finding. Under summit containment they read 1.0% and 10.3%, in agreement. **The measure did not merely inflate a number; it invented a contradiction between replicate experiments that does not exist.**
What this does and does not mean for the figures already in this corpus
Do not mass-revise. The defect is not universal — it is triggered by near-total signal loss, and the sibling beat's MOLM13 2x2 was re-scored under both measures (ana-q139-the-sibling-beats-mll1-verdict-survives-and-strengthens-under-the-stricter-measure) and agrees to within about 6 points, with its verdict intact and slightly strengthened. Effects in the 20-60% range are largely unaffected.
The direction of the error is always the same: bin overlap OVERSTATES retention, so every eviction claim computed this way is conservative about the eviction — the real displacement is at least as large as reported, and usually larger. Claims of the form "X was retained, therefore X does not depend on the fusion" are the ones at risk, because an inflated retention figure directly supports that conclusion.
The practical rule: report summit containment alongside any bin-overlap retention figure, and treat a large divergence between the two as a diagnostic that the treated arm's calls have broadened. Both are computed by analyses/what-leaves-when-the-fusion-is-degraded/run.py for every arm.
Why only a degron could have caught this
A drug arm has no known answer, so a retention figure of 40% is simply a result and there is nothing to check it against. A degron supplies the one case where the expected answer is certain — the tagged protein is destroyed, so its tag must go — which is why this defect survived through a published beat and into a king log without anything flagging it.
The general form is worth more than the instance: a metric that has never been run against a case with a known answer has not been validated, however sensible its construction. This one was constructed sensibly and was wrong.
The summit measure, and its own verification
Summit containment asks: is the DMSO peak's point of maximum pileup (abs_summit, a MACS output column) still inside any called peak in the treated arm?
It was verified before being believed, because a broken checker is a claim too. The implementation uses a binary search with a prefix-maximum of interval ends, so nested and overlapping intervals cannot be missed. It was checked against (a) a self-control — the DMSO arm scored against itself returns exactly 100.0% — and (b) an independent brute-force O(n·m) implementation on random samples of 400 peaks from six pairs, which agrees to within sampling noise (e.g. HA 3 h b1909x2: 2.5% indexed vs 2.8% brute; MEN1: 80.0% vs 80.0%).
It is not a perfect measure either. It is still presence/absence of a threshold crossing, so it inherits U2 entirely, and it is stricter than occupancy: a site whose signal halved but whose summit drifted 200 bp into a gap between calls scores as lost.
| script | analyses/what-leaves-when-the-fusion-is-degraded/run.py |
|---|---|
| status | run |
| result summary | Bin overlap passes 1 of 5 HA positive-control arms; summit containment passes 5 of 5. Median peak width in degraded arms is 1.92x control (HA channel) vs 1.09x (all other channels). The bin measure also manufactured an apparent qualitative disagreement between two replicate 4 d experiments that the strict measure shows to be in agreement. |
| preregistration | analyses/what-leaves-when-the-fusion-is-degraded/PREREGISTRATION.md - the bin measure was the PRE-REGISTERED one and the summit re-analysis is labelled POST-HOC throughout results.json |
inputs
[ "GSE173599 all five HA DMSO/dVHL pairs", "GSE173599 eighteen non-HA DMSO/dVHL pairs" ]
- This entry is itself a measurement and could be wrong in the same way the measure it criticises was wrong. Its load-bearing comparison rests on one deposit with unusual peak-calling parameters, and the summit measure it recommends has its own failure mode - it scores a site as lost when the summit drifts out of a called region, which under-reports retention for broad or shifting factors.
- If any clinical or preclinical decision in this project has rested on a bin-overlap retention figure showing that a component STAYS - and therefore does not depend on the fusion - that inference is the one most exposed, because the metric's error runs in exactly that direction.
- How many of this corpus's existing bin-overlap retention figures involve a treated arm whose calls broadened? The peak tables are deposited for most of them, so this is a re-scoring job, not a new experiment - and it should be run before any of those figures is quoted again.
- Is the broadening a MACS2 artefact of --slocal 1500 under low signal-to-noise, or does it reflect genuine redistribution of residual protein into broad domains? Re-calling the deposited BAMs with narrower local-lambda settings would separate these, and the TDF coverage tracks are deposited.
- Does a signal-level measure (normalised read counts in fixed windows) behave better than either presence/absence measure on the same positive control? The authors used one; this analysis did not, because only peak tables were downloaded.
- E1established confidence 5 of 5: Multiple independent lines of evidence; treated as settled in the field.paper
GSE173599 contains HA ChIP before and after degradation of an HA-tagged MLL::AF9, providing a positive control with a known expected answer against which a retention metric can be validated.
MLL::AF9 degradation induces rapid changes in transcriptional elongation and subsequent loss of an active chromatin landscape. · Olsen SN, Godfrey L, Healy JP, Choi YA, Kai Y, Hatton C, Perner F, Haarer EL, Nabet B, Yuan GC, Armstrong SA · Molecular Cell 2022What this evidence cannot show
The metric failure is demonstrated in ONE deposit, whose MACS2 runs used --slocal 1500 on paired-end data with unusually large fragment-size estimates (710-866 bp). A deposit called with narrower settings would show a smaller divergence between the two measures. The claim that bin overlap overstates retention is general; the magnitude measured here is specific to these peak-calling parameters and should not be transferred as a correction factor.
Evidence that argues against this (1)
- One HA arm (4 d, batch 200908) agrees between the two measures to within 0.1 points, so the bin measure is not always wrong even under total signal loss. The DMSO arm there has a median peak width of 722 bp, the narrowest of any HA arm, which is consistent with the width mechanism rather than a counter-example to it - but it shows the failure is parameter-dependent, not automatic. NCBI Gene Expression Omnibus 2022
- E2strong confidence 4 of 5: Replicated in peer-reviewed work by more than one group.dataset
MACS2 peak widths in the degraded arms are 1.92x the control arms' on average across the HA channel, against 1.09x across all non-HA arms, which is the mechanism by which bin overlap inflates retention.
GSE173599 MACS2 peak tables, median peak width per sample, computed in analyses/what-leaves-when-the-fusion-is-degraded/run.py · Olsen SN, Armstrong SA, et al. (data); re-analysis this cycle · NCBI Gene Expression Omnibus 2022What this evidence cannot show
Median peak width is a summary of a heavily skewed distribution - one MLL1 call in this deposit spans 113 kb. The widening ratio is computed from five HA arms and eighteen non-HA arms, n = 1 library each, so it is descriptive and carries no interval. The causal claim that width inflates bin overlap is supported by the arithmetic of the measure rather than by an experiment.
Evidence that argues against this (1)
- An alternative explanation for both the widening and the bin/summit divergence is that residual chromatin-bound fusion genuinely redistributes into broad low-occupancy domains rather than disappearing. Nothing in a peak table can exclude that, and it would mean the bin measure is reporting something real that the summit measure discards. Oncofusion knowledge graph 2026
The degron arm IS the positive control that arbitrates between the two measures. Without a case whose answer is known in advance, neither measure could have been shown wrong.
Adds a fourth thing occupancy cannot separate, and it is instrumental rather than biological: a 1 kb bin-overlap retention figure cannot separate a protein still being at a site from MACS calling a wider, blurrier peak where the signal collapsed.
BOUNDS THE DEFECT RATHER THAN LETTING IT SPREAD: the MOLM13 arms show no peak broadening and the two measures agree to within ~6 points there, so the metric failure is triggered by near-total signal collapse and does not invalidate mid-range retention figures.
replicated in all three HA batches: summit containment 1.9/6.4/9.1% with width ratios 2.79/1.93/2.18