Oncofusion

start / graph / concept-q141-a-binary-peak-call-cannot-see-a-two-fold-change

Explainerstrong confidence 4 of 5: Replicated in peer-reviewed work by more than one group.Q141-menin-at-fusion-targets

A binary peak call cannot see a two-fold change, so 'the peaks are still there' is not 'the protein is still there'

A peak caller answers yes-or-no: is there more signal here than background? Halving the amount of a protein at a strong site still leaves plenty of signal, so the answer stays yes. Counting peaks therefore cannot tell you that half of something has gone.

The same thing, for a scientist

A generalisable measurement principle, measured rather than asserted, on GSE173599. MACS peak calling thresholds on significance, so its output is a step function of enrichment with a threshold near the noise floor. Empirically in this deposit, summit-containment retention rises from 7% in the weakest control-peak decile to 83% in the strongest for MEN1, and to 99% for RNA Pol II. In the top decile the measure is therefore **saturated**: a site must lose almost everything to change the answer. The fusion's own HA channel loses 98-99% and its retention falls to 1.9-9.1%; menin loses ~50% at the same sites and its retention stays at 83%. **Both are correct measurements of different quantities.** The corollary for this corpus is that a peak-retention figure is a bound whose tightness is unknown unless the dynamic range of the assay at that signal level is stated.

Go deeper
The operational rule

Before quoting a peak-retention percentage as evidence that something did not change, state the smallest change the measure could have detected at that signal level. If that number is not available, the retention figure bounds only near-total loss. In this deposit the demonstrated detection floor is somewhere between 50% loss (invisible: menin at target genes, 83% retention) and 98% loss (visible: the fusion, 1.9% retention).

Relation to the width trap already recorded here

This corpus already documents a related failure: a 1 kb bin-overlap retention measure reports the destroyed fusion as still occupying 40-93% of its sites, because MACS returns wider peaks when signal collapses (measured treated/control median width ratio 1.93 in batch 190814, 2.79 in 1909x2, 2.18 in 1912x2). Summit containment fixes that specific failure - it gives 6.4%, 1.9% and 9.1% for the same three batches - but it does not fix the dynamic-range problem described here, which is a property of thresholding and not of geometry.

Where else this bites in this corpus

Any entry whose claim is 'X did not leave chromatin' resting on a peak-call census. The affected family includes the genome-wide MENIN, DOT1L, AFF4, CDK9, Pol II and histone-mark retention figures from this deposit. It does not affect claims of the form 'X did leave', which remain valid and if anything understated.

How this could bite us back
  • A pharmacodynamic assay chosen because it is robust and binary will be chosen precisely because it is insensitive, and will then be used to declare a drug inactive. The correct endpoint for a menin inhibitor is quantitative occupancy at a defined target-gene set, not a genome-wide peak census.
Still open
  1. Can the detection floor be calibrated directly - for instance by subsampling the control-arm reads to simulate a known fractional loss and re-calling peaks? That would convert every retention figure in this corpus from a bound of unknown tightness into a quantified one.
  2. How many entries in this graph rest on a peak-retention figure used as evidence of no change?
Every source behind this entry (3)
  1. E1strong confidence 4 of 5: Replicated in peer-reviewed work by more than one group.dataset

    Summit-containment retention in this deposit rises from 7% to 83% across control-peak strength deciles for MEN1 and from 9% to 99% for RNA Pol II, demonstrating saturation at strong sites.

    What this evidence cannot show

    Measured on one deposit, one peak caller (MACS2 with q<0.01, --slocal 1500, BAMPE) and one signal regime. The specific numbers are not portable; the shape of the problem is.

    Evidence that argues against this (1)
    • A different peak caller, or MACS run with a fixed fold-enrichment cutoff rather than a q-value cutoff, would have a different and possibly wider dynamic range - so this is a property of the analysis as deposited, not of ChIP-seq. Olsen SN et al. 2022
  2. E2strong confidence 4 of 5: Replicated in peer-reviewed work by more than one group.paper

    In the same cells and the same three hours, menin loses about half its signal at fusion target genes while retaining 83% of its strongest peak calls.

    MLL::AF9 degradation induces rapid changes in transcriptional elongation and subsequent loss of an active chromatin landscape · Olsen SN, Godfrey L, Healy JP, Choi YA, Kai Y, Hatton C, Perner F, Haarer EL, Nabet B, Yuan GC, Armstrong SA · Molecular Cell 82(6):1140-1155.e11 2022
    PMID 35245435DOI
    What this evidence cannot show

    The 50% figure is the authors' own and is housekeeping-normalised; the 83% is recomputed from the deposited peak tables. The two are not on a common scale, which is the point.

    Evidence that argues against this (1)
    • The 50% figure is housekeeping-normalised and the 83% is not, so part of the apparent contradiction between them is a scale difference rather than a dynamic-range limitation. Olsen SN et al. 2022
  3. E3strong confidence 4 of 5: Replicated in peer-reviewed work by more than one group.dataset

    The corpus already holds the sibling failure: a 1 kb bin-overlap measure reports 40-93% retention for a protein degraded to 2.5% of control.

    What this evidence cannot show

    That failure is geometric (wider treated peaks re-covering control bins) and is fixed by summit containment; the dynamic-range failure described here is not.

    Evidence that argues against this (1)
    • The bin-overlap failure is geometric and the saturation failure is statistical; grouping them may obscure that a tool fixing one leaves the other entirely untouched. Olsen SN et al. 2022
Connected to