Oncofusion

start / graph / ana-q156-the-promoter-tables-second-blind-spot-is-not-annotation-drift-it-is-one-row-per-symbol

Analysisstrong confidence 4 of 5: Replicated in peer-reviewed work by more than one group.

The promoter table's second blind spot is not annotation drift at 1.8% - it is one row per symbol at 14.7%, and the case that motivated the question is not displaced at all

also called q156 displacement audit · the promoter table is 81.6% TSS-identical · one row per symbol is the real blind spot · C4orf36 is not displaced

This project scores gene activity in small windows built from a 2020 gene annotation. The worry was that newer annotations have moved those windows. They mostly have not - 82% sit on exactly the same spot. The real problem is different: a gene can have several start sites, and the table keeps only one.

The same thing, for a scientist

q155 specified a whole-table audit of promoter-window displacement, marked it **PLANNED, not run**, and hypothesised that displacement against the current annotation is **systematic**. **I ran it. The hypothesis is refuted, and the refutation names a larger blind spot.** All 19,404 rows of `C:/Temp/q145irx1/promoters.tsv` (a 2020 `refGene` snapshot, NM_-derived) joined against the **current UCSC hg38 `ncbiRefSeqCurated`** (fetched 2026-09-24, HTTP 200, 4.80 MB). Computed **NM_-only** as the like-for-like comparison and **all-curated** as the broader one, because scoring an NM_-derived table against NM_+NR_ would inflate the apparent mismatch [E1]: | | NM_ only | all curated | |---|---:|---:| | rows | 19,404 | 19,404 | | **TSS identical to a current TSS** | **15,839 (81.63%)** | 15,927 (82.08%) | | shifted, windows still overlap | 2,688 | 2,683 | | **DISPLACED - window disjoint** | **355 (1.83%)** | 360 (1.86%) | | symbol absent from current release | 522 | 434 | | moved chromosome | **0** | **0** | | **symbol has multiple non-overlapping TSSs** | **2,851 (14.69%)** | 2,960 (15.25%) | | row window contains a FOREIGN current TSS | 2,754 (14.19%) | 4,391 (22.63%) | 🟢 **Annotation drift is bounded and small: 1.83%.** Against q153's first blind spot (neighbours with no row) at **7.2 percentage points**, the second blind spot as q155 framed it is **four times smaller**, and **no row has moved chromosome**. This is a reassuring negative for every promoter-scale call this project has made. 🔴 **But the case that motivated the hypothesis is not displaced.** `C4orf36`'s scored row sits at chr4:86,892,422 and **that is exactly one of its three current TSSs - `d_nearest = 0`.** It is not in the displaced set. What is true is that C4orf36 has **3 TSSs spanning 43,782 bp**, and the table keeps the **distal** one, 42.6 kb from AFF1 - while a **different, equally current** C4orf36 TSS sits **278 bp from AFF1's**. The row is not wrong; it is **one of several right answers, and the table has room for only one** [E2]. 🔵 **So the real second blind spot is one-row-per-symbol, and it is 14.69%** - 2,851 symbols with multiple non-overlapping TSSs, **median span 21,291 bp**, and **1,939 of them with an alternate TSS more than 10 kb from the scored one**. That is **8x** the displacement rate and it is not fixable by re-downloading an annotation: it is a consequence of the table's one-row-per-symbol schema [E1]. 🟠 **And an independent confirmation of q153 fell out of it.** q153's sharing census gave **14.37%** within-table and **21.60%** against the full annotation, computed on the 2020 release. My foreign-TSS-in-window count on the **current** release gives **14.19%** and **22.63%**. **Two annotation releases, two implementations, the same two numbers** - q153's census is reproducible and is not an artefact of the snapshot it was run against [E3].

Go deeper
What this does and does not license

🟢 Licensed: promoter-scale occupancy calls in this project are not broadly compromised by annotation age. 81.6% of windows are on the identical coordinate, 0 rows changed chromosome, and only 1.83% have a window disjoint from every current TSS of their own symbol.

🔴 Not licensed: any per-gene call at a symbol with multiple non-overlapping TSSs. For those 2,851 symbols the question "is gene G occupied?" is underspecified - the answer depends on which TSS the table happened to keep, and C4orf36 shows the two answers can differ by 42.6 kb and flip the call. The 355 displaced rows are listed by symbol in results_NM_only.json, and so are all 2,851 multi-model symbols, because a count nobody can re-derive is a claim.

🟠 The 522 symbols absent from the current NM_ set are a third, smaller category worth separating: those rows score a symbol that no longer has a curated protein-coding transcript under that name. They were not part of q155's question and are not analysed further here.

🔵 The displacement tail is real but is not a leukaemia problem. The largest displacements are ICOSLG, PWP2, CRYAA, U2AF1 at 36-39 Mb on chr21 - almost certainly the chr21 coordinate revision between releases - followed by segmental-duplication families (GOLGA6L7, RGPD5, NUDT4B, TRIM73, SPDYE13/15, NPIPA7, TBC1D3I). U2AF1 is the one with disease relevance and is worth a targeted re-score before any U2AF1 statement is made from this table.

Facts and figures
scriptanalyses/q156-promoter-window-displacement-audit/run.py
statusrun
executes plan fromhyp-q155-window-displacement-against-the-current-annotation-is-systematic-not-a-one-off-at-aff1
identities persistedtrue
multi model span median bp21291
multi model span p90 bp163303
multi model alt tss over 10kb1939
irx1 d nearest0
irx2 d nearest0
NM only
moved_chromosome0.000moved_chromosome: 0pct_displaced1.830pct_displaced: 1.83pct_foreign14.2pct_foreign: 14.193pct_multi_model14.7pct_multi_model: 14.69displaced_disjoint355.0displaced_disjoint: 355absent_from_current522.0absent_from_current: 522shifted_overlapping2,688shifted_overlapping: 2688foreign_tss_in_window2,754foreign_tss_in_window: 2754multi_model_symbols2,851multi_model_symbols: 2851tss_identical15,839tss_identical: 15839n_rows19,404n_rows: 19404
all curated
pct_displaced1.855pct_displaced: 1.855pct_foreign22.6pct_foreign: 22.629displaced_disjoint360.0displaced_disjoint: 360multi_model_symbols2,960multi_model_symbols: 2960foreign_tss_in_window4,391foreign_tss_in_window: 4391
q153 reproduction
q156_nm_only_pct14.2q156_nm_only_pct: 14.193q153_within_table_pct14.4q153_within_table_pct: 14.37q153_full_annotation_pct21.6q153_full_annotation_pct: 21.6q156_all_curated_pct22.6q156_all_curated_pct: 22.629
inputs
[
  "C:/Temp/q145irx1/promoters.tsv (19,404 rows, refGene snapshot 2020-08-17)",
  "UCSC hg38 ncbiRefSeqCurated.txt.gz, fetched 2026-09-24, 4,801,131 bytes, HTTP 200"
]
outputs
[
  "results_NM_only.json",
  "results_all_curated.json",
  "results_summary.json"
]
NM only — the numbers behind the chart
{
  "n_rows": 19404,
  "tss_identical": 15839,
  "shifted_overlapping": 2688,
  "displaced_disjoint": 355,
  "pct_displaced": 1.83,
  "absent_from_current": 522,
  "moved_chromosome": 0,
  "multi_model_symbols": 2851,
  "pct_multi_model": 14.69,
  "foreign_tss_in_window": 2754,
  "pct_foreign": 14.193
}
all curated — the numbers behind the chart
{
  "displaced_disjoint": 360,
  "pct_displaced": 1.855,
  "multi_model_symbols": 2960,
  "foreign_tss_in_window": 4391,
  "pct_foreign": 22.629
}
c4orf36
{
  "row_tss": 86892422,
  "current_nm_tss": [
    86892422,
    86934732,
    86936204
  ],
  "d_nearest": 0,
  "displaced": false,
  "span_bp": 43782,
  "n_tss": 3
}
aff1
{
  "row_tss": 86935001,
  "current_nm_tss": [
    86935010,
    87007270
  ],
  "d_nearest": 9,
  "foreign_tss_in_window": [
    "C4orf36"
  ]
}
largest displacements
[
  "ICOSLG 39.2 Mb",
  "PWP2 39.0 Mb",
  "CRYAA 36.6 Mb",
  "U2AF1 36.6 Mb",
  "GOLGA6L7 6.1 Mb"
]
q153 reproduction — the numbers behind the chart
{
  "q153_within_table_pct": 14.37,
  "q156_nm_only_pct": 14.193,
  "q153_full_annotation_pct": 21.6,
  "q156_all_curated_pct": 22.629
}
measurements
[
  {
    "quantity": "promoter rows whose 2 kb window is disjoint from every current TSS of their own symbol",
    "value": 1.83,
    "unit": "percent",
    "subject": "annotation drift in the project's 2020 refGene-derived promoter table, NM_-only comparison",
    "universe": "19,404 rows; 81.63% TSS-identical, 0 moved chromosome, 522 symbols absent from the current release"
  },
  {
    "quantity": "symbols in the promoter table having multiple non-overlapping TSSs, of which only one is scored",
    "value": 14.69,
    "unit": "percent",
    "subject": "the one-row-per-symbol schema, the actual second blind spot",
    "universe": "2,851 of 19,404 symbols; median span 21,291 bp; 1,939 with an alternate TSS >10 kb away"
  }
]
Still open
  1. Re-score the 355 displaced rows and the 1,939 wide multi-model symbols at both windows and count how many occupancy calls change. That is q155's planned step 5 and the only step that converts this ambiguity bound into an error rate.
  2. Re-score U2AF1 specifically. It is the one gene in the large-displacement tail with disease relevance, and no U2AF1 statement should be made from this table until it is checked.
  3. Decide whether the promoter table should carry one row per TSS rather than one per symbol. The C4orf36 case shows the schema, not the annotation, is what loses the information.
  4. Resolve the 522 symbols absent from the current NM_ release. Symbol renaming hides displacement from this method, so those rows are the uncertainty in the 1.83% bound.
Every source behind this entry (3)
  1. E1strong confidence 4 of 5: Replicated in peer-reviewed work by more than one group.dataset

    Joining all 19,404 rows of the project's 2020 refGene-derived promoter table against the current UCSC hg38 ncbiRefSeqCurated release gives, for NM_-derived transcripts: 15,839 rows (81.63%) whose TSS is identical to a current TSS of the same symbol, 2,688 shifted but with overlapping 2 kb windows, 355 (1.83%) displaced so that the windows are disjoint, 522 symbols absent from the current release, 0 rows that changed chromosome, and 2,851 symbols (14.69%) having multiple non-overlapping TSSs with a median span of 21,291 bp of which 1,939 have an alternate TSS more than 10 kb from the scored one.

    What this evidence cannot show

    Comparison is by gene SYMBOL, so a symbol that was renamed between the two releases appears as 'absent from current' rather than as displaced, which means the 522 absent rows may contain genuine displacements that this method cannot see - the 1.83% is therefore a lower bound on displacement and the 522 are the uncertainty. Alt, random and fix contigs were dropped from the reference. 'Multiple non-overlapping TSSs' is defined as a >=2 kb span between the extreme TSSs of the symbol on the row's chromosome, which counts long first-intron alternate promoters as separate models. Occupancy was NOT re-scored at the alternate windows in this run, so the number of calls that would change is unmeasured - q155's planned step 5 remains undone.

    Evidence that argues against this (2)
    • A 1.83% displacement rate does not mean 1.83% of this project's CONCLUSIONS are safe from annotation age, because the project's conclusions are concentrated in a small number of genes rather than spread over the table. A single displaced row at a load-bearing locus would matter more than 355 at random ones; the load-bearing loci checked here (IRX1, IRX2, AFF1) all have d_nearest of 0 or 9 bp, but that is four genes, not an argument about the rest.
    • The 14.69% multi-model figure counts symbols, not calls, and most of those symbols are never used by this project. Treating it as the size of a defect in the project's output would be the same error as treating a name-drop as a dependency, which this corpus has recorded. The honest statement is that it bounds the AMBIGUITY of the table, not the error rate of the findings.
  2. E2strong confidence 4 of 5: Replicated in peer-reviewed work by more than one group.dataset

    C4orf36, the single case that motivated the displacement hypothesis, is not displaced: its scored row TSS chr4:86,892,422 is exactly one of its three current NM_ TSSs (86,892,422, 86,934,732, 86,936,204), giving d_nearest = 0. It is instead a multi-model symbol with a 43,782 bp span, and the table keeps the distal TSS while a different current C4orf36 TSS lies 278 bp from AFF1's current TSS (86,935,010). AFF1's own scored window chr4:86,934,001-86,936,001 contains C4orf36's TSS as a foreign occupant, and AFF1's own row is displaced by only 9 bp.

    What this evidence cannot show

    This corrects the framing of q155's hypothesis but not its underlying observation: q155 was right that the scored C4orf36 window is ~42 kb from AFF1 and that a C4orf36 TSS sits within a few hundred bp of AFF1's, and right that this makes the AFF1 window unattributable. Only the diagnosis - 'displaced against a newer annotation' - is wrong. The 278 bp figure is computed from the current release; q155 reported 269 bp, and the small difference was not chased to its source, which is most likely a different transcript chosen as AFF1's representative TSS.

    Evidence that argues against this (1)
    • Calling C4orf36 'not displaced' depends on accepting symbol-level matching: if the 2020 table intended the promoter-proximal C4orf36 model and the release changed which transcript is canonical, then from the table-builder's point of view the row IS pointing at the wrong place, and d_nearest = 0 merely records that the old coordinate still exists somewhere in the new release. The two framings differ in what they treat as the unit of truth, symbol or transcript.
  3. E3moderate confidence 3 of 5: One strong study, or several smaller consistent ones.dataset

    The foreign-TSS-in-window census computed here on the current annotation reproduces q153's census computed on the 2020 annotation: 14.19% of rows have a foreign NM_ TSS inside their 2 kb window against q153's 14.37% within-table figure, and 22.63% against the full curated annotation against q153's 21.60% full-annotation figure.

    What this evidence cannot show

    The two censuses are not identical in construction: q153 counted shared windows between table rows and, separately, against a full annotation, while this run counts any foreign current TSS falling inside a row's window. They agree to within 0.2 and 1.0 percentage points respectively, which is close enough to call reproducible but not a re-implementation of the same statistic. The all-curated figure being HIGHER here than q153's (22.63 vs 21.60) is expected, since NR_ transcripts add TSSs.

    Evidence that argues against this (1)
    • Agreement between two censuses that both count TSS proximity in the same genome is weaker evidence than it looks - they share the dominant driver, which is the real density of bidirectional promoters, so they could agree while both mis-specifying the window. This corpus has recorded that two derivations asking the same question measure consistency rather than validity.
Connected to