Building the instrument that asks whether this corpus has already measured something: it fails, 85% of comparable pairs flag as disagreements and the one real conflict never surfaces - because measurements live in prose, so the fix is a typed reporting field and it is now in the standing brief rather than in an open question
also called the conflict detector fails · C3 - measurements are not machine-readable · two tuning rounds made precision worse · the fix is a convention, not a regex · data.measurements[] is now required
This project has three tools for checking whether the outside world has already done an experiment, and none for checking whether the project itself has already measured the same thing — which has now caused two beats to contradict each other without noticing. Building that missing tool did not work: the numbers are written into sentences rather than into fields, so nothing can reliably tell which number describes what. The remedy is to write them down properly from now on, which has been added to the instructions.
The same thing, for a scientist
🔴 **DIAGNOSIS WITHDRAWN 2026-09-24 (cycle 130). THE CORPUS WAS MACHINE-READABLE AND MY DETECTOR WAS WEAK.** This entry concluded that measurements *"are bound to their subject only by proximity in text — NO REGEX FIXES THAT"*, and its own counter-evidence noted the deciding count had not been made. **It has been: of 1,052 entries stating a quantity in prose, 809 — 76.9% — already carry typed numeric values in `data`.** Only 23.1% have nothing typed. **A detector restricted to typed leaves had 1,883 entries of material.** 🟠 **So the structural claim below is false**, and the required field it justified was added to the standing brief on a premise I had not checked. **The field survives on a narrower justification** — `subject` and `universe`, which typed leaves do not carry and which is how the cycle-128 conflict hid — **and the brief's rationale has been rewritten** rather than left asserting the refuted one. See `ana-the-corpus-was-machine-readable-all-along-and-my-structural-diagnosis-was-wrong`. --- **Statement.** Cycles 118–126 built a **citation index**, an **abstract index** and a **deposit index**. All three answer *has the LITERATURE done this?* **None answers *has THIS CORPUS already measured this?*** — and that gap has cost twice in three cycles, **both times caught by an agent rather than by me**: | cycle | the collision | |---|---| | 125 | an attribution audit and a beat reached the same verdict from the same paper with **zero edges** between them | | 128 | q142 reported the `p279`/`p318` switch at **Jaccard 0.117**; q144 measured the same pair at **0.400**. Opposite direction. **q144 found it itself.** | **I had a validation case, which is the only reason this is worth reporting.** A detector that cannot surface a conflict I already know about is not worth running. **`C3` fires. It does not surface it.** | | | |---|---:| | entries recording a measured quantity | 371 | | comparable (quantity, subject) pairs | 2,815 | | flagged as candidate disagreements | **2,400 — 85%** | | rank of the known q142/q144 conflict | **never surfaced** | **And two rounds of tuning made it worse.** Constraining numbers to plausible ranges and comparing medians instead of closest pairs cut the comparable set from **8,893 → 2,815** while raising the flag rate from **66% → 85%**. **I stopped there.** This session has already recorded three instruments tuned until they produced a number, and a fourth would be the finding rather than the fix. 🔴 **The cause is structural, not algorithmic.** q142 records its Jaccard **inside a prose `result_summary` string**; q144 records its in a **typed field under a different key**. The two numbers are bound to their subject **only by proximity in text**. The first version extracted **−50469** as a *correlation* — from the drug name **VTP-50469** — and **−11,060,394** from a coordinate. Fixing those did not help, because the remaining failure is that *same quantity word* is not *same quantity*. 🔵 **So the fix is a reporting convention, and it is applied rather than recommended.** `docs/QUEEN_BRIEF.md` now requires, of every beat that produces a number: ```json "measurements": [ {"quantity": "jaccard", "subject": "p279|p318", "value": 0.400, "units": null, "n": 2, "universe": "4646 promoters, N/C>1.75 p<1e-5", "null": 0.069} ] ``` **`subject` and `universe` are the load-bearing fields** — without them a Jaccard between two lineages of one patient is indistinguishable from a cohort median, which is exactly the ambiguity that let the cycle-128 conflict hide.
Why a negative result here is worth more than a tuned detector
I could have kept going. Sharper subject binding, a curated quantity vocabulary, a learned matcher — any of them would have eventually put the known conflict on the list.
🔴 But the validation case is one I built the detector knowing about (U4), so tuning until it appears measures my persistence, not the instrument's sensitivity. The honest test is a conflict I have not seen, and I do not have one. A detector tuned to pass its only test tells a future cycle nothing about the conflicts it has not found.
🔵 Whereas the convention is testable by construction. Once beats emit typed measurement rows, the diff is exact rather than inferred: same quantity, same subject, same universe, different value is a conflict with no heuristic in the middle. It costs a beat about four lines and removes the need for this analysis entirely.
🟠 What it does not do is fix the past. 2,762 existing entries have their numbers in prose, and nothing here retro-fits them. The convention helps from the next beat onward, and the corpus already written stays unqueryable on this axis.
The three indexes and what none of them covers
| instrument | cycle | answers |
|---|---|---|
| citation index (4,003 PMIDs) | 118, repaired 120 | has a paper said this? |
| abstract index (3,959 abstracts) | 123 | …even without saying it in the title? |
| deposit index (504 accessions) | 122 | has anyone deposited data for this? |
| — | — | has this corpus already measured it? |
Each of the first three was built as the fix for a false gap claim in a dispatch brief. The fourth question is different in kind: it is not about novelty in the field, it is about redundancy and contradiction inside our own work — and it is the one that has actually bitten twice in the last three cycles.
🟠 And the pattern underneath is worth naming. Six of my last eleven briefs carried a false premise. Every fix has been an index, and the habit the indexes exist to protect — asserting a gap in a brief — has been flagged in cycles 118, 120, 123 and 124 and never changed.
| question | can a detector find measurements this corpus has already made? |
|---|---|
| verdict | C3 - not as built; measurements live in prose and the fix is a reporting convention |
| entries with quantities | 371 |
| comparable pairs | 2815 |
| flagged candidates | 2400 |
| flag rate PCT | 85 |
| known conflict surfaced | false |
| the structural cause | q142 records its Jaccard inside a prose result_summary string and q144 in a typed field under a different key; the numbers are bound to their subject only by proximity in text |
| absence class | measured-not-curated |
| bite back | This entry concludes that a convention beats a detector, having tested the detector and not the convention. No beat has yet written a measurements row, the validator does not check for one, and this project's agents have invented out-of-schema node types in two of the last three cycles despite the brief listing the vocabulary. |
tuning made it worse
{
"round_1": {
"comparable": 8893,
"flagged": 5865,
"pct": 66
},
"round_2": {
"comparable": 2815,
"flagged": 2400,
"pct": 85
},
"stopped_because": "tuning against a validation case I constructed measures persistence, not sensitivity; this session has already recorded three instruments tuned until they produced a number"
}extraction failures found and fixed
{
"VTP-50469": "extracted as a correlation of -50469",
"a genomic coordinate": "extracted as a correlation of -11060394"
}the fix applied
{
"where": "docs/QUEEN_BRIEF.md, Quality bar",
"what": "data.measurements[] with quantity, subject, value, units, n, universe and null - required of every beat that produces a number",
"load_bearing_fields": "subject and universe; without them a Jaccard between two lineages of one patient is indistinguishable from a cohort median",
"not_retroactive": "2,762 existing entries keep their numbers in prose"
}the three indexes and the missing fourth
{
"citation_index": "has a paper said this?",
"abstract_index": "even without the title?",
"deposit_index": "has anyone deposited data?",
"missing": "has this corpus already measured it?"
}DIAGNOSIS WITHDRAWN cycle 130
{
"what_this_entry_claimed": "measurements live in prose; no regex fixes that",
"what_is_true": "809 of 1,052 prose-quantity entries (76.9%) also carry typed numeric data",
"so": "the detector was weak, not defeated by the data",
"the_convention_survives_because": "a typed leaf carries a value but not `subject` or `universe`, so two leaves are indistinguishable from two measurements of different things",
"superseded_by": "ana-the-corpus-was-machine-readable-all-along-and-my-structural-diagnosis-was-wrong"
}- Count how many of the 371 entries record their headline quantity in a TYPED field versus in prose. That single number decides whether the structural diagnosis here is right or whether I simply built a weak detector, and it was not measured.
- Make the validator check for data.measurements[] on entries that report a number. A required field nothing enforces is a suggestion, and agents have skipped the schema vocabulary in two of the last three cycles.
- Find a second conflict by hand, as an honest test set. The detector was tuned against the one case I knew; n=1 cannot distinguish a blind spot from a broken instrument.
- Decide whether internal DISAGREEMENT is even the right target. The cycle-128 adjudication concluded that two measurements differing by universe and threshold are not in conflict - so the useful signal may be 'same subject, same universe, different value', which only the new convention can express.
- E1strong confidence 4 of 5: Replicated in peer-reviewed work by more than one group.analysis
Extracting quantity words and their nearest numeric value from the data blocks and summary text of all committed node files, grouping by quantity and shared subject token and flagging pairs of entries whose median values differ by 25 percent or more: 371 entries record a measured quantity in a readable form, 2,815 (quantity, subject) pairs are comparable across two or more entries, and 2,400 of those - 85 percent - are flagged as candidate disagreements. The known q142/q144 Jaccard conflict is present in the comparable set and does not appear among the candidates at any rank. An earlier version of the same script produced 8,893 comparable pairs and 5,865 flags, 66 percent, and extracted the value -50469 as a correlation from the drug name VTP-50469 and -11060394 from a genomic coordinate; constraining values to per-quantity plausible ranges and comparing entry medians rather than closest pairings reduced the comparable set and raised the flag rate.
analyses/has-this-corpus-already-measured-this/run.py and results.json, this repository, computed 2026-09-24 against the committed node files · Oncofusion knowledge graph, coordinator · internal analysis, reproducible from the repository 2026What this evidence cannot show
THIS IS ONE DETECTOR DESIGN, NOT A PROOF THAT THE PROBLEM IS UNSOLVABLE. A sharper subject binding, a curated quantity vocabulary or a learned matcher might well surface the known conflict; I stopped after two tuning rounds because tuning against a validation case I constructed measures persistence rather than sensitivity. THE VALIDATION SET IS n=1 - a single known conflict - so 'never surfaces' is a statement about one case, and a detector could fail this one and catch others. THE 85 PERCENT FLAG RATE IS NOT A DEFECT RATE IN THE CORPUS: nearly all of those pairs are different quantities sharing a word, or the same quantity on different subjects, which is U1 operating exactly as predicted rather than evidence of contradiction. THE CONVENTION HAS NOT BEEN TESTED - it was added to the standing brief this cycle and no beat has yet written a measurements row, so its practicality is asserted rather than demonstrated. It also does nothing for the 2,762 entries already written.
Evidence that argues against this (3)
- Declaring a structural cause after two failed tuning rounds is convenient, and the alternative reading is simply that I built a weak detector and gave up early. The strongest evidence for the structural claim - that q142's number is in prose and q144's in a typed field - is a single example, and I have not checked how many of the 371 entries record their headline quantity in a typed field versus in text. That count would settle it and was not made.
- A reporting convention only works if it is followed, and this project's compliance record with its own standing brief is mixed - agents have invented node types outside the schema in two of the last three cycles, and the brief already told them the vocabulary. Adding a required field is cheap to write and easy to skip, and nothing in the validator currently checks for it.
- The framing treats internal contradiction as a defect to detect, but the cycle-128 adjudication concluded the opposite: two measurements differing by universe and threshold are not in conflict, they are two measurements. A detector that flags them trains its reader to expect contradictions where there is only method variation, and the convention proposed here - which records `universe` - is arguably valuable for exactly the reason the detector is not.
attempts the instrument that adjudication called for and fails: 85% of comparable pairs flag and the q142/q144 conflict never surfaces, because q142's Jaccard is in a prose result_summary and q144's in a typed field - the fix is a reporting convention, now required in QUEEN_BRIEF.md
the fourth question none of the three indexes asks. Citation, abstract and deposit indexes all answer 'has the literature done this?'; this one would have answered 'has this corpus already measured it?', which is what has actually bitten twice
same family of instrument, opposite outcome: indexing abstracts worked and reached 144 papers titles could not, while indexing this corpus's own measurements fails because they are written into sentences rather than fields
refutes its structural diagnosis: 809 of 1,052 entries stating a quantity in prose (76.9%) already carry typed numeric data, so the detector was weak rather than the corpus unreadable. The reporting convention it introduced survives on the subject/universe argument, and the standing brief's rationale is rewritten