Six questions, asked over your own data

01The claim

That is the easy half to say. The hard half is that I built the inference engine, ran it, and it produced a confidently wrong answer on the study that ships with the product — with a better confidence score than the right answer on the study it got right. So this design cannot be "detect and pre-fill", which is what the state of the art does. It has to be something narrower and more disciplined.

02The four measurements

Each of these was executed against real files, not reasoned about. Method, code and full output are on the evidence page; the summary is here because the design is downstream of all four.

03The staircase

Computed by walking every workflow in demo-301/workflows/ and resolving each metric and module through its Mapped_* dependencies back to raw domains. The shape of that graph is the shape of the interview.

Rung 1 is the design's whole bet. The interview's success condition is not a complete mapping — it is one chart, drawn from the user's own rows, inside six answers. Everything after that is an offer, not a requirement, and every offer names its prize.

04The hard case, walked all the way

A CRO ships two files: an ADaM-shaped adsl.csv (1,005 rows, 16 columns) and a central-lab extract lab_central.csv (57,200 rows, 11 columns). No define.xml. Nothing is named the way gsm names it, the two files disagree about what a subject identifier is, three columns the pipeline wants do not exist, and the analytes are spelled by three different vendors. Every number below is measured. The clickable version is here.

1. Drop the folder. The product reads, it does not ask.

   Two files, 1,005 and 57,200 rows. The product profiles every column — type, cardinality, value shape, date-parseability — and says what it thinks each file is: "one row per person" and "many rows per person, with a measurement." It asks the user to confirm the kind of file, not the schema. That is question 1 and it is answerable by anybody.

2. The join, before anything else.

   The product sweeps every character column in file A against every character column in file B and ranks by shared distinct values. The result, run:

   adsl	lab	shared
   SUBJID	SUBJECT	765	proposed
   TRTSDT	LBDTC	88	decoy — two date columns
   USUBJID	SUBJECT	0	what the name detector picks

   Question 2 is: "These two files agree on 765 people using SUBJID and SUBJECT. Is that the right pairing?" — with the zero-overlap runner-up shown underneath, so the person can see the alternative was tested and lost. If the user picks USUBJID anyway, the product refuses and offers the repair it found: a single constant prefix, AA-AA-000-0000, whose removal takes the overlap from 0 to 765.

   Why this is question two and not question ten

   LB.yaml and AE.yaml both end in an inner join against the participant table. Getting this wrong does not produce an error. It produces zero rows out of 57,200, and six charts quietly stop existing. Today's og_validate() reads one file at a time and cannot see it.

3. Four questions over real values.

   Now the lab file. The product has already profiled it, so each question arrives with a proposal, three sample values, and the reason:

   • Which column is the result? → LBORRES. Reason shown: every one of the 16 test names carries its own pair of limits, and this is the number that sits between them.
   • Which columns are the normal range? → LBORNRLO / LBORNRHI, asked as one question because they are one fact.
   • Which column names the test? → LBTEST, with its 16 distinct values listed.
   • Which column is the visit? → VISIT, with Baseline, Week 1, … shown.

   Six answers total, counting the file kinds and the join. Units, dates, visit number, study id, site, sex, race, age, arm and treatment duration are all inferred and listed in a collapsed "also matched" block, never asked about.

4. First light — the wizard's success condition.

   The product writes the two mapping specs, runs them, and draws a histogram of the user's own ALT values. This step was executed for real: the emitted spec fed through gsm.mapping::ApplySpec() and the study's inner join yields 57,200 rows over 765 participants, with lbstresn numeric and lb_dt a Date.

   The interview should end here. Not because the mapping is complete — it is not — but because this is the moment the person acquires the only thing that lets them judge everything after it: a picture of their own study.

5. The Ledger, and the moment of highest friction.

   What is not working is now shown, priced in the product's own units rather than in column names:

   ●Six safety charts are drawing, over 765 peopleworking

   △Nothing here looks like a toxicity gradeGrade 3+ Lab KRI off

   △Nothing here marks which visit is baselineshift plot uses visit 1

   △The hepatic explorer cannot find its four analytes1 chart off

   ●Site Risk Score is computed over 1 of 12 KRIsnot comparable

   The fourth and fifth rows are where this direction is under the most strain, and they are different problems. The hepatic one is answerable — keep reading. The risk score one is not answerable by any question, because the only fix is more data.

   The absent column is where a guided setup is structurally weak

   A wizard is excellent at "which of these?" and has nothing to offer for "you need a thing you do not have." There is no candidate to click. The best this direction can do is name the absence in the user's language, name the consequence in the product's, and refuse to let it pass silently — which is strictly better than today, where the column is dropped by a purrr::keep filter with no message at any log level, and strictly worse than a direction that could offer to derive the grade from the result and the limits.

6. The one question that is about values, not columns.

   The hepatic explorer needs four analytes by name. Its module YAML carries them as literals: ALT: Alanine Aminotransferase and three more. The user's file spells them ALT, SGOT, TBILI, Alk Phos. Matched: zero of four.

   Guided setup handles this well as an interaction — a person who has never seen gsm can absolutely answer "which of your 16 test names is the liver enzyme ALT?" from a dropdown of their own values, and safetyCharts::meta_hepExplorer already publishes both the ADaM and SDTM spellings to seed the guesses. It handles it badly as an artefact: there is nowhere declarative to write the answer. The spec accepts exactly two keys, type and source_col, and neither expresses a value. The wizard has to patch a module YAML.

7. Emit, and get out of the way.

   Three files are written into the project folder: config/data-config.yaml pointing at the two CSVs where they already sit, workflows/1_mappings/SUBJ.yaml and LB.yaml carrying source_col: lines, and a new config/setup-log.md recording each question, the answer, the evidence the product offered, and the acknowledged absences in the user's own words.

   The interview is over and does not run again. Which is the beginning of this direction's real problem — see §07.

05What it does silently, and what it asks

The line is drawn by one rule: the product decides anything a wrong answer would make obvious, and asks about anything a wrong answer would hide. A mis-detected unit column is visible the moment a chart draws. A mis-detected join key is invisible forever.

06What it costs to build

Leans on, unchanged

• gsm.mapping::ApplySpec() — source_col is the entire column-rename seam and it already works. The wizard's output is a spec file; nothing new executes it. Verified end to end in this session.
• The workflow snapshot. og_init() already copies 44 workflow YAMLs into the project, so the mapping specs the wizard edits are the user's own files. No new storage, no new format.
• config/data-config.yaml — already routes domain to file path. The wizard writes it instead of the user.
• safetyCharts::meta_* — nine tables that already publish a plain-English role vocabulary and ADaM/SDTM spellings per role, including analyte values for the hepatic explorer. This is the single most valuable unclaimed asset in the portfolio for this problem, and it is sitting in the ancestor package.

Has to be built

What could go wrong

• The profiler is O(n³) in numeric columns. Eleven columns is 990 combinations and instant; an 80-column ADLB is over half a million. Pruning is needed, and every cheap pruning heuristic is name-based — which reintroduces exactly the weakness M1 measured.
• Structural purity needs an analyte column to exist. On a wide file — one column per test, one row per visit — there is no categorical to group by and the ranker has nothing to say. That shape is common in exported vitals.
• The wizard writes files the user then edits by hand. Two authors, one file, no reconciliation. The second run has to read what it wrote and detect divergence, or it will overwrite a person's corrections.
• Six questions is a claim about rung 1 only. A study that wants the operational KRIs needs four more files, and the interview grows toward the form it was designed to avoid.

07Where this direction is weakest

Five, ordered by how much they should count against it. The first is the one I would lose the argument on.

08What I would ask @jwildfire

• Ship the join check regardless of which direction wins. The overlap sweep is small, it is prototyped, and it catches a failure that today produces a plausible empty study. It belongs in og_validate() and in the run, and it is independent of every question about surfaces.
• Is "first light" an acceptable success condition? This design deliberately stops at one chart from two files and leaves the study incompletely mapped, on the argument that a picture of your own data is what makes the rest of the decisions possible. The alternative reading is that it ships people a half-configured study and calls it done.
• How much should a partial map be allowed to compute? M4 says a six-KRI study reports doubled risk scores with no warning. The options are to suppress the score below a threshold, to label it, or to leave it and let the Ledger carry the warning. I would suppress it.
• The role table — sidecar or upstream? Building it inside open.gismo is fast and needs no write to Gilead-BioStats. It is also a second vocabulary in the ecosystem, which is the thing this workspace's conventions exist to prevent.