The recommendation, before anything else. A user should put the folder on disk exactly as the
CRO sent it — wrong column names, three lab vendors, a SAS transport, no define.xml —
and run one command: og_read() opens every file it can, changes nothing, and
answers with one document that prices each gap in the charts it turns off rather than
in the columns it is missing, leaving four or five questions whose answers are single lines of
text in the study's own mapping YAML. To make that true we would build three things and no
more: the reader and its report; a diagnostic, og doctor, that re-checks every
answer on each later transfer, names the file and line that fixes each finding, and refuses to
run the pipeline while an identifier does not join; and — as v2, not v1 — a
worksheet behind the report for the transfer where four questions are not enough. Nothing here
maps data: gsm.mapping stays the engine, and every answer is written as the two
spec keys it already reads.
Ranked across the three lenses, inspect-first — the intake report — is the only direction that is never below second. Convention is one point behind it. The interesting part is that they do not disagree about anything.
| Direction | Statistician never used gsm |
Clinical data manager messy EDC delivery |
Engineer owns the codebase |
|---|---|---|---|
Inspect-first — og_read(), the intake report |
1st | 2nd | 2nd |
Convention — og scan / og doctor |
2nd | 3rd | 1st |
| Mapping bench — the dense worksheet | 4th | 1st | lower |
| Guided setup — the wizard | 3rd | 4th | lower |
The engineer's judgement named a first and a second and did not separate the remaining two, so those two cells say lower rather than inventing an order. On four-three-two-one points with the unseparated pair split, inspect-first scores 10 and convention 9 — close enough that the ranking is not the argument. What follows is.
The intake report is a document. og doctor is a
diagnostic. A document with no diagnostic behind it is true for exactly as
long as the folder does not change, which is about a month. A diagnostic with no document in
front of it prices its findings in cou0001 and kri0002 — a
vocabulary the statistician lens explicitly does not read and should never have to learn.
Ship one path: og_read() writes the document, the diagnostic is what keeps it
true, and the worksheet is where the document sends you when four questions is not enough.
toxgrg_nsv; the Grade 3+ lab rate. That is the whole translation layer between
gsm's vocabulary and theirs, and it means never having to learn gsm at all.
og_validate() already is — no new framework,
no interaction-test harness that does not exist, one new dependency.
Those three are satisfiable together. The first sets the unit of the report, the second sets the vocabulary of an answer, and the third sets what v1 is allowed to be.
og_read() opens the folder as
delivered — no renaming, no conversion, no input/Raw_*.csv ritual first. Today
og_validate() answers a six-file delivery with twelve lines of
file not found and names not one file the user actually sent.
Nothing on this page is a second mapping layer. gsm.mapping::ApplySpec() is the
engine, it stays the engine, and every answer any surface here collects is written as one of
the two spec keys it already reads. What is missing sits beside the engine, not underneath it.
source_col becomes the
SELECT … AS …. My column is called something else is
already a supported sentence.
type reaches RunQuery(bUseSchema = TRUE),
so casts happen in DuckDB rather than in study code.
Raw_* into
Mapped_* and carry the joins; Ingest() mapped a hand-built CRO
delivery perfectly in the companion session — 14 of 14 columns, 1,000 rows.
source_col cannot say. Prefix strips, value
recoding and derived grades are ordinary RunQuery steps in the same YAML.
The bench direction established this by building them, which is why nothing designed
here needs a change from Gilead-BioStats.
subjid, invid (not
siteid), studyid, grouping through strGroupCol.
Fixed, and not ours to move.
SELECT silently — measured below, and the reason a readiness screen
can be green over a study that produces no rows.
toxgrg_nsv is the Grade 3+ lab-abnormality KRI, or that one identifier
convention costs 16 of 42 displays.
Re-run against the installed gsm.mapping 1.1.3 while writing this page. Give
ApplySpec() a two-column frame and a three-entry spec whose third column was never
delivered. purrr::keep() removes that entry before the query is built, so the
guard that follows —
if (!all(sourceCols %in% names(dfSource))) LogMessage(level = "error", …)
— tests a vector from which the offending entries have already been deleted. It is
unreachable code.
spec entries subjid, invid, timeonstudy delivered columns SUBJECT, SITE columns returned subjid, invid timeonstudy present FALSE messages / warnings / conditions raised about it none
Every condition was captured with withCallingHandlers: the only messages emitted
are DuckDB connect, query complete and disconnect. This is not an argument against the
package — dropping unmatched entries is what makes partial specs work at all. It is the
argument for a surface that reports what the engine chose to ignore.
Thirteen platforms, read on 2026-08-27 from public documentation in the companion session and cited rather than re-run here. The gap analysis set the rule this inherits: seen on means documented, never verified. No demos, no trials, no vendor contact, no accounts.
The July gap analysis scored row D1 — a data-mapping surface — as present on five of the thirteen: safetyGraphics, JReview, Spotfire clinical, tidyCDISC and elluminate. Reading each platform's documentation capsule by capsule does not reproduce that list.
| July D1 said | The August reading |
|---|---|
| safetyGraphics | ✓ confirmed — a Mapping tab, per-field dropdowns auto-filled from a detected standard, exported back out as settings |
| elluminate | ✓ confirmed — the Mapper, a drag-and-drop transformation builder, described by the vendor and read as a claim |
| JReview | ● qualified — the mapping lives in the integrated data dictionary and is configured by an administrator, not by the end user |
| tidyCDISC | ✗ contradicted — no mapping surface of any kind. Conformance is the contract: a missing required variable rejects the upload outright |
| Spotfire clinical | — not established — nothing published at this grain. A statement about the documentation, not about the product |
| — not listed — | ✓ three more have one — Medidata Clinical Data Studio, Veeva CDB and Oracle Empirica Signal, all behind a configuring role |
The count survives — five platforms with a documented column-mapping surface, out of
thirteen — but three of the five names change. That matters here because the two the
July row got right are also the two whose shape this design borrows from, and the one
it contradicts, tidyCDISC, is the closest thing in the survey to og_validate()
today: conformance or nothing.
The honest summary of the whole survey: it establishes what vendors and maintainers chose to publish about their front doors, which flatters. It is strong enough to say what shape of surface exists in this market and where every one of them stops. It is not evidence that any of them works.
What the three runners-up contributed, and which lens demanded it. Nine come from a sibling direction; one is an addition none of the four made.
The intake report ends a finding with a question. og doctor ends one with a
patch: workflows/1_mappings/SUBJ.yaml:23, the three lines to replace, and the
metrics it costs if you do not. Grafted whole. A finding that says what is wrong and not
where to type is a report; a finding that says where to type is work, and work is what the
user came for.
Demanded by the statistician lens, which called og doctor the best-designed single piece in the session.
Answers write source_col: lines into the study's own
workflows/1_mappings/*.yaml, in the study's own git history, reviewed as a
diff. The writer edits the file as text and re-parses, refusing to save unless the parsed
structure matches what it intended — because open.gismo already ships the naive version and
it destroys the study's own explanations. See measurement M-c.
Demanded by the engineer lens, which found the defect at R/og_app_helpers.R before any of this was designed.
The intake direction detects broken joins and reports them. Convention refuses to run. Take the refusal. A wrong key does not crash — it flatters: every column is present, every type checks, the run completes, and sixteen of forty-two displays quietly report zero while the site risk score reweights over the ones that survived. Roughly sixty lines, and it is the cheapest item on any build list here.
Measured independently in this session on demo-301's own data — M-d and M-e.
Bound to a source column · Derived from a generated step · Constant for the whole study · Declined, with a reason and a printed price. The intake report's findings offered two buttons each; the bench's four dispositions generalise them, and the fourth is the one no other direction has.
Demanded by the data-manager lens: today her spreadsheet says N/A in a cell and that judgement dies there.
Decline Raw_LB.toxgrg_nsv and the product prints the two displays that stop,
takes the reason in the user's own words, and writes both into the YAML. This is the only
place in the whole session where a person can tell the product something it never thought
to ask.
Never a count of filled boxes. Today's og_validate() makes the opposite bargain
and gets it wrong in both directions at once — see M-b.
Every column the pipeline declares beside every column the study delivered, on one screen. The report is the front door and is where a first-time user should stay; the bench is where the fortieth transfer gets worked, and where somebody who knows their data can answer a question the four findings never raised.
Demanded by the data-manager lens, whose one killer objection to the intake report was that it hands you back to nothing once the four findings are answered.
That session built a name-blind value profiler, scored it 9 of 9 on a CRO
lab extract where the published name detector scored 3 of 9 — and then ran it on demo-301's
own Raw_LB.csv, where it picked visnum as the lab result and
toxgrg_nsv as the lower limit, scoring 1.000: higher than the
0.849 it scored on the answer it got right. There is no threshold that admits one and
rejects the other. So the number is never shown and the guess is never applied.
Does some categorical column give exactly one reference range per level? That test puts the correct result-and-limits triple first on both lab files and demotes the impostor to purity 0.00. It is the difference between an ordering worth reading and one worth ignoring.
Not "we fixed 765 identifiers" but "strip this constant prefix from this column in these
domains" — a rule a person can read, argue with, and reuse next month. Beside it,
config/setup-log.md: the user's own sentence about why, sitting next to the
machine-readable spec, for whoever opens the diff in December.
The bench deferred reuse explicitly and the data-manager lens called that the worst call in the session — month two of the same study on the same EDC costing what month one cost. The answers are already files in the study's git, so replay is not a new store: the second read applies what is there, re-measures every assumption it rests on, and reports only what changed. A new analyte appearing in the lab feed is a finding. Nothing else is.
Not designed by any of the four directions. It is the graft with the least evidence behind it and the first thing to challenge.
git checkout.
recode: key to gsm.mappingsource_col plus generated RunQuery steps already express value
recoding. Nothing here needs anything from Gilead-BioStats.
ApplySpec() is the engine and stays the engine. Everything designed here writes
its two keys and nothing else — see M-a.
og_validate() in its current formsource_col, so it fails projects the engine maps perfectly and passes projects
that produce zero rows. Both errors are in the report today.
This artifact synthesises four sessions that each measured a great deal; it re-ran six things
rather than trusting the summaries, because three of the grafts above depend on them.
Five are below; the sixth is M-f, in What gsm.mapping already
solves above. R 4.3,
gsm.mapping 1.1.3, open.gismo @ dev v0.2.0,
demo-301 @ main. Everything else on this page is cited to the session that
measured it.
gsm.mapping::ApplySpec, deparsed from the installed 1.1.3:
mapping$source <- spec$source_col %||% name mapping$type <- spec$type %||% NULL ... %>% purrr::keep(~ .x$source %in% colnames(dfSource))
Two keys, and a spec entry whose source column is absent is dropped from the generated
SELECT with no message at any verbosity level. All four directions write into
exactly this, which is why none of them needs a change upstream — and why a silent partial map
is the failure mode the whole design has to defend against.
open.gismo/R/og_validate.R:201 is
missing_cols <- setdiff(required, info$cols) — the target names the pipeline
wants, set against the column names in the file. It never reads source_col. A
correct spec that ApplySpec() maps perfectly still produces the full
missing-column error report; a spec whose join yields nothing passes clean.
R/og_app_helpers.R ends a settings write with
yaml::write_yaml(y, target). Round-tripping demo-301's own
workflows/1_mappings/LB.yaml through read_yaml, setting one
source_col, and writing it back:
before lines: 80 comment lines: 13
after lines: 70 comment lines: 0
CAST(toxgrg_nsv present before: TRUE after: TRUE
The thirteen destroyed lines are the study's note explaining that toxgrg_nsv must
be cast to character or the Grade 3+ KRI's
WHERE toxgrg_nsv IN ('3','4') matches nothing. The CAST step
survives, because it is a value in the document; the reason it exists does not, because it is
a comment. This is a defect open.gismo already owns, and it is the reason G2 specifies a text
editor rather than a serialiser.
Package the subject file the way a CRO does — USUBJID built as
studyid-subjid — while the lab file keeps the bare identifier:
Raw_SUBJ rows: 1005 enrolled (enrollyn=='Y'): 765 Raw_LB rows: 57200 subject file delivered as AA-AA-000-0000-S384 lab file delivered as S1000 overlap 0 of 765 overlap after stripping "AA-AA-000-0000-" 765 of 765 inner_join rows with the ADaM key 0 inner_join rows with the bare key 57,200 participants: 765
Both columns are present, both are character, both pass every type check open.gismo runs today. This independently reproduces the guided-setup session's M3b and the convention session's blocking finding, from a third direction.
All 42 consumers in demo-301/workflows/ — 30 metric and 12 module workflows —
resolved back through their Mapped_* dependencies to the raw domains that feed
them:
| Raw domain | Consumers | Which |
|---|---|---|
| Raw_SUBJ | 36 of 42 | Nearly everything. It is the join target, so it is also the single point of failure. |
| Raw_LB | 9 | cou0005 · kri0005 · saf0001 · hep_explorer · safety_delta_delta · safety_histogram · safety_outlier_explorer · safety_results_over_time · safety_shift_plot |
| Raw_AE | 7 | cou0001 · cou0002 · kri0001 · kri0002 · saf0003 · ae_explorer · ae_timelines |
| — none — | 1 | srs0001 is the only consumer that declares no spec: at all. It does not fail with the others; it stacks whatever kri* produced results and re-normalises over them. |
One wrong identifier convention across a delivery takes out 16 of 42 displays — and moves the site risk score's denominator rather than failing it. Nothing errors. The study looks exemplary.
og_read() plus the
diagnostic is v1 and is useful with no bench at all; the bench is v2. The honest risk is
that v1 ships with the report linking to a room that does not exist, which is exactly the
objection the data-manager lens raised in the first place.
srs0001 is a hole none of this closes. It declares no
spec, so no resolver can price it and no diagnostic can block on it. Declining a domain
moves its denominator instead of failing. The best available response is a printed warning,
which is what all four directions settled for, and it is not enough.
source_col: line is one line and an exercised feature. Prefix strips, value
recoding and derived grades have no declarative home in the spec, so they arrive as
generated RunQuery steps — code the data manager cannot fully audit and the
statistician did not write. G10's setup log records the human sentence beside it; it does
not make the SQL reviewable.
Competitor research is read-only and from public sources only — product documentation, package reference indexes, user guides and conference papers. No demos, no trials, no vendor contact. Seen on means documented, never verified.
Six questions this page cannot answer for itself. Each carries the answer the page argues for, so a nod is enough and only a disagreement needs writing.
| # | Question | The recommendation on this page |
|---|---|---|
| R1 | Two surfaces in v1, or report-plus-diagnostic first and the bench in v2? | Staircase. v1 is the read and the diagnostic; the bench is v2 and the report is designed to link into it. |
| R2 | Does a blocking finding refuse og_run(), or only warn? | Refuse. A key with zero overlap is not a preference. |
| R3 | Does answering write into the study's own mapping YAML, or into a file open.gismo owns? | The study's own, edited as text — with the writer fixed first (M-c). |
| R4 | Do answers replay across transfers, and is the scope per study or per sponsor? | Replay, per study. Per-sponsor reuse is the obvious next ask and the obvious leakage risk. |
| R5 | Which formats does the door accept on day one? | Add haven. It is already installed in this stack and never called, which is why a six-file delivery gets twelve file not found lines today. |
| R6 | Do lossy substitutions get carried onto the study site? | Yes, on the display itself. A chart built on severity standing in for toxicity grade should say so where somebody reads it. |