gsm.qtl report_qtl.yaml — upstream audit

Not seeing these in production use. Did you look at how the examples in gsm.qtl manage to navigate usage? @jwildfire, on the original report

Right on both counts. The examples do navigate around this — data-raw/Example_QTL.R builds the report’s inputs in hand-written R and never runs the workflow. That is precisely the problem: nothing in gsm.qtl runs this file, the R and the YAML have drifted, and the YAML is the copy that ships to studies.

For the gsm team

This audit was produced outside Gilead, by an AI agent working on the obot portfolio — an open-source safety-graphics effort that consumes the gsm pipeline. It came out of running gsm.qtl’s QTL report module in a demo study, and a first pass that got the diagnosis wrong; this page is the re-run.

Nothing has been filed against gsm.qtl. Each finding below is written as a draft issue — title, file and line, a repro that runs from a clean R session, the captured output, the blast radius and a suggested fix — so it can be posted, edited, or dismissed on its merits. Everything was executed against gsm.qtl’s own bundled inputs rather than the demo study’s data, and the one claim that did not survive is kept on the page with the reason it failed.

Corrections welcome, particularly on the two things this audit could not see from outside: whether any live study enables the QTL module, and whether real STUDCOMP extracts use "" or NA for participants who completed.

The ledger

Every claim, and what survived

Ranked by consequence: a wrong result outranks an error, an error outranks a misplaced file. Findings 2 and 3 both stop the run before finding 1 can be seen — which is why the wrong result has never been noticed.

#	Claim	Original verdict	Audit verdict	Consequence
1	Discontinuation listing contains the whole cohortline 55 · `compreas` fill	Real, but blamed on `!expr` never being evaluated	Real — different mechanism. The fill only tests `is.na()`; the data uses `""`.	Wrong result
2	Unqualified `pull` does not resolveline 87 · `strSnapshotDate`	Real, “caller-conditional”	Real, and neither shipped driver attaches dplyr	Halts the workflow
3	`invid` lost in the SUBJ/STUDCOMP joinlines 43–50	Real	Real, reproduced on the package’s own data	Halts the render
4	Output directory never createdlines 95–103	Downgraded to “hardening, not a defect”	Upgraded — the report is silently written to `tempdir()`	Wrong location
5	`names:` used instead of `name:`lines 62, 105	“Hardening, works as shipped”	Agreed. Partial matching resolves it; the test suite codifies the tolerance.	Latent only
—	`yaml` never evaluates the `!expr` tagthe original headline claim	Headline defect	False. `workr::RunStep()` re-parses and evaluates it.	Withdrawn

Findings

Draft issues, ready to file

Each repro runs from a clean R session with no external data — inputs are built from gsm.core::lSource through gsm.mapping. All console output below is captured verbatim.

FINDING 01 Wrong result — silent

QTL0002 reports completed participants as discontinuations

inst/workflow/4_modules/report_qtl.yaml · line 55

The compreas fill only tests is.na(). In every dataset in the ecosystem, a participant who completed the study has compreas = "" — not NA — so the fill never fires for them. The WHERE compreas != 'Completed/Ongoing' filter on the next step then keeps them, and everything downstream of qtl0002_num inherits the error: the site and country bar charts, all four reasons charts, and the Discontinuation Listing.

Listing shows78

Actually discontinued8

Same page’s headline metric14

The report contradicts itself: the Study Overview computes the metric independently from Mapped_STUDCOMP and is right; the listing beneath it is not.

The steps as shipped

  - output: qtl0002
    name: dplyr::mutate
    params:
      .data: qtl0002_preprocess
      'compreas': !expr rlang::expr(ifelse(is.na(.data[["compreas"]]), "Completed/Ongoing", .data[["compreas"]]))
  - output: qtl0002_num
    name: workr::RunQuery
    params:
      df: qtl0002
      strQuery: "SELECT * FROM df WHERE compreas != 'Completed/Ongoing'"

Reproducible example

library(gsm.core); library(gsm.mapping); library(gsm.qtl)

mappings_wf <- workr::MakeWorkflowList(
  strNames = c("SUBJ","ENROLL","IE","PD","STUDY","SITE","COUNTRY","EXCLUSION","STUDCOMP"),
  strPath = "workflow/1_mappings", strPackage = "gsm.mapping")
mapped <- workr::RunWorkflows(
  mappings_wf, gsm.mapping::Ingest(gsm.core::lSource, gsm.mapping::CombineSpecs(mappings_wf)))

# the module's join, then the module's fill, verbatim
pre <- dplyr::left_join(mapped$Mapped_SUBJ, mapped$Mapped_STUDCOMP, by = c("studyid","subjid"))
q2  <- dplyr::mutate(pre, compreas = ifelse(is.na(.data[["compreas"]]),
                                            "Completed/Ongoing", .data[["compreas"]]))

print(table(q2$compreas, useNA = "ifany"))
cat("rows the listing shows            :", sum(q2$compreas != "Completed/Ongoing"), "\n")
cat("rows that are real discontinuations:", sum(!q2$compreas %in% c("Completed/Ongoing","")), "\n")
cat("headline metric says discontinued  :", sum(pre$compyn == "N", na.rm = TRUE), "\n")

Captured output

                  Completed/Ongoing             Death Lost to Follow-Up
          70              682                 1                 2
 Withdrew Consent
                5

rows the listing shows            : 78
rows that are real discontinuations: 8
headline metric says discontinued  : 14

70 participants with a blank reason survive the filter. The 682 that are NA — participants with no STUDCOMP row at all — are the only ones the fill catches.

Same module, end to end, one line changed

# shipped:   ifelse(is.na(compreas), ...)
[INFO] Workflow Step 5 of 14: `workr::RunQuery`
[INFO] SQL Query complete: 78 rows returned.

# corrected: ifelse(is.na(compreas) | compreas == "", ...)
[INFO] Workflow Step 5 of 14: `workr::RunQuery`
[INFO] SQL Query complete: 8 rows returned.

The rendered consequence

Discontinuation Listing as shipped: rows for UK, Japan and US participants with the Discontinuation Reason column left blank — **As shipped.** 78 rows, most with no discontinuation reason at all — these are participants who completed the study.

Discontinuation Listing with the fill corrected: eight rows, each with a reason of Lost to Follow-Up, Withdrew Consent or Death — **Fill corrected.** 8 rows, every one a real discontinuation with a recorded reason.

QTL0002 Study Overview table: Participants Discontinued - Study 14, Total Enrolled 760, Current Premature Discontinuation Rate 1.8 percent, Deviation? No — The Study Overview from the same page as the 78-row listing. It reads `Participants Discontinued — Study: 14` and `Deviation? No`, while the site bar chart below it footnotes “Excludes 100 site(s) with no prematurely discontinued participants” where the correct figure is 140.

Why this is not one dataset’s quirk

Source	Rows	`compreas == ""`	`compreas` is `NA`
`gsm.core::lSource$Raw_STUDCOMP`	100	87	0
a `gsm.datasim`-generated study	765	621	0

The is.na() branch is dead code on both.

How it got here

69401ff (13 Mar 2026) changed the YAML fill value from "Ongoing" to "Completed/Ongoing".
c08bebf (13 Mar 2026), commit message “datasim and live studies don’t quite match”, added | compreas == "" to the same fill in data-raw/Example_QTL.R.

Same day, same author, same expression. The empty-string case landed in the R and not in the YAML.

Suggested fix

      'compreas': !expr rlang::expr(ifelse(is.na(.data[["compreas"]]) | .data[["compreas"]] == "", "Completed/Ongoing", .data[["compreas"]]))

Or as a query, avoiding !expr entirely — workr::RunQuery is already used by the surrounding steps:

  - output: qtl0002
    name: workr::RunQuery
    params:
      df: qtl0002_preprocess
      strQuery: "SELECT * REPLACE (COALESCE(NULLIF(compreas, ''), 'Completed/Ongoing') AS compreas) FROM df"

Either way, a test asserting nrow(qtl0002_num) == sum(Mapped_STUDCOMP$compreas != "") would have caught this, and would catch the next variant of it.

FINDING 02 Error — halts the workflow

Step 9 calls `pull` unqualified and fails unless the caller attached dplyr

inst/workflow/4_modules/report_qtl.yaml · line 87

workr:::GetStrFunctionIfNamespaced() resolves unqualified names through the search path, so this step runs only if the calling session happens to have library(dplyr) in effect. Neither of the two shipped ways of running module workflows does that. The module dies at step 9 of 15, before producing anything.

This is the only unqualified reference to a non-base function in any of the twelve module workflows in gsm.library’s snapshot. Every other unqualified name across those files is base R: paste0, list, getwd, max, file.path, normalizePath, system.file.

The step

  - output: strSnapshotDate
    name: pull
    params:
      .data: Reporting_Results_Longitudinal
      var: SnapshotDate

Reproducible example

The library() list is exactly what gsm.template’s run-snapshot.R attaches, minus gsm.endpoints, grail and gsm.template — none of which declare Depends: dplyr, so they cannot put pull on the search path either.

library(workr); library(gsm.mapping); library(gsm.kri); library(gsm.qtl); library(gsm.reporting)
cat("dplyr attached:", "package:dplyr" %in% search(), "\n")

# ... build `mapped` and `reporting` as in Finding 01, then:

workr::RunWorkflows(
  workr::MakeWorkflowList(strPath = "workflow/4_modules", strPackage = "gsm.qtl"),
  c(mapped, reporting, list(Reporting_Results_Longitudinal = reporting$Reporting_Results)))

Captured output

dplyr attached: FALSE

...
[INFO] Workflow Step 9 of 15: `pull`
[INFO] Evaluating 2 parameter(s) for `pull`
[INFO] .data = Reporting_Results_Longitudinal: Passing lData$Reporting_Results_Longitudinal.
[INFO] var = SnapshotDate: No matching data found. Passing 'SnapshotDate' as a string.
[INFO] Calling `pull`
Error in GetStrFunctionIfNamespaced(lStep$name) :
  Function 'pull' not found.
Calls: <Anonymous> ... RunWorkflow -> RunStep -> do.call -> GetStrFunctionIfNamespaced
Execution halted

Add library(dplyr) at the top and the same script gets past step 9. That is the whole difference.

Who runs it without dplyr

gsm.template — inst/file-structure/scripts/run-snapshot.R. The canonical study runner attaches workr, gsm.mapping, gsm.kri, gsm.endpoints, gsm.qtl, gsm.reporting, grail and gsm.template, then runs every workflow found in ./workflow/4_modules. None of those eight declares Depends: dplyr.
open.gismo::og_run(). Its .og_attach_pipeline_packages() attaches exactly gsm.core, gsm.mapping, gsm.kri and gsm.reporting. Verified: with that set and nothing else, workr:::GetStrFunctionIfNamespaced("pull") throws.

module_outputs <- './workflow/4_modules' %>%
    workr::MakeWorkflowList(strPath = ., strPackage = NULL) %>%
    workr::RunWorkflows(lConfig = lConfig)

If this is not being seen in production today, the likeliest explanation is that no live study has the QTL module enabled in its 4_modules — not that the step resolves. It does not resolve under either shipped driver.

Suggested fix

    name: dplyr::pull

FINDING 03 Error — halts the render

QTL0002 render fails with “object `invid` not found”

inst/workflow/4_modules/report_qtl.yaml · lines 43–50

Mapped_SUBJ and Mapped_STUDCOMP both carry invid. The module joins them on studyid and subjid only, so dplyr disambiguates and the result has invid.x and invid.y — no bare invid. Six chunks of inst/report/QTL0002.Rmd reference it, and the render dies on the first.

data-raw/Example_QTL.R does not hit this: it joins select(Mapped_SUBJ, subjid, country) — three columns, no invid — to Mapped_STUDCOMP, so invid arrives unambiguously from STUDCOMP.

The step

  - output: qtl0002_preprocess
    name: dplyr::left_join
    params:
      'x': Mapped_SUBJ
      'y': Mapped_STUDCOMP
      'by':
        - 'studyid'
        - 'subjid'          # invid missing

Reproducible example

# ... build `mapped` as in Finding 01, then:
pre <- dplyr::left_join(mapped$Mapped_SUBJ, mapped$Mapped_STUDCOMP, by = c("studyid","subjid"))
print(names(pre))
cat("\nbare `invid` present?", "invid" %in% names(pre), "\n")
cat("`invid` is in both inputs:",
    "invid" %in% names(mapped$Mapped_SUBJ), "/", "invid" %in% names(mapped$Mapped_STUDCOMP), "\n")

Captured output

 [1] "studyid"              "invid.x"              "country"
 [4] "subjid"               "subject_nsv"          "enrollyn"
 [7] "timeonstudy"          "firstparticipantdate" "firstdosedate"
[10] "timeontreatment"      "agerep"               "sex"
[13] "race"                 "mincreated_dts.x"     "invid.y"
[16] "compyn"               "compreas"             "mincreated_dts.y"

bare `invid` present? FALSE
`invid` is in both inputs: TRUE / TRUE

Running the module through to the render

With pull qualified so it gets that far:

[INFO] Workflow Step 15 of 15: `gsm.kri::RenderRmd`
processing file: Report_QTL.Rmd
...
4/10 [QTL0001 Eligibility]
5/10
6/10 [QTL0002 Premature Study Discontinuation]

Quitting from Report_QTL.Rmd:57-78 [QTL0002 Premature Study Discontinuation]
Error in pull(., !!enexpr(varGroupID)) :
Caused by error:
! object 'invid' not found

Blast radius

The whole QTL0002 section fails and the render aborts, so no report is produced at all. QTL0001 is unaffected — qtl0001 comes straight from Mapped_EXCLUSION, which keeps its own invid. The collision is a property of the mapped specs, not of any one dataset: gsm.mapping’s SUBJ and STUDCOMP mappings both emit invid, so this reproduces on any study.

A note on the spec block

The module’s own spec declares only studyid, subjid and country for Mapped_SUBJ, which reads like the author intended the narrow projection the working R uses. workr::RunWorkflow() calls CheckSpec() for validation only — it does not subset — so the full frame reaches the join.

Suggested fix

invid is a key, not a duplicate: a participant belongs to one site, and both domains agree on it.

      'by':
        - 'studyid'
        - 'subjid'
        - 'invid'

Alternatively, project Mapped_SUBJ down to studyid/subjid/country before the join — what the working R does, and what the spec block appears to promise.

FINDING 04 Wrong output location

The report is written to `tempdir()`, not to `outputs/{SnapshotDate}`

inst/workflow/4_modules/report_qtl.yaml · lines 95–103

The module builds outputs/{SnapshotDate} and normalizes it, but never creates it. normalizePath() defaults to mustWork = NA, so a missing directory is a warning rather than an error and the un-normalized relative path is passed on. gsm.kri::RenderRmd() then finds it unwritable and silently redirects the report to a session temp directory, which is discarded when R exits.

No error, no failed run — just no report where the pipeline says reports go. The earlier pass filed this as “hardening, not a defect”. Running it shows otherwise.

Captured output

[INFO] Workflow Step 12 of 15: `normalizePath`
[INFO] Calling `normalizePath`
[INFO] character of length 1 saved as `lData$strOutputDir`.
...
[INFO] Workflow Step 15 of 15: `gsm.kri::RenderRmd`
[INFO] Calling `gsm.kri::RenderRmd`
You do not have permission to write to outputs/2012-03-29. Report will be saved
to /var/folders/_9/_l3b4x016gjbd376c37kpp580000gn/T//RtmplwIX8B
...
In addition: Warning message:
In (function (path, winslash = "\\", mustWork = NA)  :
  path[1]="outputs/2012-03-29": No such file or directory

Blast radius

Every first run in a fresh working directory. The message is not an error and does not stop the pipeline, so an automated snapshot run reports success with the report written somewhere that is deleted on exit. The path itself is correct — gsm.template’s outputs/README.md documents exactly this convention. Only the creation is missing. report_qtl.yaml is the only module in gsm.library’s 4_modules that constructs a nested output path; the other two that set strOutputDir use getwd(), which always exists.

Suggested fix

  - output: strOutputDir
    name: file.path
    params:
      outputs: outputs
      SnapshotDate: strSnapshotDate
  - output: created
    name: dir.create
    params:
      path: strOutputDir
      recursive: true
      showWarnings: false
  - output: strOutputDir
    name: normalizePath
    params:
      path: strOutputDir

Correction

The claim that does not hold

Withdrawn

“`yaml` never evaluates the `!expr` tag, so every row gets the literal expression string”

This was the original report’s headline defect, and it is false.

yaml::read_yaml() does return the tag as a literal string — that part is right. But workr::RunStep() has a parse_expr_param() branch that matches ^(rlang::)?exprs?\( and evaluates the string before the call. The mutate runs correctly and compreas is never polluted.

Verified in isolation

[INFO] Evaluating 2 parameter(s) for `dplyr::mutate`
[INFO] .data = qtl0002_preprocess: Passing lData$qtl0002_preprocess.
[INFO] compreas = rlang::expr(ifelse(...)): Parsed expression parameter.
[INFO] Calling `dplyr::mutate`

  subjid          compreas
1      A Completed/Ongoing   <- was NA, correctly filled
2      B             Death
3      C                     <- was "", the actual defect (Finding 01)

The earlier session’s own run log records Parsed expression parameter. on that step and the conclusion was drawn anyway. The consequence it observed — the discontinuation listing containing the whole cohort — is real; the mechanism is Finding 01.

Root cause

Why four defects accumulated in one file

Worth filing on its own, as the parent of the four above.

FINDING 05 Coverage gap

The module is shipped to studies but is not run by any test, example or vignette

inst/workflow/4_modules/report_qtl.yaml

Nothing in the repo runs it

$ grep -rn "report_qtl" . --exclude-dir=.git
./pkgdown/menus/examples/Example_QTL.Rmd:52:report_qtl <- knitr::knit_child(
./pkgdown/menus/examples/Example_QTL.Rmd:57:cat(report_qtl, sep = "\n")
./inst/workflow/4_modules/report_qtl.yaml:3:  ID: report_qtl

The two pkgdown hits are a local variable holding knitted child output, unrelated to the workflow. tests/testthat/ has no reference — the qualification tests qual_T1_1, qual_T2_1, qual_T3_1 cover the 2_metrics workflows and the analysis functions. vignettes/IntroQTL.Rmd does not run it.

It ships anyway

gsm.library’s snapshot-main and snapshot-dev branches both carry workflows/4_modules/report_qtl.yaml, byte-identical to the copy in this repo apart from a trailing newline. Snapshot last refreshed 24 July 2026. The module declares Active: true, so MakeWorkflowList() selects it, and gsm.template’s runner executes everything it finds.

The drift, concretely

	`data-raw/Example_QTL.R` — works	`report_qtl.yaml` — ships
SUBJ→STUDCOMP join	`select(Mapped_SUBJ, subjid, country)`, `by = "subjid"`	full frame, `by = c("studyid","subjid")`
blank-reason fill	`is.na(compreas) \| compreas == ""`	`is.na(compreas)` only
`pull`	`library(dplyr)` at the top	unqualified, no attach guarantee
output directory	`getwd()`	`outputs/{SnapshotDate}`, never created

Also here, latent

Lines 62 and 105 use names: where every other step uses name:. They run today only because lStep$name partial-matches the names key through R’s $ semantics, and tests/testthat/test-workr-migration.R codifies the tolerance with step_keys <- c("name", "names"). Not a defect now; it breaks the day workr reads the field with [[. Worth normalizing while the file is open.

Suggested fix — pick one

Cover it. A smoke test that runs the module on gsm.core::lSource mapped through gsm.mapping and asserts the report renders and nrow(qtl0002_num) matches the participants with a recorded reason. Roughly twenty lines, and it fails today on all four defects. Then have data-raw/Example_QTL.R call the workflow instead of rebuilding the listings in parallel, so there is one definition of the report inputs.
Or retire it. Remove it from inst/workflow/4_modules/ so it stops being aggregated into gsm.library snapshots, and document RenderRmd() as the supported entry point.

Shipping it uncovered is the option that is not working.

Limits

What this audit could not establish

Whether any live study enables the QTL module. Not visible from outside Gilead. What is established: the file ships byte-identical through gsm.library, gsm.template’s runner executes every module it finds, and the module is marked Active: true.
Whether a study’s own driver attaches dplyr. Neither shipped driver does. A study that has written its own runner may.
Whether real (non-synthetic) STUDCOMP extracts use "" or NA for completers. Both datasets available here use "". The strongest indirect evidence is the author’s own commit message — “datasim and live studies don’t quite match” — on the commit that added the empty-string case to the R.

Handoff

Where everything lives

Committed on branch gsm-qtl-audit in jwildfire/obot.agent, under drafts/gsm.qtl/. Nothing has been posted — agents have no write access to Gilead-BioStats.

README.mdIndex, the withdrawn claim, and the evidence map

ISSUE_N_qtl0002-blank.mdFinding 01 — ready to paste into a new issue

ISSUE_N_pull-unqual.mdFinding 02

ISSUE_N_invid-join.mdFinding 03

ISSUE_N_outputdir.mdFinding 04

ISSUE_N_yaml-untested.mdFinding 05 — the root cause, parent of the other four

assets/setup_block.RShared setup used by every repro

assets/repro_*.ROne-file repros: pull, invid, compreas

assets/runA.logShipped module, no dplyr — dies at step 9

assets/runB.logShipped module, dplyr attached — reaches the render, dies on invid

assets/runC.logTwo renders: shipped fill (78 rows) vs corrected fill (8 rows)

assets/prod_path.logSame failure under gsm.template’s run-snapshot.R package set

assets/*.jpgRendered-report screenshots

Every claim, and what survived

Draft issues, ready to file

QTL0002 reports completed participants as discontinuations

The steps as shipped

Reproducible example

Captured output

Same module, end to end, one line changed

The rendered consequence

Why this is not one dataset’s quirk

How it got here

Suggested fix

Step 9 calls pull unqualified and fails unless the caller attached dplyr

The step

Reproducible example

Captured output

Who runs it without dplyr

Suggested fix

QTL0002 render fails with “object invid not found”

The step

Reproducible example

Captured output

Running the module through to the render

Blast radius

A note on the spec block

Suggested fix

The report is written to tempdir(), not to outputs/{SnapshotDate}

Captured output

Blast radius

Suggested fix

The claim that does not hold

“yaml never evaluates the !expr tag, so every row gets the literal expression string”

Verified in isolation

Why four defects accumulated in one file

The module is shipped to studies but is not run by any test, example or vignette

Nothing in the repo runs it

It ships anyway

The drift, concretely

Also here, latent

Suggested fix — pick one

What this audit could not establish

Where everything lives

Step 9 calls `pull` unqualified and fails unless the caller attached dplyr

QTL0002 render fails with “object `invid` not found”

The report is written to `tempdir()`, not to `outputs/{SnapshotDate}`

“`yaml` never evaluates the `!expr` tag, so every row gets the literal expression string”