Data & config framework: workr pipelines, config in a study repo

The core finding

Architecture

Four layers, one direction of flow

Engine anatomy

Why workr is the right substrate

A workflow is three YAML keys. meta is identity plus arbitrary study-tunable values; spec declares the input domains and columns (validated before the run, and the hook that tells LoadData what to fetch); steps is an ordered list of {name, output, params}. The whole config-as-code story rests on one mechanism — RunStep() resolves each string param through five rules in order:

Rule 2 is the payoff: changing a KRI's threshold, a chart's grouping, or a flag scheme is a meta edit in the study repo — a reviewable one-line PR, no code. gsm.kri's kri0001.yaml carries Threshold: -2,-1,2,3, GroupLevel: Site, AccrualThreshold: 30 in meta; a chart workflow carries the entire safetyGraphics-shaped settings object the same way:

# gsm.safety/inst/workflow/3_reports/safety_histogram.yaml (abridged)
meta:
  Type: Report
  ID: safety_histogram
  domains:                    # logical slot -> pipeline domain
    labs: Mapped_LB
  widgetName: safetyHistogram
  widgetSettings:             # the chart's settings object, verbatim
    id_col: subjid
    measure_col: measure
    value_col: value
    group_by: sex
spec:
  Mapped_LB:
    subjid:  {type: character}
    measure: {type: character}
    value:   {type: numeric}
steps:
  - {output: InitializedWidget, name: list,
     params: {data: Mapped_LB, settings: widgetSettings}}
  - {output: strOutputDir, name: getwd}
  - {output: Report, name: gsm.safety::RenderSafetyChartsWidget,
     params: {lInitialized: InitializedWidget, strWidgetName: widgetName,
              strOutputDir: strOutputDir, strOutputFile: ID}}

Every chart workflow is this same three-step wrapper: collect {data, settings}, name an output dir, render. meta.domains is currently declarative only — the natural hook for the app's readiness panel (D-APP6) and domain wiring (below).

Domains × framework

A domain is a config entry, not code

The two launch domains from the design directions bind cleanly to existing workflow families — each domain names the phases it runs and the chart library that renders it. The registry lives in study config (D-APP7), which makes "add a domain" a study-repo PR:

# config/study-config.yaml (proposed)
study:
  id: DEMO-301
  label: "DEMO-301 — Safety Review"
domains:
  safety:
    label: Safety
    charts: safety.viz            # renderer bundle
    workflows: [1_mappings, 3_reports]     # gsm.mapping + gsm.safety
  rbqm:
    label: RBQM
    charts: gsm.viz               # widget bundle
    workflows: [1_mappings, 2_metrics, 3_reporting, 4_modules]
                                  # gsm.mapping + gsm.kri + gsm.reporting

Both domains share 1_mappings — mapping is study-level, domains are lenses over the mapped data. That's also why the readiness panel is per data domain (LB, AE, SUBJ), not per app Domain.

The study repo

Proposed layout — the K3 fork target

demo-301/
├── config/
│   ├── study-config.yaml        # identity + domain registry (proposed)
│   ├── data-config.yaml         # domain -> input location overrides
│   └── packages.yaml            # human-edited package list
├── manifest.csv                 # org,package,version,repository,url,sha  (generated)
├── rproject.toml                # rv-format, SHA-pinned                   (generated)
├── workflows/                   # materialized from packages at pinned SHAs
│   ├── 1_mappings/  2_metrics/  3_reporting/  3_reports/  4_modules/
├── input/                       # study data (synthetic for the demo)
├── .github/workflows/
│   ├── run-pipeline.yaml        # schedule + dispatch -> new ps-NNN
│   ├── create-snapshot.yaml
│   └── build-site.yaml          # publish branch root -> Pages
└── ps-NNN/… + snapshots.json    # on the publish branch: the app's whole backend

Everything above the .github/ line already has a producer: og_init() scaffolds the folder, workr's pkgManifest() writes the pins and pulls workflows/ from each package at the pinned SHA (the gsm.library pattern), and the three Actions exist in open.gismo today. The forkable-K3 increment is essentially joining the two halves that already work: run-study.yaml's install-and-run with build-site.yaml's validate-and-publish. hub#131 asks the same questions for the CSR study repo — name/layout, engine pinning, PR previews — and the answers should be shared: "set up a study repo" must be a command, not an artisanal clone; og_init() plus an Actions template is that command.

Ground truth

What's real, what's aspirational, what's inconsistent

Read from source this morning (workr v1.0.0 at ~/Documents/github/workr, both open.gismo lines, gsm.kri/gsm.safety workflow dirs, the four Actions). The framework holds; these are the seams a plan rewrite has to close rather than discover later:

Decisions

The D-FW ledger

Next steps