Recommendation · 30 October 2026

qcthat for safety.viz

Decisions

30 October 2026, in the working session. @jwildfire: "Agree with the approach, but I want to hear a bit more about the adapter. If possible, I'd like to make this a formal qcthat extension that adds to the framework."

This settles the adapter over a JavaScript port. It reopens what the adapter should be, and where its code lives depends on that answer. The follow-up is qcthat evidence extensions.

2,982tests on the dev branch, all passing
84%name a GitHub issue qcthat can read
72%reach a hub requirement through that issue
0changes needed to qcthat for the first version

Decisions for you

Adapter, or a JavaScript port of qcthat?

Recommendation: adapter. qcthat is about 8,000 lines of R covering GitHub fetching, reports and user acceptance. A port would rebuild that and then drift from the R version. The adapter only has to write one results file, and most of that code already exists in safety.viz.

What it costs: R has to run in one new CI job. Neither the main check nor the site build installs R today.

What counts as the requirement?

Recommendation: GitHub issues are the backbone, as you described, and the matrix rows stay as finer detail inside them. The chain becomes hub requirement, safety.viz issue, test, with row IDs such as HEP-CTRL-007 kept in the test names. There are 189 issues and 1,442 rows for what is already built, so turning rows into issues would bury the tracker.

What it forecloses: nothing. The evidence pages keep their row-by-row view.

Where does the code live?

Recommendation: in safety.viz first, then a separate qcthat-js repository when bio.viz adopts it. bio.viz has the same evidence pipeline, so it is the natural second user, and two users are what tell us the right shape. Both qcthat and qcthat-js are unclaimed on npm today.

The proof

I took the test results committed on safety.viz dev and passed them straight to qcthat's public CompileIssueTestMatrix(), alongside the repository's real issues. This is its report for release 1.11.2, with long test names shortened:

└─█─Milestone: v1.11.2 (4 issues, 6 tests)
  ├─[x]─Issue 339: Fix what the review of the v1.11.2 release candidate found
  │ └─(no tests)
  ├─[x]─Issue 336: Prepare release 1.11.2
  │ └─(no tests)
  ├─[x]─Issue 334: Hep Explorer: add the new R ratio (nR) to Point Size and the tooltip
  │ ├─[v]─HEP-DISPLAY-008/HEP-CTRL-007/…: Point Size can scale points by the nR ratio…
  │ ├─[v]─hep-explorer configure HEP-CTRL-007: Point Size offers Uniform, rRatio and…
  │ ├─[v]─hep-explorer quadrant and legend polish (#54) HEP-CHART-004: pointTooltip…
  │ ├─[v]─hep-explorer quadrant and legend polish (#54) HEP-CTRL-014: the nrRatio…
  │ ├─[v]─hep-explorer structureData HEP-DISPLAY-008: buildPoints carries each…
  │ └─[v]─hep-explorer structureData HEP-DISPLAY-008: computeNRRatio is the higher…
  └─[x]─Issue 332: Release notes: promote v1.11.1 and open the v1.12.0 section
    └─(no tests)
# Issue state: [o] = open, [x] = closed (completed), [-] = closed (won't fix)
# Test disposition: [v] = passed, [!] = failed, [S] = skipped

[v] All tests passed
[ ] 3 issues lack tests

How it would work

  1. Run tests

    Vitest and Playwright run exactly as they do now.
  2. Write results

    safety.viz writes one file in CTRF, an open test report format.
  3. Build matrix

    qcthat joins that file to the GitHub issues.
  4. Publish

    The matrix goes on the evidence pages and on each release.

What has to be cleaned up

These came from running the matrix on the real data. The first is a bug in safety.viz today, whatever you decide about qcthat.

168

Tests whose issue numbers are missing or incomplete in evidence.json. The parser reads (#182) but not (#182, #212), so 157 of them show no issue at all. It is a one-line fix to match qcthat's rule. evidence-lib.mjs

472

Tests that name no issue, all of them unit tests: 290 participant profile, 102 Hep Explorer, 53 QT Explorer and 27 with no requirement ID either. For 202 of them, another test on the same requirement ID points to exactly one issue, so that link can be proposed automatically and reviewed. The rest need a person or an agent to look.

36

Closed issues with no test. 18 are release preparation or release review. Of the other 18, six already carry your orphan-accepted label, which qcthat can be told to ignore. That leaves 12 for you to rule on, for example the demo data migration (#37) and the check's workflow test (#316).

24

Tests that cite a pull request where they should cite an issue, across four pull requests. qcthat cannot place them in a milestone until they point at the issue behind each one.

79

Issues with no hub parent. Tests under them never reach a hub requirement. Some will be fine as standalone fixes, so this one is a report to read and not a target to hit.

Existing JavaScript tools

Nothing I found links GitHub issues, tests and milestones into a matrix for a JavaScript project. Downloads are npm's figures for the last seven days.

CTRF

MIT · reporter 413k/wk

An open JSON format for test results, with reporters for many languages. It has no notion of requirements. Use it as the file between safety.viz and qcthat, written from results safety.viz already parses, because its Vitest reporter was last published in September 2025.

Use

Allure Report

Apache-2.0 · 1.6M/wk

The strongest general test report, and it can attach an issue link to a test. But the link is a call inside each test, and reporters have no direct access to requirements. It would add a second report without giving you the matrix.

Skip

spec-trace

MIT · 5/wk · May 2025

The closest in spirit: it links Markdown requirements to Vitest and Playwright tests and measures coverage. It looks dormant, its README shows no GitHub issues support, and links run from the requirement to the test, the opposite of your convention.

Skip

Qase, Testomat.io, Xray

Reporters 86k–170k/wk

Reporters for hosted test management products. Xray keeps requirements in Jira. The record of what was required would move out of GitHub, which removes the point. I did not evaluate their GitHub integrations.

Skip

OpenFastTrace, LOBSTER, reqstool, StrictDoc, Doorstop

Java and Python

Serious traceability tools, some built for safety standards. They keep requirements in their own files, and none is a JavaScript tool. Adopting one would replace qcthat on this half of the stack, so the R and JavaScript sides would stop matching. OpenFastTrace is GPL and LOBSTER is AGPL.

Skip

Cucumber, playwright-bdd

MIT · 889k–2.1M/wk

Requirements written as Gherkin scenarios. That would mean rewriting 2,982 tests.

Skip

Playwright annotations, Vitest tags

Built in

Structured places to put an issue number. They are cleaner than parsing names, but evidence files are keyed on test names, qcthat parses names, and the convention is already in place. Keep this in mind only if names become a problem.

Not now

One search of pharma validation sources found R tools only, and nothing for JavaScript. That was a single search, so read it as a lack of evidence and not as proof that nothing exists.

What this does not do

The matrix does not qualify a chart. Every chart keeps its current tier. The matrix shows that a test exists and passed for an issue, and it cannot say whether the chart is clinically right.

Next steps

  1. You make the three decisions above. I then write the design into the existing hub requirement and give it a milestone. v1.12.0 is open and would suit. Hub requirement small
  2. Fix the issue-number parser so it follows qcthat's rule, with a test. small
  3. Clean the links: propose the 202 automatic ones, review the other 270, repoint the 24 pull request citations, label the chores and put the 12 rulings to you. medium
  4. Add a check that fails when a new test names no issue, as gsm.safety already does. small
  5. Write the results file, add the R script as its own workflow and attach the matrix to every release. medium
  6. Add a by-issue view to the evidence pages, with a roll-up to hub requirements. medium
  7. Repeat for bio.viz, then move the shared code into qcthat-js. medium
  8. Ask upstream for three things, none of them blocking. I will draft the issue and you post it, since qcthat sits outside your own account. They are a reader for CTRF, the "something like QCRepo()" helper a maintainer has already floated for repositories that are not R packages, and nesting issues under their parents. small

How far to trust this