Recommendation · 30 October 2026
qcthat for safety.viz
- Build a thin adapter, not a JavaScript port. safety.viz hands its test results to the R package, and qcthat builds the same matrix it builds for gsm.safety.
- This already works. All 2,982 current safety.viz tests went through unmodified qcthat and came back as a matrix of milestone, issue and test.
- No existing JavaScript tool does this job, so the gap is real, but it is a small one. Most of the work is cleaning the links between tests and issues.
Decisions
30 October 2026, in the working session. @jwildfire: "Agree with the approach, but I want to hear a bit more about the adapter. If possible, I'd like to make this a formal qcthat extension that adds to the framework."
This settles the adapter over a JavaScript port. It reopens what the adapter should be, and where its code lives depends on that answer. The follow-up is qcthat evidence extensions.
Decisions for you
Adapter, or a JavaScript port of qcthat?
Recommendation: adapter. qcthat is about 8,000 lines of R covering GitHub fetching, reports and user acceptance. A port would rebuild that and then drift from the R version. The adapter only has to write one results file, and most of that code already exists in safety.viz.
What it costs: R has to run in one new CI job. Neither the main check nor the site build installs R today.
What counts as the requirement?
Recommendation: GitHub issues are the backbone, as you described, and the matrix rows stay as finer detail inside them. The chain becomes hub requirement, safety.viz issue, test, with row IDs such as HEP-CTRL-007 kept in the test names. There are 189 issues and 1,442 rows for what is already built, so turning rows into issues would bury the tracker.
What it forecloses: nothing. The evidence pages keep their row-by-row view.
Where does the code live?
Recommendation: in safety.viz first, then a separate qcthat-js repository when bio.viz adopts it. bio.viz has the same evidence pipeline, so it is the natural second user, and two users are what tell us the right shape. Both qcthat and qcthat-js are unclaimed on npm today.
The proof
I took the test results committed on safety.viz dev and passed them straight to qcthat's public CompileIssueTestMatrix(), alongside the repository's real issues. This is its report for release 1.11.2, with long test names shortened:
└─█─Milestone: v1.11.2 (4 issues, 6 tests)
├─[x]─Issue 339: Fix what the review of the v1.11.2 release candidate found
│ └─(no tests)
├─[x]─Issue 336: Prepare release 1.11.2
│ └─(no tests)
├─[x]─Issue 334: Hep Explorer: add the new R ratio (nR) to Point Size and the tooltip
│ ├─[v]─HEP-DISPLAY-008/HEP-CTRL-007/…: Point Size can scale points by the nR ratio…
│ ├─[v]─hep-explorer configure HEP-CTRL-007: Point Size offers Uniform, rRatio and…
│ ├─[v]─hep-explorer quadrant and legend polish (#54) HEP-CHART-004: pointTooltip…
│ ├─[v]─hep-explorer quadrant and legend polish (#54) HEP-CTRL-014: the nrRatio…
│ ├─[v]─hep-explorer structureData HEP-DISPLAY-008: buildPoints carries each…
│ └─[v]─hep-explorer structureData HEP-DISPLAY-008: computeNRRatio is the higher…
└─[x]─Issue 332: Release notes: promote v1.11.1 and open the v1.12.0 section
└─(no tests)
# Issue state: [o] = open, [x] = closed (completed), [-] = closed (won't fix)
# Test disposition: [v] = passed, [!] = failed, [S] = skipped
[v] All tests passed
[ ] 3 issues lack tests
- The full matrix has 3,208 rows, and it saved to JSON and read back intact with qcthat's own functions.
- The report already catches what a reviewer would ask about: three issues in that release have no test. Two are release chores, which a label can excuse. The third is the fix list from the release review, which a person should look at.
- It also shows the cross-repository link you need. qcthat reads each issue's parent, including parents in the hub: 110 safety.viz issues roll up to 39 hub requirements, and 32 of those requirements have tests.
How it would work
Run tests
Vitest and Playwright run exactly as they do now.Write results
safety.viz writes one file in CTRF, an open test report format.Build matrix
qcthat joins that file to the GitHub issues.Publish
The matrix goes on the evidence pages and on each release.
- Test names do not change. The
(#N)convention is already qcthat's own. - The matrix step needs its own workflow. The main check runs with a read-only token, and
tests/unit/ci-workflow.test.jsholds it that way. - The matrix stays out of the evidence freshness guard, because it depends on the state of GitHub and the guard compares only test results.
- qcthat's ready-made GitHub Action cannot be used directly, because it runs testthat. The first version is a short R script around the same functions.
- CTRF is a deliberate choice over a private format. If qcthat later learns to read it, every language with a CTRF reporter gets qcthat, with no JavaScript in the R package.
What has to be cleaned up
These came from running the matrix on the real data. The first is a bug in safety.viz today, whatever you decide about qcthat.
Tests whose issue numbers are missing or incomplete in evidence.json. The parser reads (#182) but not (#182, #212), so 157 of them show no issue at all. It is a one-line fix to match qcthat's rule. evidence-lib.mjs
Tests that name no issue, all of them unit tests: 290 participant profile, 102 Hep Explorer, 53 QT Explorer and 27 with no requirement ID either. For 202 of them, another test on the same requirement ID points to exactly one issue, so that link can be proposed automatically and reviewed. The rest need a person or an agent to look.
Closed issues with no test. 18 are release preparation or release review. Of the other 18, six already carry your orphan-accepted label, which qcthat can be told to ignore. That leaves 12 for you to rule on, for example the demo data migration (#37) and the check's workflow test (#316).
Tests that cite a pull request where they should cite an issue, across four pull requests. qcthat cannot place them in a milestone until they point at the issue behind each one.
Issues with no hub parent. Tests under them never reach a hub requirement. Some will be fine as standalone fixes, so this one is a report to read and not a target to hit.
Existing JavaScript tools
Nothing I found links GitHub issues, tests and milestones into a matrix for a JavaScript project. Downloads are npm's figures for the last seven days.
An open JSON format for test results, with reporters for many languages. It has no notion of requirements. Use it as the file between safety.viz and qcthat, written from results safety.viz already parses, because its Vitest reporter was last published in September 2025.
UseThe strongest general test report, and it can attach an issue link to a test. But the link is a call inside each test, and reporters have no direct access to requirements. It would add a second report without giving you the matrix.
SkipThe closest in spirit: it links Markdown requirements to Vitest and Playwright tests and measures coverage. It looks dormant, its README shows no GitHub issues support, and links run from the requirement to the test, the opposite of your convention.
SkipQase, Testomat.io, Xray
Reporters for hosted test management products. Xray keeps requirements in Jira. The record of what was required would move out of GitHub, which removes the point. I did not evaluate their GitHub integrations.
SkipOpenFastTrace, LOBSTER, reqstool, StrictDoc, Doorstop
Serious traceability tools, some built for safety standards. They keep requirements in their own files, and none is a JavaScript tool. Adopting one would replace qcthat on this half of the stack, so the R and JavaScript sides would stop matching. OpenFastTrace is GPL and LOBSTER is AGPL.
SkipRequirements written as Gherkin scenarios. That would mean rewriting 2,982 tests.
SkipPlaywright annotations, Vitest tags
Structured places to put an issue number. They are cleaner than parsing names, but evidence files are keyed on test names, qcthat parses names, and the convention is already in place. Keep this in mind only if names become a problem.
Not nowOne search of pharma validation sources found R tools only, and nothing for JavaScript. That was a single search, so read it as a lack of evidence and not as proof that nothing exists.
What this does not do
The matrix does not qualify a chart. Every chart keeps its current tier. The matrix shows that a test exists and passed for an issue, and it cannot say whether the chart is clinically right.
- A single passing test turns an issue green, so issue coverage is a weaker claim than row coverage. Today 1,215 of 1,442 rows have a test that names them. Keep that figure on the evidence pages beside the matrix.
- Screenshots stay on the evidence pages. qcthat has no place for them, so each test in the matrix should link to its evidence page.
- qcthat's
ExpectUserAccepts()user acceptance flow is tied to testthat. It is out of scope here.
Next steps
- You make the three decisions above. I then write the design into the existing hub requirement and give it a milestone. v1.12.0 is open and would suit. Hub requirement small
- Fix the issue-number parser so it follows qcthat's rule, with a test. small
- Clean the links: propose the 202 automatic ones, review the other 270, repoint the 24 pull request citations, label the chores and put the 12 rulings to you. medium
- Add a check that fails when a new test names no issue, as gsm.safety already does. small
- Write the results file, add the R script as its own workflow and attach the matrix to every release. medium
- Add a by-issue view to the evidence pages, with a roll-up to hub requirements. medium
- Repeat for bio.viz, then move the shared code into
qcthat-js. medium - Ask upstream for three things, none of them blocking. I will draft the issue and you post it, since qcthat sits outside your own account. They are a reader for CTRF, the "something like
QCRepo()" helper a maintainer has already floated for repositories that are not R packages, and nesting issues under their parents. small
How far to trust this
- The proof used the test results committed on safety.viz dev at
14c5847(18 October), not a fresh test run. Every record there passed, so failures and skips have not been exercised. safety.viz records a skipped test as a failure today, and the adapter will need to keep them apart. - It ran on qcthat 1.1.3, the version installed here. Installing the latest release, 1.2.0, was blocked in this session. I compared the two instead: eight R files changed, all in the user acceptance and GitHub Action code, and the matrix code did not change.
- The 202 automatic links are an estimate of what can be proposed. Each needs review, because a shared requirement ID does not prove that the same issue covers both tests.
- Row counts match IDs exactly, so a test naming a base ID does not count toward split rows such as
SH-FUNC-012A. The true row coverage may be somewhat higher. Another 77 IDs in tests match no row. - The full matrix file is 9.9 MB, and 1.7 MB without issue bodies. The site should load the smaller one.
- The step sizes are my judgement, not measurements.