Blog drafts for review — R/Pharma diary #5–#7

My last post laid out the plan for {safetyGraphics} v2 and ended on the obvious question: can we actually do this? Here’s the first big answer: safety.viz is live.¹

Intro to safety.viz

safety.viz is a charting library for monitoring clinical trial safety.

safety.viz is an updated JavaScript framework that modernizes² the {safetyGraphics} renderers. Seven of the ten original {safetyGraphics} renderers are now shipped! The site includes robust demos of all the charts — go play with them here.

As of now, 7 of the core {safetyGraphics} charts are live:

Point any of its seven interactive charts at your study data and review it in the browser: filter, group, zoom, and click through from a pattern on the screen to the participant records behind it. These are currently JavaScript only (usage instructions are here), but the {gsm.safety} R package should be done soon. Will share a follow-up post when it’s live.

Quality Framework

The most frequently asked question about {safetyGraphics} has always been: “Is it validated? Can I use it on actual studies?” The original answer was, “Not really. It’s exploratory”. safety.viz comes with a robust audit trail and extensive test evidence that is laying the foundation for GxP usage. Every chart traces to a reviewed requirement matrix from the original renderer’s documentation, with 249 unit tests and 94 browser tests keyed to requirement IDs and published as audit-style evidence reports.

Development Process

The next few posts will share lots more details about how I built this, but here’s a preview:

• v0.1.0 (Jul 11, 8 am) — The first prototype was the heavy lift — I spent multiple sessions across a few weeks updating safety-histogram. I spent time reviewing the original charts, including the helpful requirement documents we had saved in the wikis. Made lots of decisions about packages and test frameworks and then spent several sessions updating the website so that the example pages looked nice and clean and there was solid, transparent test evidence.
• v1.0.0 (Jul 11, 10:30 pm) — The structure from v0.1 unlocked agentic development in a meaningful way — v1.0.0 added 5 renderers in one session. The release came out later that same day.
• v1.1.0 (Jul 11, 11:50 pm) — The next release took about an hour. It updated all of the examples to use data from {pharmaverseadam}, added a paneled all-measures view to the histogram, and updated the README.
• v1.2.0 (Jul 12, 10 pm) — One day later, one of the most complex {safetyGraphics} renderers, hep-explorer, went live in safety.viz! AND it ported the incredibly robust clinical guide from PDF to HTML as part of the chart documentation.

The obvious question after a weekend like that is how³? The next post describes how I moved from the OpenClaw Obot experiment of diary #2 to Obot v3 — a Claude Code scaffold built around a public roadmap and an agent playbook.

Back in June, I closed the Obot post with a scorecard — 51% fun, 49% frustrating — and a promise: I was “strongly leaning towards moving on to experiment with other tools. More on that in a future post.”¹

This is that post. It took a month to write, and the reason it took a month is the story.

A quick note on sequencing: the last post promised an introduction to {gsm.safety} next. That post is still coming — this one jumped the queue.

The Dream

What I wanted from Obot was always simple to state:

Give the agent a goal, go do the day job, and come back to useful progress.

That’s the whole dream. Not a chatbot, not fancy autocomplete — a coworker that keeps moving while I’m in meetings.

The Reality

It didn’t happen. Work got busy, my attention drifted, and Obot did not magically keep advancing the project in the background. Every time I checked in, it was right where I’d left it: perfectly capable, patiently idle. The project moved exactly as fast as my attention did — which was the one thing the whole setup was supposed to fix.

The humbling part is that the agent was never the problem. Give it a bounded task — fix this bug, draft this doc, clean up these issues — and it did fine. Autonomy is different. Autonomy is a systems problem, not a model capability. For “come back to useful progress” to work, the system around the agent needs:

• durable memory, so context survives between sessions
• scoped permissions, so it can act without me hovering
• a task queue, so it always knows what’s next
• heartbeat/liveness checks, so silence means “working,” not “dead”
• status reporting, so I can see progress without interrogating it
• review gates, so nothing ships without a human
• recovery paths, so stuck work gets unstuck instead of quietly dying

None of that is glamorous. All of it, I now think, is the actual product. The boring stuff is the autonomy.

OBot v2

So I started building the boring stuff. OBot v2 was a much more structured framework: a PM agent to own issues, scope, prioritization, and handoffs; a Dev agent for implementation and PRs; a Testing agent for browser checks, visual validation, and regression evidence. OpenClaw heartbeats gave me liveness — a pulse I could check instead of wondering whether anything was actually happening. And a local control-plane layer, Paperclip, to tie the pieces together.

I spent a few days working the problem with GPT-5.5, and it genuinely felt like it was getting close. The roles made sense, the heartbeats beat, and I could squint and see the loop closing.

Then Fable 5 Happened

You can guess what happened next, because it’s what always happens in this field: the frontier moved. Just as my homegrown supervision stack felt close, Anthropic shipped Fable 5, and it suddenly seemed worth trying the whole problem inside Claude Code instead of on my own scaffolding. So in early July, the Obot of the June post was formally retired: the always-on OpenClaw runtime shut down, the obot-claw machine account archived — and no more heartbeats, because nothing needs to stay alive anymore. The ideas survived the retirement even though the implementation didn’t: the public roadmap repo became the memory and the task queue, sessions report status as they run, and every piece of work lands as a pull request behind a review gate. I didn’t abandon the supervision problem — I just stopped hand-building the plumbing and started configuring it.

Last Night

Which brings me to why I can finally finish this post.

On the night of July 11–12, I ran the first honest test of the dream. A little after midnight, I launched two overnight agent jobs, both deliberately ambitious stretch goals, with a monitor watching each one for stalls — a direct descendant of those OpenClaw heartbeats, checking job state on a timer and flagging anything silent for more than forty minutes. Then I went to sleep.

Both jobs landed clean in about two and a quarter hours — roughly 1.4 million tokens between them, zero human intervention — and the morning digest was waiting when I woke up:

• A full port of hep-explorer — {safetyGraphics}’s eDISH liver-safety chart, coordinated participant drill-down and all — to safety.viz, as draft PR #44: +8,487/−32 across 41 files, 307 unit tests and 15 end-to-end tests, CI green, with a live preview I could click around in the browser.
• A v1.0 plan for {open.gismo} with a working Phase-0 prototype, as draft PR #1: +7,243/−69 across 103 files, 640 passing R tests, plus a deployed design report laying out the plan and its open decisions.

The detail I care about most: both landed as draft pull requests. Neither agent merged anything, released anything, or declared victory — they published previews and a plan for me to read, but nothing shipped. The work rolled up to the review gate and stopped there, waiting for a human.

And the morning review had actual judgment in it. The eDISH port looked great — that one’s now the high-priority lane. The {open.gismo} plan was thorough, and I disagreed with one of its headline design decisions (roughly: how central GitHub should be to the platform). So that decision is now an open discussion on the roadmap instead of an assumption baked into merged code. That disagreement is the review gate doing its job: autonomy between the gates, judgment at them.

What I Learned

The hard part is not getting an agent to write code. The hard part is making the work observable, reviewable, and recoverable.

Or shorter: the bottleneck was never intelligence. The bottleneck was supervision.

Look back at the list in The Reality section. That night, every item on it was accounted for: the roadmap was the queue, memory carried context into the jobs, the monitor was the heartbeat, the digest was the status report, scoped permissions let the jobs build, test, and open PRs without waking me, and two draft PRs were the review gates. Recovery was the one item that went untested — the stall alarm was armed but never fired, because nothing stalled. The autonomy that finally worked isn’t an agent that never sleeps. It’s an agent I can hand a goal at bedtime — because the plumbing around it holds until morning.

It took two frameworks, three agent roles, a control plane, and a model release to get one good night of sleep. Worth it.

Next post: the anatomy of one of these multi-agent sessions from the inside — what ran, what broke, and the bill.

At 9:17 on the morning of July 11th, safety.viz — the Chart.js library that’s replacing the old {safetyGraphics} renderers — had one working chart. By 11:40, a release-candidate PR was open with six: histogram, four new lab renderers, and a stretch-goal adverse-event timeline that needed a whole new data domain. 247 unit tests and 92 browser tests, all green. I sent about six messages the entire time.¹

This series has promised an honest accounting of what agentic development actually costs, so this post is that: what ran, what broke, what I actually did, and the bill.

The Shape of the Morning

The plan was something I’ve started calling an “ultracode” session: instead of one agent grinding through a task list, an orchestrator runs multiple workstreams in parallel and I supervise from a distance.

Two workstreams launched together:

• A website agent — a single agent doing a full overhaul of the docs site: gallery, evidence pages, API reference.
• A scripted workflow — and this is the interesting part — a piece of deterministic orchestration code, not a model improvising a plan. The script defined the stages: fix two gating defects, then fan out five renderer builders in parallel, then integrate. Every step journaled to disk with a run ID.

That distinction matters more than I expected. When things went wrong (they did — see below), the scripted workflow resumed from its journal like nothing happened. The improvising agent did not.

Gates Before Fan-Out

The sequencing insight of the day: two fixes ran before the fan-out, because the five new renderers were all going to clone the histogram module’s pattern. Ship a defect in the template and you ship it five times.

Gate one was a histogram binning bug (#19) with a genuinely fun diagnosis. The new Chart.js histogram didn’t match the 2017-era original. To find out why, the agent cloned the original renderer, actually executed its d3-v3 source, and pinned it down: Webcharts used a quantile scale for binning, and a quantile scale over a two-point domain degenerates to uniform bins. Nine-year-old behavior, reproduced from primary sources rather than guessed at. The fix went up as its own small PR and was green in about an hour.

Gate two generalized the evidence pipeline — the machinery that produces the screenshots and requirement-coverage reports we treat as validation evidence — from “works for one module” to “works for N modules,” with CI stamping provenance on every evidence set.

Then the fan-out: five renderer builders in parallel git worktrees on parameterized ports, each doing red-green TDD against requirement matrices harvested from the original {safetyGraphics} charts. All five shipped, including the adverse-event-timelines stretch goal, which needed a newly vendored ADAE dataset. Between them: 139 new unit tests, 68 new browser tests, and — importantly — requirement rows that didn’t make the cut were descoped honestly and documented per module, not quietly dropped.

Picking Models Like Picking Staff

Mid-session I sent one piece of guidance: “pick the best model, don’t have to use fable for everything.” The orchestrator split the work accordingly — Claude Fable 5 for the judgment-heavy jobs (the two gates, the stretch renderer, integration, website design, conflict reconciliation) and Claude Opus 4.8 for the four lab-renderer builders, which were following an established template.

The mechanics of the reallocation were the striking part. The workflow script is a persisted artifact, so the orchestrator edited the model assignments in the script and resumed from the run ID. Completed steps replayed from cache; only new work ran on the new models. Editing a config file mid-flight and resuming is the most boring possible description of that, which is exactly the point — boring is what you want from orchestration.

What Broke

Honesty section. Four things went wrong, and they’re the most instructive part of the morning.

The session limit killed everything, 13 minutes in. I run this on a Claude subscription plan with rolling session limits, and at 9:30 the ceiling landed on both workstreams at once. The scripted workflow shrugged — journal on disk, resumed cleanly when I said “continue.” The website agent was another story. Its ~1,060 lines of uncommitted work on disk survived fine, and a successor agent triaged all of it and kept everything. But its design-review catalog — the analysis of what needed fixing across the site — lived only in the dead agent’s transcript and had to be redone from scratch. Lesson, now written in the project memory: transcript-only deliverables die with the session. Commit working notes.

The permission classifier refused to let the orchestrator merge. Even after I’d approved a PR, the agent’s attempt to merge it was blocked. Annoying in the moment, correct on reflection: this workspace has a hard “never merge without Jeremy’s explicit approval” rule, and the classifier can’t see into a conversation to verify approval happened. So the boundary holds mechanically and the human clicks merge. I’ll take a guardrail that’s occasionally overcautious over one that’s occasionally absent.

Large binary pushes died on HTTP/2 with a git error message that pointed everywhere except the actual problem. Not an agent failure — just ordinary infrastructure friction that agents hit exactly the way we do, and then have to debug through the same misleading error text.

All four Opus builders independently made the identical one-line framework fix. Same defect, same file, same fix, four times, in four isolated worktrees. Convergent evolution is a nice validation signal — four independent agents agreed on the diagnosis — but integration had to notice and dedupe it. Parallel agents don’t just divide work; they occasionally quadruplicate it.

Review Gates as Pull Requests

My other mid-session redirect: no giant release-candidate dump. Instead, three staged PRs, each with a live GitHub Pages preview I could open in a browser:

1. #23 — the binning fix. Small, reviewable, green in about an hour.
2. #22 — the docs-site overhaul. Approved the same morning; my entire review comment was “website looks great!”
3. #28 — the RC itself: five renderers plus framework changes, superseding the other two.

This is the “autonomy with review gates” thesis from the last post, made concrete. The agents ran autonomously between the gates; the gates were PRs with rendered previews, sized so a human could actually review them.

Integration deserves a line of its own. Five branches merged, with every shared-file conflict reconciled rather than force-resolved — including one where two agents had independently re-vendored the same CSV and the reconciler verified the files value-identical, row for row, before picking one. Final battery: 247 unit tests, 92 Playwright tests, 6 evidence sets with CI provenance, and an authorship audit showing all 46 commits correctly attributed to the obotclaw[bot] app identity.

What the Human Actually Did

My complete contribution, reconstructed from the transcript: the kickoff message, the model-allocation guidance, two “continue”s after the outage, the staged-PR redirect, and the website approval. Six short messages, maybe ten minutes of typing, plus the PR reviews themselves. The reviews were the real work, and they were the right work — that’s where the judgment belongs.

The Bill

The numbers, as promised.

Wall clock: kickoff 09:17 ET, RC PR open ~11:40 ET. About 2 hours 20 minutes, including the 15-minute session-limit outage.

Agents: 1 orchestrator plus 12 subagent spawns — 10 completed, 1 killed mid-work by the usage limit, 1 that died instantly on the same limit. The scripted workflow alone ran 8 agents through 918 tool calls over 98 minutes, reporting 1,973,684 subagent tokens. The successor website agent: 190,341 tokens across 105 tool calls in 32 minutes. The reconciliation agent: 72,178 tokens, 33 tool calls, six and a half minutes. The aborted first workflow run burned 136,069 tokens before dying. Total reported subagent tokens: about 2.37 million (the orchestrator’s own overhead isn’t in these counters).

Cost, two ways:

Plan-adjusted: I run this on a subscription plan, so the day’s marginal dollar cost was $0. The real cost was hitting the session ceiling 13 minutes in and spending 15 minutes waiting and recovering. On a subscription, the scarce resource isn’t dollars — it’s the rolling limit, and multi-agent fan-out eats it fast.

Raw API-equivalent (rough estimate): at current list prices — Fable 5 at $50 per million output tokens, Opus 4.8 at $25 — the ~2.37M reported tokens would be somewhere around $60–$120 in output tokens depending on how the Fable/Opus split shakes out (my best guess with the day’s allocation: roughly $90). Big caveat: these counters are output-weighted and don’t capture input or cache-read traffic, which on agentic work is substantial. A defensible order-of-magnitude for the full morning is low hundreds of dollars — call it $150–$400 — and I wouldn’t quote it tighter than that.²

For a morning that produced a six-renderer release candidate with 339 passing tests and full evidence trails, either framing seems like a bargain. For a morning that produced the wrong six renderers, it would be an expensive way to generate review work. The gates are what keep it the first thing.

What This Means for October

The keynote thesis this diary keeps circling: the interesting question isn’t whether agents can write the code — the morning confirmed, again, that they can — it’s whether the surrounding system makes the work observable, reviewable, and recoverable. The morning’s evidence, point by point: the scripted workflow was recoverable (journal, run ID, resume) where the improvising agent was not; the work was observable (evidence sets, provenance, authorship audit); and the review gates were real PRs a human could approve between coffee refills.

The failures all landed in the same place, too. Nothing broke because a model wasn’t smart enough. Things broke where state lived in the wrong place (a transcript instead of a file), where infrastructure hiccupped (HTTP/2), and where parallel workers lacked a shared view (four identical fixes). The bottleneck still isn’t intelligence. It’s plumbing.

Next up: getting these renderers wired into {gsm.safety} as htmlwidgets, so R users can actually call them.

Postscript (July 12): The release candidate landed, and v1.0.0 and v1.1.0 both shipped within a day. A seventh chart — a port of the hep-explorer eDISH display — is already in review as PR #44, from an overnight run.