What Can We Actually Do With AI Agents Right Now? Quite a lot. Are clinical trials ready to implement these tools? I’m not so sure…

What are these models good at?

Throughout this series, I’ve tried to stay focused on one core question: “What is AI good at right now?” I think the answer is: quite a lot. Obot, the agentic framework I created for this project — Claude Code under the hood — has made meaningful progress on every problem I’ve pointed it at. The biggest win was modernizing safetyGraphics and merging it into the OpenRBQM {gsm} framework via the new safety.viz JavaScript library and its gsm.safety R bindings. I’d wanted to do that work for years but put it off because I thought it would take a team about a year to refactor so much code. With the current AI tools, I got it done as a side project over the summer. safety.viz went from zero to thirteen chart renderers across six releases between mid-July and mid-August; two published clinical workflows became working prototypes in a single evening; and the whole summer ran on a $200-a-month plan.1 I’ve had many similar experiences at work in recent months: projects I would have expected to need a day or two now take an hour, and multi-week projects can be developed in a day.2

Using agentic AI as a cornerstone of clinical development is no longer about capabilities — it’s about execution. The models are capable. We’ve just got to figure out how to bring them into clinical development carefully and safely.

Closing the execution gap

AI best practices are evolving quickly. The Model Context Protocol launched in November 2024 and was donated to the Linux Foundation a year later. Agent Skills became an open standard in December 2025. Claude Code didn’t exist at the start of last year. GxP standards, on the other hand, are built on decades of precedent and evolve slowly by design. FDA’s AI-in-drug-development guidance, for example, has been a draft since January 2025. The capabilities are becoming obvious; the adoption pathway isn’t.

Fortunately, the meaningful progress on precompetitive frameworks for running clinical trials in recent years provides a viable path forward. Tools from this project like OpenRBQM and safetyGraphics sit alongside pharmaverse, the R Consortium submissions working group, the R Validation Hub, and the CDISC data standards foundation that has been in development for more than twenty years. Developers across these projects are already using coding agents to speed up development of the core frameworks and to build early versions of an agentic scaffold, including the skills needed to run them at scale.

The next step is to truly modernize the pipelines that sit behind these tools. While pharmaverse and CDISC form a strong portfolio-level foundation, standards around study-level implementation are much less mature. Moving from a mix of legacy scripted statistical programs and closed commercial systems to a foundation of configurable R and Python pipelines built on GxP-qualified packages is a good first step, but more is needed. For example, foundational practices such as version control and semantic versioning are still not broadly adopted, and they are arguably a prerequisite for using agentic AI at all. That said, the same agents are also the best change-management tool we have. I suspect that adoption of version control will be significantly less challenging when an agent can handle most of the day-to-day git commands using well-documented skills backed by a controlled MCP layer.

But adopting git is the easy part. In my opinion, the clinical trial community is not ready for the coming conversation about managing the volume of high-quality code that agents can now produce. So far, Obot has generated roughly 315,000 lines of source across 367 pull requests and 23 releases in 12 repositories — from one person and a few agents, working in spare time.3 I’ve tried to build GxP best practices into my workflows (more on that in the next entry), but I can’t say with confidence how long it would take to fully qualify this work using current standards. I can say that others using agents at scale are wrestling with the same problem — Addy Osmani and Milestone both describe review, not writing, as the bottleneck. If agentic coding is overwhelming modern SDLC pipelines, I worry about how it will impact an industry where unversioned scripts and double programming remain in active use.

All told, progress toward a robust software development life cycle for all code written in support of clinical development has been slow, and a unified technical framework — with robust APIs and MCP functionality across the many components involved in clinical development — is just beginning.4

What’s next?

I believe the upside for an AI-ready precompetitive framework for clinical trials is enormous. Anthropic CEO Dario Amodei’s Machines of Loving Grace argues that AI-enabled biology and medicine could “compress the progress that human biologists would have achieved over the next 50-100 years into 5-10 years.” Every discovery those tools produce still has to pass through a clinical trial. If discovery accelerates by even a fraction of what Amodei predicts, our current development pipelines have no chance of keeping up. A compressed century of biology is only possible with a compressed century of trials.

The good news is that my experience suggests the capabilities are real and already applicable to clinical trials. We’re just starting to understand how to deploy these tools in ways that are safe for clinical research. Even so, I believe that if the frontier models were frozen today, clinical trials would still be completely different in five years. Who knows what will be possible as the frontier continues to evolve.5

Next entry: the concrete version of this one — GxP best practices for agentic engineering: validation as code, generated documentation and traceability, human gates, and what I learned about monitoring agents the hard way. Then, October.

  1. The $200 is the monthly Claude Max subscription every session in this series ran on. The dollar figures quoted in earlier entries — roughly $7,000 across the twenty-four active days in the second interview — are what the same token usage would have cost at API list prices, not out-of-pocket spend. When the weekly allowance ran out in August, the work paused until it reset. ↩

  2. Unfortunately, hour-long meetings still take an hour. AI does take good notes, though… ↩

  3. Counted on 2026-09-06 over the repositories Obot has worked in since June 10: safety.viz, gsm.safety, open.csr, open.gismo, obot.agent, obot.roadmap, demo-301, scaffold, showlist, the keynote repository and the two safetyGraphics forks. Lines are net additions on each repository’s integration branch with generated outputs, rendered pages, data files and lockfiles excluded; counting everything the agents committed, including generated artifacts, the figure is about 4 million. Pull requests are those merged in the window (nearly all opened by Obot); releases are those published in it. ↩

  4. Well, at least it’s finally started, and is arguably accelerating pretty fast right now. About time! ↩

  5. AI collaboration note — I dictated the first draft of this post in a Claude voice session, working from an outline I had built with Claude, and edited it with Claude over several passes. Obot (Claude Code, using Fable 5.1) then closed the gaps I had left it: the canonical project links, the evidence link, the verification of the Amodei quote against the essay, the code and release figures (counted from the repositories, method in the footnote), the cost footnote, and this footer, which I edited. ↩

<
Previous Post
R/Pharma Diary #8 — A Second Interview with Obot
>
Blog Archive
Archive of all previous blog posts