the app goal · decision artifact · 2026-08-15
The app goal is the standing goal to build the user-facing safety-monitoring app that replaces safetyGraphics and brings the core gsm tools into the gsm.safety framework, on the open.gismo app-first arc. Reviewing the plan-rewrite artifact, you said the goal statement itself needs rework — you hold detail about what the app should be that is written down nowhere, you do not have all of it in your head, and you asked to be interviewed: "I basically want you to interview me to figure out what I want. I remember seeing a 'grill-me' skill somewhere… I think we did a helpful gap-analysis exercise for the CSR side quest…" (CSR being the clinical study report — the open.csr work.) This page is the research spike you asked for: what method actually extracts unspoken product intent from one expert, run by an agent over chat. The method is designed and already shipped as a skill; four calls in it are yours.
“sounds good. let’s try it.”
All four calls settled as recommended: he changed none of them, and this page’s own rule is that leaving the defaults standing is a complete answer. The shape: build things for you to correct first, then ask targeted questions. The pace: small rounds of at most four questions, one topic per 15–25-minute sitting. The record: a published folder on this site. The goal statement: the agent writes the ratified text into the goal itself once you approve it at the wrap. Because the shipped skill already carried exactly these as its defaults, nothing needed amending — “let’s try it” starts the interview rather than changing it.
Follow-through, filed 2026-08-16 under the app goal (milestone 2026q3, nothing run yet): #192 builds the things you correct — the capability matrix, the workflow walkthrough, three divergent drafts of the goal statement, and the mined candidate requirements — which costs you no time at all; #193 runs the sittings and ratifies the goal, and is the one that needs you.
Two questions are already queued for the first sitting: the two calls held open on the app plan rewrite — what ships in version 1.0 versus what is written down as deferred, and whether the demo study repo becomes the canonical template others fork — and the one you raised yourself the same evening, whether this program should keep using GitHub itself as its datastore.
The stakes: the app goal currently blocks its own work — nothing under it is ready to build until the goal says what the app is. Two calls on the plan-rewrite artifact are parked waiting on that same answer: what ships in version 1.0 versus what is written down as deferred, and whether the demo study repo becomes the canonical template that others fork. This page does not answer any of that — it decides how the answers get extracted from you. Every recommendation above was already the shipped default, and you left all four standing on 2026-08-15; the record is at the top of this page.
The protocol shipped as the grill-me skill in the agent repo, merged on the standard lane and wired into the workspace. It encodes the recommendations above as defaults; the four calls amend the skill, they do not gate it. To start: open a fresh obot2 session and type /grill-me 79 --prep — the prep phase costs you nothing; the first sitting is about 30 minutes whenever you want it. (Skill: obot.agent PR #96.)
The app goal's written Intent and Boundaries no longer describe what you actually want built; the missing detail is partly unspoken and partly not yet formed even for you. Because of that, two calls on last night's plan-rewrite artifact are held open rather than answered: what belongs in version 1.0 of the app and what gets written down as explicitly deferred, and whether the demo study repo is the canonical template people fork, formally attached to the app goal. Asking you to simply write the goal down is the one method known to fail — experts unintentionally omit up to about 70% of what matters when describing from a blank page, and a single open interview captures roughly a third of a domain — so this artifact answers how to run the extraction, not what the answers are.
References for a deep dive: the goal is hub issue #79; the two held-open calls are A3 and A4 on the app plan-rewrite artifact.
There is a real, widely-installed (~800k installs) Claude Code skill called grill-me, by Matt Pocock. It is a one-line wrapper over a grilling engine whose mechanics are excellent and worth adopting: model the subject as a tree of design decisions; ask only the questions whose prerequisites are already settled; attach the agent's recommended answer to every question so a round can be answered by number; and treat looking up facts as the agent's job, never yours — only genuine decisions reach the human. But it is deliberately stateless and single-session (its own docs call "~46 questions across four rounds" an ordinary session), which fails the shape of this goal outright. Pocock's resumable sibling skill, wayfinder, contributes the missing half: a persistent map of open decisions, one resolved per session, with an explicit "fog of war" section for questions you can tell are coming but cannot yet phrase — exactly the "you do not have all of it in your head" problem. The term also has local history: a file in our own agent repo, obot.agent/interviews/p004-grill-queue.md, is literally titled "P004 grill-me queue" and you answered its questions in May 2026. It ran on the Telegram-era stakeholder-interview skill, which is where the capture format our new skill forks comes from — verbatim answer, the operational decision it implies, and the follow-up artifacts it generates.
The exercise you remember is the change-request framework report we did for the clinical study report side quest — the open.csr work, 2026-07-27. Your memory likely blends it with the platform gap analysis of 07-25, the only artifact literally named "gap analysis." Importantly, it was not an interview. It was a pre-built, source-linked proposal — a landscape of 18 products, an eleven-step table of how the work is done today, five proposed increments — ending in five numbered questions, each carrying a draft position you could accept or overturn. Your cost was a three-sentence brief in, a ~30-minute read, and one decisive message out.
The single most important observation from it: the one unspoken requirement it surfaced arrived as an unsolicited objection to the agent's proposed sequencing, not as an answer to any question asked. That requirement was that each study needs its own GitHub repo, so change requests, reviews and sign-offs live with the study rather than with the engine that produces it — a load-bearing structural idea nobody had asked you about. The contrast case is just as sharp: every question that came with a draft position was answered the same day, while an open-ended gate in an old morning review checklist — "pick which of these twelve app proposals to pursue" — has still never been answered, three weeks on. Open-ended asks do not get answered; draft positions do.
Two holes in that precedent are worth fixing. Every one of its questions was about mechanism and none about boundaries or ownership — and both of the things it missed landed in exactly that territory. And approving all five draft positions in a single clause recorded no signal at all about which ones you actually hold an opinion on.
References for a deep dive: the CSR change-request framework report, the platform gap analysis, the study-level repo requirement (hub #131), and the unanswered twelve-proposal gate (hub #114, item D5).
| Finding | Design consequence |
|---|---|
| Experts omit up to ~70% of critical information in free description (Clark & Feldon, cognitive task analysis); one open interview captures ~27–48% of a domain (Aranda et al., 12 quasi-experiments); structured elicitation has effect size g≈0.87 (Tofel-Grehl & Feldon 2013) | Never run a blank-page "what do you want?" interview; plan multiple passes and let a fog-of-war ledger carry the tail |
| The instrument beats the interviewer: analyst experience shows no effect, structured protocols do (Agarwal & Tanniru; Davis et al. RE'06 systematic review — structured interviews beat card sorts, ranking, think-aloud) | Encode the protocol as a skill; a checklist-driven agent is enough, a clever one is not the point |
| Reaction to ONE concrete design is systematically inflated; criticism unlocks with a comparison set of 3 (Tohidi et al., CHI 2006); critique without a supplied lens is nearly worthless, premortem framing is the strongest lens (Veinott et al. 2010); role-played "devil's advocate" gets discounted (Nemeth 2001) | Show three genuinely divergent goal drafts, never one; frame critique passes (premortem, perspective reading); never announce a straw man as bait |
| LLM interviewers already perform near human level on coverage (LLMREI, RE 2025: 73.7% of planted requirements; mistake-guided question generation preferred over human analysts' 68% of the time) — but chat read-backs manufacture false agreement (81% user confirmation vs 72% coder agreement) and models flip under conversational pushback while judging correctly in evaluative framing; only a hard devil's-advocate ROLE produces real dissent (99.2% vs ~55% for "push back" prompt lines) | Separate interviewer / critic roles; quote-anchored read-backs asking "find what's wrong", never "is this right?"; challenges routed through a separate evaluative call |
| Loading the agent with context suppresses clarifying questions — models resolve ambiguity from priors instead of asking; closed/multiple-choice framing systematically truncates answers (Schuman & Presser) | Run the interviewer context-light (the context-rich agent preps and critiques); every menu is preceded by an open turn and carries a live "none of these" escape |
| No published study exists of an LLM interviewing a single expert repeatedly and asynchronously about a product they own | This deployment is an experiment, not applied known practice — instrument it (the ledger records everything) and expect to tune the protocol after round one |
Full citations and the per-technique fit table are in the spike record (README lists sources).
/grill-me)A three-layer hybrid, each layer earning its place from a different strand:
reports/goal-79-elicitation/: a running log, the answers as structured data, the open-question list including the fog-of-war tail, and a published index), committed after every batch of answers. One topic per 15–25-minute session, resumable any day. The wrap phase drafts the goal prose out of the decided entries, with traceability in both directions between question and issue, and closes the two held-open calls on the plan-rewrite artifact and its discussion thread.Honest time budget: first sitting ≈ 30 min; then 2–4 sessions of 15–25 min; wrap review ≈ 20 min. Total ≈ 1.5–2.5 hours of your attention across 4–6 short sittings. The research says one pass misses about a quarter even for a good LLM interviewer — the fog-of-war list is the design admitting that, not failing at it.
Each is already baked into the shipped skill as the default. Answering "defaults" in one line is a complete response; any pick amends the skill. The letter-number tags are the labels used in the discussion thread.
How the interview is shaped E1 · Recommended: (b) build things for you to correct first, then ask targeted questions
How many questions at a time E2 · Recommended: (b) small rounds, one topic per sitting
Where the written record lives E3 · Recommended: (a) a folder on this roadmap site
Who edits the goal statement at the end E4 · Recommended: (a) the agent applies it, on your approval
/grill-me 79 --prep; the prep artifacts get built and published; you start the first sitting whenever you have ~30 minutes. The wrap closes the two held-open calls from the plan-rewrite artifact — what is in version 1.0 versus written-down deferrals, and whether the demo study repo is the canonical fork template — and produces a ratified goal statement plus the requirements under it. That in turn unblocks executing the two calls you already accepted: retiring the stale v1.0 plan report, and standing up the surface-anchored requirement set.References for a deep dive: the held-open and accepted calls are A1–A4 on the app plan-rewrite artifact, recorded on discussion #149.