Prompt Coach — How It Adapts To You

A static rule catalog would nag forever. This page covers the parts that make the coach fade — how it learns which rules are worth firing for you, how mastery is earned rather than assumed, and why mastery is not permanent.

Start at the overview for what the coach is, or the rule catalog for what each rule catches.

Encouragement layer

The coach also praises the specific positive behaviors that mirror the negative rules — 42 positive detectors, one per rule (v0.40 filled the last 13 gaps). Praise is deliberately sparing (default 1 per 10 clean prompts with a positive, plus milestone events on rule mastery and first-after-fire), grounded in behavioral-science literature (Brophy 1981, Mueller & Dweck 1998, Fogg 2019, Deci & Ryan 2000, Kohn 1993). Beyond praise, these detectors now do double duty: each positive fire is the demonstration that drives earned mastery (see Earned mastery (v0.40+)).

Design constraints:

  • Never on a coached prompt. Praise + correction on the same prompt dilutes both.

  • Specific, not generic. Describes what you did, not who you are.

  • Process, not trait. Praises the technique.

  • Variable-ratio. Intermittent praise stays potent.

  • Milestone events. Rule graduating to mastered (🎓) + first-after-fire always recognized.

Distinct from praise, the ✓ clean-prompt acknowledgment (v0.35) is a frequent, informational liveness heartbeat — a green-dot signal that the coach ran, plus your progress toward the next mastery. Grounded in the informational-vs-controlling-feedback distinction (Deci & Ryan 2000) and the endowed-progress effect (Nunes & Drèze 2006).

Typo tolerance

A pre-pass normalizes each prompt against a curated set of trigger words drawn from the rule catalog. Dyslexic-friendly, transposition-friendly, deterministic (banded Levenshtein, ~1 ms). Corrections happen only when the closest trigger word is uniquely the closest (ties = no correction, so text doesn’t become test), and adaptive tolerance is stricter on short tokens.

Examples: refacotr → refactor, evrything → everything, veryfy → verify, cleanre → cleaner.

Set typo_tolerance: 0 to disable.

Conversational short-circuit

Many prompts in an ongoing conversation aren’t full asks — they’re approvals (sure, publish, go), multi-choice picks (1 and 2, option a), or continuations (continue, thanks). The coach detects these and skips analysis entirely — no rule matching, no praise, no streak updates. It also reads the session transcript to skip picker answers (replies to an AskUserQuestion menu) so a multiple-choice pick never triggers a false coaching pass.

Adaptive coaching (v0.41+)

The coach learns which rules are worth firing for you by closing the loop on every rewrite.

Acceptance loop

The collaborator rewrite is only as good as whether you take it — acceptance rate is the north-star metric for inline AI suggestions (it’s GitHub Copilot’s core adoption metric). On the turn after a rewrite, the coach reads your reply and records a per-rule outcome:

Reply Recorded as

yes / ok / proceed …

accepted

edit <thing>

edited — a positive signal (the coaching landed, the specifics didn’t), distinct from a rejection

no / original …

rejected

a fresh, unrelated prompt

nothing (we don’t guess at implicit rejections — the ledger stays high-precision)

From these it computes a per-rule acceptance rate = (accepted + edited) / outcomes.

Two refinements make that rate trustworthy (v0.42):

  • Attribution — when several rules fire on one prompt, the reply credits only the primary (highest-priority) rule, not all of them, so a single no doesn’t wrongly penalize every candidate.

  • Blind-reject filter — a rejection that arrives too fast to have read the rewrite (estimated from the block’s length vs the reply’s timing) is bucketed as blind_reject and excluded from the rate — it’s a reflex, not a considered rejection (arXiv 2601.21379).

See the ledger any time with /prompt-coach:stats or /prompt-coach:config acceptance (--json for machine-readable): overall rate + a per-rule table, with low-acceptance rules flagged ⚠ dormant-risk.

Precision-gated activation + fatigue cap

That acceptance rate then decides which rules fire:

  • Precision gate — a rule whose acceptance rate falls below precision_floor (default 0.15) over at least min_outcomes_for_gating outcomes is demoted to dormant and stops firing. This generalizes the mastery inactive status to dismissed rules — the path static-analysis research endorses over hard-disabling (which drives the alert-fatigue that makes developers ignore all warnings).

  • Explore slot — every explore_period prompts (default 10) one dormant rule is re-admitted to refresh its estimate, so a rule that was noisy in only one context isn’t buried forever (a deterministic exploit/explore split).

  • Fatigue cap — at most max_nudges_per_window visible rewrites per rolling window of nudge_window prompts (default 6 per 20); over the cap, fires are still logged and bookkept but the rewrite isn’t rendered. Firing too often makes the signal non-stationary — you tune it out.

Earned mastery (v0.40+)

Mastery is driven by demonstrations — the times you actively used the good technique, not by the mere absence of the mistake. This fixed a real flaw: before v0.40 a rule graduated after a run of clean prompts, but "clean" only meant the anti-pattern was absent — and most prompts don’t exercise most rules, so a rule could master on prompts that never had anything to do with it.

Each of the 42 rules now has a mirroring positive detector (e.g. improve-without-metric ↔ stated-metric, incremental-routing ↔ batched-routing). When a positive fires, that’s a demonstration:

Signal Role in mastery

demonstrations

Drives mastery — increments each time the mirroring positive fires (you used the technique)

clean_streak

Demoted to a recency/no-regression guard, no longer the mastery driver

fires_total

Times you tripped the rule (a regression signal, not evidence of skill)

A rule graduates to mastered once demonstrations ≥ min_demonstrations (default 3) and it hasn’t relapsed in the last regression_guard prompts (default 3). A rule you neither demonstrate nor trip over a long window retires inactive ("N/A to how you work") instead of claiming a mastery you never earned.

Config knobs (rule-activation category): min_demonstrations, regression_guard, inactive_after. The legacy min_fires_for_mastery gate is retained for forward-compat but ignored.

Migration: existing masteries are grandfathered — kept as-is and tagged mastery_basis: legacy (vs demonstrated for newly-earned ones), so nothing is wiped. To make a rule re-earn mastery honestly under the new model, run mastery-reset <rule>.

Mastered rules aren’t permanently dormant. They still evaluate every prompt, and when only a mastered rule matches, a refresher fires — a lighter re-fire rather than the full coaching block. The cooldown for mastered rules is much longer (50 prompts vs 5 for practicing rules).

Optional self-healing: demote_on_regression auto-demotes a mastered rule that fires threshold+ times within a rolling window. Off by default.

Decaying mastery (v0.41+)

Prompting is an accuracy-based cognitive skill, and those decay with non-use (Arthur et al. 1998; Psychological Bulletin 2024 finds ~half of an accuracy skill lost by ~6.5 months of non-use). So mastery is not terminal. A mastered rule carries a review clock on an expanding schedule (review_intervals_days, default [30, 90, 180] days). Every time you naturally use the technique, that demonstration is spaced retrieval — it resets and advances the clock. If the clock lapses with no natural use, the rule decays to a watch tier and re-enters active coaching; it must be freshly re-demonstrated to re-graduate (a retrieval-practice loop). Tripping the rule while in watch is a genuine regression and drops it back to practicing to re-earn from scratch. Grounded in the spacing effect + expanding intervals (Nature Reviews Psychology 2022).

Reporting bad calls

If the coach mistreated a prompt, mark it on the very next turn with any of these phrases:

  • "coach that was wrong"

  • "coach missed this"

  • "coach false positive"

  • "bad nudge" / "wrong nudge"

The analyzer flags the prior substantive prompt’s analysis (not the current complaint) into .claude/prompt-coach/candidates.jsonl. Later, run /prompt-coach:report-issue to review candidates, add annotations, preview a redacted payload, and (only after explicit confirmation) file a GitHub issue.

Privacy guarantees: the report includes a structural signature (word count, question-shape, has-file-ref, etc.), the first 5 words of the prompt only, and your annotation. Full prompt content stays local in log.md.