Recursive Self-Improvement in Claude Code

This is a field procedure for making Claude Code better at later software tasks. You will diagnose a repeat failure from traces, change one part of the project harness, replay matched tasks, and decide from independent product evidence whether the change stays.

The example is a navigation feature. Users need temporary road closures respected by route search and spoken directions. The first implementation shows the closure on a map but still drives across it. The proposed improvement is a narrow route-constraint skill that forces the agent to trace a restriction through ingestion, graph eligibility, route geometry, and guidance. The audit task switches to a vehicle-height restriction, so the candidate must transfer beyond the exact closure it was written for.

You need a Git repo, Claude Code CLI, an executable route or browser check (or a named reviewer with a concrete checklist), and room for two disposable worktrees. Replace the navigation adapter and commands with your application's real interfaces. For the product story, read the blog; for the equivalent Codex mechanics, see the Codex guide.

1. Draw the control boundary

There are two changes. Delivery edits application code for a user task. Improvement edits Claude Code's future instructions, skills, or checks. The proposer may diagnose and draft H1, but it cannot rewrite the task pool, fixtures, evaluator, score rule, or promotion decision. Put those controller files outside the agent's writable worktrees. CLAUDE.md is useful context, not an access-control mechanism; file permissions, tool policy, and workspace isolation enforce boundaries.

H0 is today's project harness. H1 is H0 plus exactly one candidate change. A richer prompt, new model, extra tools, and a skill added together make the result uninterpretable. Start with a bounded skill; test other changes separately.

Write an outcome-based contract before the agent starts. The brief belongs to the controller and is identical in both arms. Example:

Build the independent oracle around the application, not Claude's summary. For this case an adapter calls the real route API or simulator and writes { "caseId": "NAV-41", "routeEdgeIds": ["..."], "maneuvers": [{"edgeId":"..."}], "etaSeconds": 420 }. An evaluator outside the worktree checks that neither route nor spoken maneuvers use restricted E17, and checks the control route and ETA separately. The Codex guide contains a complete Node evaluator for the edge checks. Copy it unchanged for both Claude arms, then add an ETA assertion for your application's expected tolerance. A hand-written response file only tests the file, not the product.

2. Collect a real H0 trace

Record a clean starting commit, claude --version, resolved model, dependency lockfile hash, service image versions, permissions, and the task brief. Run the current harness on several tasks and preserve the per-run stream. claude -p is the scriptable entry point; --output-format stream-json --verbose exposes messages and tool events. The agent's final answer and tool transcript help diagnose the failure, but the external evaluator determines acceptance.

The command below is an example for a repository whose tests are npm test and npm run test:routes. --tools limits built-in tool availability; --allowedTools preapproves listed uses; it does not restrict tools on its own. --disallowedTools "mcp__*" removes MCP tools for this controlled run. dontAsk denies any unresolved permission request, so a missing command grant becomes a visible blocked run. Keep the permission set equal for H0 and H1.

Do not add --bare here: it skips project instructions, skills, hooks, and MCP, so it would erase the harness difference under test. Store event streams away from the worktrees and redact credentials before sharing them. If a run ends at the turn cap, hits the cost cap, or cannot invoke a needed tool, record that exact result; do not convert it into “pass.” Claude's cost field is an estimate, not a business-cost measure.

Read the trace like a developer, not like a prompt editor. Identify the first unsupported assumption. Did Claude miss the closure ingestion code? Did it connect the map marker to the routing graph? Did it run a test that only checked UI state? Could it have reached the route API? Record the evidence path and the smallest recurrent cause. If only one task failed, first repair the application or oracle. A durable harness change needs repeated evidence or a high-impact failure with a clear, testable mechanism.

3. Encode one candidate in a project skill

Keep stable repository commands and constraints in CLAUDE.md or .claude/CLAUDE.md. Put the narrow procedure in .claude/skills/<name>/SKILL.md. The skill description controls when Claude considers it. Use a positive trigger and an exclusion so a route check does not consume time on unrelated map styling. Do not hardcode NAV-41's answer or fixture edge E17 into the skill; the point is transfer.

The project skill below may be invoked when its description matches. In a diagnostic session you can call it explicitly with /verify-route-constraints to check discovery; the trial itself should use ordinary user requests so it measures natural selection.

If the root cause is a permission or environment failure, a skill may be the wrong repair. Test a tool grant, service fixture, or build command as its own H1. Hooks can log or veto specific tool calls: PreToolUse may block before a call, while PostToolUse can only observe after success. A command hook must exit 2 to block. Hooks still do not make a writable evaluator trustworthy; keep evaluation outside the worktree.

4. Replay equal tasks from equal starts

Each paired task gets two clean worktrees at the same application commit. H1 gets only the candidate .claude/skills/verify-route-constraints/SKILL.md; H0 does not. Launch a fresh claude -p process for each arm. Pin the model, CLI version, max turns, budget, permissions, environment and services. Reset databases and caches between arms. Run serially when a shared simulator or service could leak state. Do not reuse conversation history or let H1 see the H0 diff.

This shell skeleton prepares one pair. It assumes the baseline commit has no skill at that path. Save the H1 skill shown above at $LAB/candidate/verify-route-constraints/SKILL.md and the brief at $LAB/tasks/NAV-41.md, outside the app repo.

After each run, the controller calls the real application adapter, then the protected evaluator. Save its JSON report, raw route response, worktree diff, stream trace, and a reviewer verdict. For each pair record pass/fail/blocked/unreviewed, human rework minutes, and estimated model cost. A pass requires the same frozen acceptance checks on both arms. A closure case that improved while a truck-height audit case failed is a regression, not a clean win.

Use the workbench's paired rows to record the outcomes. Pick a decision rule before reading H1: validation wins must exceed losses, the untouched audit must not regress, critical safety failures reject immediately, and extra rework or cost must stay below your limit. Small samples are a screening signal. Repeat runs and add domain coverage before making a broad productivity claim. A human owner reviews the exact H1 commit and keeps H0 available for rollback.

Fill the cost fields from the run metadata and keep raw traces alongside the report. The values above illustrate the record shape; they are not a measured Claude Code result. Preserve blocked and unreviewed as distinct outcomes.

5. Use Claude's plugin eval for a skill probe

If your candidate is a reusable skill, Claude Code v2.1.269+ has claude plugin eval. Package a copy of the skill as a test plugin, add realistic cases, and compare runs with and without the plugin. It starts fresh isolated sessions and normally repeats each case three times per arm. This can expose a skill that never activates or harms unrelated tasks. It is a supplement to the product-level worktree test: plugin graders over text or created files do not prove your navigation service routes safely.

Create a small plugin directory with .claude-plugin/plugin.json, skills/verify-route-constraints/SKILL.md (a copy of the project skill), and evals/<case>/prompt.md. The case prompt should describe a real request without naming the skill. A skill-invocation grader is useful for diagnosis but cannot be the outcome score: without the plugin it is impossible to invoke that skill, which would inflate the measured delta. Grade the result, include an unrelated control case, and use the separate product oracle for release. For code tasks, seed the same Git fixture in each run with a trusted context.scaffold_script and --scaffold; that script runs as you, so review it first.

This rubric detects an obvious workflow miss, but it judges a response. It can be fooled by an incorrect claim; keep the controller-run route evaluator as the acceptance source. Add a second control case for a map-copy edit where the route skill should be inert. If you create a tool_used: Skill grader for debugging invocation, read its indicator separately from the with/without outcome score.

The first command's with/without delta reflects the plugin eval suite and its graders; the second is a single-arm diagnostic, not a promotion result. If your Claude build does not expose plugin eval or says it is unavailable, use the manual worktree experiment above. For a skill-only plugin test, remember that Claude's isolated eval workspace does not automatically contain your full project's CLAUDE.md or dependencies; scaffold those when they are part of the behavior under test.

Transfer this to your own work

For an auth change, the oracle uses two identities and checks denied operations leave no data behind. For a data migration, it checks old/new reads, backfill counts, and rollback on production-shaped fixtures. For a refactor, it freezes behavior tests and performance limits. For a new feature, it tests the complete user journey and a neighboring journey the feature could break. The harness candidate changes the repeated working method (for example, test forbidden identities or trace derived state), not the answer to one task.

If you direct agents without writing code, keep the same structure: record a real user outcome, ask an engineer or separate reviewer to own a concrete checklist, collect a screenshot or API response for each arm, and keep the same model, task, budget, and starting commit. You can judge a feature manually; you cannot infer improvement from Claude saying it followed the skill.

Troubleshooting: A missing skill may be at the wrong path or have a weak description; test discovery in a fresh session. A permission denial is a blocked run, not a model failure. A large plugin-eval delta with unchanged product acceptance means the grader is measuring the wrong thing. If H1 changes a protected test or task brief, discard that pair and repair the boundary before replaying it.