Harness Engineering Skill Based on Mitchell Hashimoto’s Workflow

Agent harness improvement loop

Make each observed agent failure improve the environment for future sessions instead of correcting only the current prompt.

Capture a real failure

Before changing the harness, preserve:

• the assignment and relevant context;
• the environment and agent;
• changed files or proposed action;
• the incorrect behavior;
• the check, review, or evidence that proved it was wrong;
• the expected correct behavior.

Reduce it to the smallest reproducible failure. Do not create repository-wide guidance from an unverified anecdote.

Classify what was missing

Choose the primary failure class:

1. Stable fact: the agent did not know a repository convention, supported API, boundary, or required command.
2. Poor interface: the correct developer command is slow, confusing, nondeterministic, or returns unhelpful errors.
3. Missing observability: the agent cannot see logs, screenshots, generated output, application state, or another result it needs.
4. Missing verifier: no focused test or tool distinguishes correct behavior from incorrect behavior.
5. Codebase defect: the repository itself encodes an unsafe or misleading pattern that should be fixed rather than documented.

Choose the smallest durable remedy

• Add a concise AGENTS.md-equivalent instruction for a stable fact.
• Improve or wrap a developer command when the interface is the problem.
• Expose compact logs, screenshots, or state when observability is missing.
• Add a focused test or verification script when correctness lacks a signal.
• Fix the codebase when documentation would merely preserve a workaround.

Pair a new tool with the minimum instruction needed to make it discoverable. Scope instructions as narrowly as the repository permits.

Design agent-friendly feedback

The remedy should be fast, deterministic enough to trust, and explicit about misuse. Successful output should be short. Failure output should identify what failed and the next useful action. Avoid dumping entire logs when a filtered view can answer the question.

Replay in fresh context

Start a fresh agent session that has the improved harness but not the earlier solution. Give it the same class of task and observe whether it:

1. discovers the instruction or tool;
2. selects the correct command or API;
3. detects the failure itself;
4. reaches the expected result with less steering.

Retain the harness change only when the replay prevents the failure or exposes it early enough to guide correction. Tighten, relocate, or remove guidance that is ignored, ambiguous, redundant, or too broad.

Return

Include the exact instruction or tool, evidence from the replay, and any obsolete guidance removed during the change.

Source attribution

Independently compiled by XSkills from Mitchell Hashimoto's account of adopting coding agents. Hashimoto did not author or review this skill. The source defines harness engineering as turning observed bad agent behavior into better implicit prompting through repository instructions or programmed verification tools. The failure taxonomy, fresh-context replay, pruning criteria, and output table are XSkills adaptations. Attribution does not imply endorsement.