Hands-On Harness Engineering

This is the practical companion to the 12-lecture course. The lectures explain why a harness exists; this course walks you through building one, step by step, in a working codebase.

By the end you will have shipped a small but real Node.js + TypeScript CLI called noted-cli with a complete harness around it: routing instruction files, a feature_list.json source of truth, an init.sh bootstrap, multi-session continuity artifacts, a three-layer verification pipeline, runtime observability, and a clean-state exit. You will also have run an ablation study comparing how the same agent performs against your code with and without that harness.

Get started

Syllabus

Prerequisites, time budget, learning outcomes, and the shape of the project you will build.

Start here

Module 00 — install Node, scaffold the empty repo, run the first command.

Capstone

Run an agent against your finished harness and measure the difference.

How the project grows

Each module adds one harness primitive on top of the previous module's checkpoint. The repository file tree at four key milestones:

Modules

• Module 00 — Onboarding: install the toolchain and scaffold the empty repo. Basic, ~30 min.
• Module 01 — Why Models Need Harnesses: A/B test prompt-only vs. one-line spec on the same task. Basic, ~45 min.
• Module 02 — Anatomy of a Harness: wire the five subsystems and run the first end-to-end loop. Basic, ~75 min.
• Module 03 — Repo as System of Record: add AGENTS.md, ARCHITECTURE.md, PRODUCT.md; pass the cold-start test. Intermediate, ~60 min.
• Module 04 — Splitting Instructions: refactor a bloated entry file into a routing file plus topic docs. Intermediate, ~60 min.
• Module 05 — Multi-Session Continuity: add PROGRESS.md and DECISIONS.md; rehearse clock-in / clock-out. Intermediate, ~75 min.
• Module 06 — Bootstrap Phase: write init.sh so a fresh clone reaches a verified start. Intermediate, ~60 min.
• Module 07 — Scope Boundaries (WIP=1): enforce one active feature, refuse the second until VCR returns to 1.0. Intermediate, ~60 min.
• Module 08 — Feature Lists as Primitives: turn the list into machine-checkable (behavior, verification, state) triples. Advanced, ~75 min.
• Module 09 — Three-Layer Termination: build a verify script that runs lint → unit → end-to-end. Advanced, ~90 min.
• Module 10 — Observability and Clean State: structured logs, sprint contract, evaluator rubric, five-dimension exit. Advanced, ~90 min.
• Module 11 — Capstone: Ablation Study: run a real agent against the harness vs. without it; produce a quality document. Advanced, ~2-3 hrs.

Who this course is for

You are a working software engineer who has used Claude Code, Cursor, Codex, or a similar agent at least a few times. You have noticed that the agent sometimes finishes faster than you would have, and sometimes confidently ships broken code. You want to understand the why and, more importantly, the how to fix it.

You do not need prior experience with Anthropic or OpenAI APIs. The CLI you build does retrieval with plain string matching — there is no LLM call required, no API key, no rate limit. The point is to feel the harness, not to integrate a model.

What you'll have built by the end

A noted-cli/ repository that:

• runs noted import <dir>, noted index, noted ask "<query>", noted status
• has a routing AGENTS.md plus topic docs under docs/
• ships an init.sh that takes a fresh clone to a verified start in under three minutes
• maintains feature_list.json, PROGRESS.md, DECISIONS.md as machine- and human-readable state
• has a ./verify.sh that runs lint → unit → end-to-end against the real binary and exits non-zero on failure
• emits structured logs and a sprint contract per task
• ends every session at clean state across the five dimensions
• includes an ablation-report.md showing measurable differences between agent runs with and without the harness

That repo is your portfolio artifact. Every module ends with a verification command you can paste into your terminal to confirm you are at the expected checkpoint.

Next steps

• Read the syllabus to confirm prerequisites and time budget.
• Open Module 00 and start building.
• Skim Lecture 01 if you want the theoretical motivation before you start coding.