Passions Play Vol. IX
Filed under Development

A Quiet Validate Script for Coding Agents

· development ai cli typescript

Early on with Claude Code I added an npm run validate script to one of my projects. The idea was to give an AI agent a single command to verify its own work while iterating toward a solution. That script chains together a TypeScript check, two Jest suites, and a Biome lint pass. If all four checks are happy, the work is probably sound.

The script worked. The default output cost more than it was worth.

Jest prints a green wall listing every passing test file. Biome summarizes what it checked. A clean run dumps close to a thousand tokens of “everything is fine” into the agent’s context, and twenty runs in a session is twenty thousand tokens of noise crowding out the code that actually matters.

Every tool in the shop running full-tilt just to tell me nothing’s wrong.

So I was designing tool output for a reader who isn’t human — and whose attention I pay for by the token.

Rule of silence

I was reminded of a core piece of Unix philosophy, the rule of silence, which says:

if a program has nothing surprising, interesting, or useful to say, it should say nothing.

make had this right decades ago: say nothing on success, let the exit code carry the signal, and when something breaks print a terse, located error — file:line: message — and stop. Fail fast, fail hard, don’t narrate the happy path.

A validate script for an agent should behave the same way. The exit code is the primary channel. On success it should cost almost nothing to read. On failure it should surface the minimum an agent needs to locate and fix the problem, and not one line more.

What I landed on

Here’s what one clean run looked like before — just the Jest slice of it:

 PASS  src/lib/graph.test.ts
 PASS  src/lib/resolve.test.ts
 PASS  src/lib/dedupe.test.ts
 PASS  src/api/nodes.test.ts
 PASS  src/api/edges.test.ts
 PASS  src/cli/render.test.ts
   ... 6 more suites

Test Suites: 12 passed, 12 total
Tests:       142 passed, 142 total
Snapshots:   0 total
Time:        4.08 s
Ran all test suites.

Multiply that by two suites, add Biome’s summary and tsc’s output, and run it twenty times while the agent iterates and you have a recipe for extreme token burn.

Here’s what I roughly wanted to end with though I may trim the success output even more:

Success:

✓ tsc   ✓ biome   ✓ jest:unit (142)   ✓ jest:ui (38)
VALIDATE: PASS

Failure:

✗ tsc (3 errors)
  src/lib/graph.ts:44:7  TS2322  Type 'string' is not assignable to type 'NodeId'
  src/lib/graph.ts:51:12 TS2322  Type 'string' is not assignable to type 'NodeId'
  src/api/nodes.ts:19:3  TS2554  Expected 2 arguments, but got 1
✓ biome
✗ jest:unit (1 failed)
  graph > resolves transitive conflicts
  Expected: 2  Received: 3   src/lib/graph.test.ts:88
✓ jest:ui (38)
VALIDATE: FAIL  tsc=3  biome=0  jest:unit=1  jest:ui=0

Thirty-ish tokens when things are green, a scannable list when they’re not, and a trailer line the agent can parse at a glance.

Don’t grep — parse what the tools already give you

With the general direction in place, my first instinct was a shell wrapper that greps stdout for “error”. That broke fast: some of my tests print the word “error” in their console.log output.

After a little research, I found that all three tools can emit something structured or stably formatted, so there’s no reason to pattern-match against unstable output.

  • tsc has no JSON reporter, but tsc --noEmit --pretty false prints exactly one path(line,col): error TSxxxx: message per line. Stable, parseable, easy to dedupe.
  • Jest takes --json, or you can lean on jest-silent-reporter, which suppresses passing output entirely and only surfaces failures plus a summary. Either way, pass --collectCoverage=false during iteration — instrumentation output is pure noise mid-loop.
  • Biome has --reporter=summary (already terse) and --reporter=json if you want to format it yourself.

So the orchestrator became a thin script written in TypeScript, since the project already has a TypeScript toolchain. It runs each stage in a defined order, captures stdout and stderr, parses each tool’s output into a small { stage, failures[] } shape, and prints the compressed view above.

Design considerations

Beyond hiding success, three things it does to cut tokens further:

Budget every stage. Each failing tool is capped at ~20 lines with an escape hatch: …12 more errors — run npm run validate:tsc for full output. The agent can ask for the rest, and it rarely needs to.

Deduplicate. One bad shared type throwing the same TS2322 across thirty files should report once with a count, not thirty times.

Order by cost and likelihood. Run the cheap, fail-fast checks first — Biome, then tsc, then Jest. A syntax error should never wait behind a thirty-second test run.

Wiring it into the loop

This is the part of the process I’m working on now.

The output is cheap, so validation can happen earlier and more often. A Claude Code PostToolUse hook can run validate after edits and use suppressOutput on success so nothing hits the transcript, then emit a non-zero exit plus terse stderr on failure so the agent self-corrects before I’m even in the loop. The catch is that it fires on every edit, so it has to be genuinely fast — full tsc on every keystroke-equivalent is too slow. Scope it, debounce it, or save it for a pre-commit trigger.

Where this leaves me

Applying the Unix rule of silence to a machine reader instead of a human one was a great shift in mindset. Same instinct — say nothing when there’s nothing to say — pointed at a context window instead of a tired developer at a terminal.

The other thing that fell out of this work: once the gate is cheap to run and cheap to read, AGENTS.md doesn’t have to carry “don’t do X” anymore. The validator enforces what a good solution looks like, every time, and the prompt file gets shorter.