322 playbooks in 4 days: our first fleet

The Team Talk (Alex's version)

SHOT needed a coaching playbook for every sport we support: what a player should work on, week by week, for a whole season, at every level from a five-year-old having a kickabout to a pro. Written by hand, that is years of work for a room full of coaches.

We are a small product-engineering core. So we built a fleet instead.

Think of it like this: instead of one very clever robot writing the whole encyclopaedia, we ran a squad of robot workers, each given one small, checkable job: one block of one season for one sport. A supervisor handed out the work, a watchdog checked nobody had stalled, and every finished piece went through a quality gate before it counted. Fail the gate, and only that piece gets redone, not the whole book.

Four days later we had a season-by-season development playbook for every sport on the platform: over 300 frameworks, more than 20,000 rows of coaching content, built by thousands of short robot shifts that cost less than a week of one contractor's time.

This blog is us showing our working. Everything we publish here really runs, the numbers come from our own logs, and when something breaks we will write that up too; that is the deal.

The Deep Dive

The shape of the problem

SHOT PERFORM is our player-development engine. Each framework covers one sport at one level: 13 season blocks by 5 pillars (Technical, Psychological, Social, Physical, Personal), 65 rows, each row carrying prepare/reflect/coach questions, five context-driven answers, and five coach next steps. Multiply by 46 sports and 7 levels per sport and you get 322 frameworks and north of 20,000 rows of structured coaching content.

That is not a "write a good prompt" problem. A single long-running AI session degrades as its context fills, drifts in format, and one bad hallucination poisons everything after it. It is a manufacturing problem, and we treated it like one.

The unit of work is the unit of verification

The core design decision: the unit of work is one block, a five-row CSV fragment taking roughly three minutes of worker time. Small enough that a cheap model does it well, small enough to verify deterministically, and small enough that a rejected fragment costs three minutes to rebuild, not a day.

Every fragment passes a deterministic checker (fragcheck) before it counts: column shape, row counts, required fields, format contract. No "looks good to me" from another model as the primary gate; the gate is code. AI checks quality where taste is needed (a separate native-QA pass on translations), but structure is enforced mechanically.

The supervisor pattern

The fleet is plain PowerShell driving headless CLI workers (no orchestration framework, no queue infrastructure):

• Fixed-batch concurrency. Workers launch in fixed batches (launch, wait for all, launch the next), hard-capped so the machine cannot fork-bomb itself. Our sweet spot on one laptop was about six concurrent workers.
• Detached workers. Workers run detached from the editor session, so closing the laptop lid on VS Code does not kill a build. Progress lives in JSON files on disk; any process can read the state and resume.
• A watchdog, not a babysitter. A monitor process tracks per-phase timeout rates and flags stalls. Humans look at a dashboard, not at terminals.
• Rebuild only what failed. State tracking is per fragment. A bad block is regenerated alone; hand-fixed content is protected from being overwritten by stale cached chunks (a real bug we hit; see scars below).

Total: 4,600+ worker sessions across four days for the initial English build.

Multi-model, deliberately

The build and its follow-up localisation (five languages, 325 frameworks each) ran as two independent worker tracks on different model families (Claude on one, Codex/GPT on the other), with different concurrency settings per track. Partly throughput, partly rate limits, partly deliberate: two model families means a systematic weakness in one shows up as disagreement rather than as silent uniform error. Translations then face a second, native-speaker-persona QA pass at roughly 23 chunks an hour.

The scars (what we got wrong)

• Wrong engine by default. A resumed run picked up the wrong CLI default and burned hours on the wrong model before the monitor's cost curve gave it away. Fix: engine is now explicit per run, never inherited.
• Cached chunks overwrote hand edits. A worker re-emitting a cached fragment reverted human corrections. Fix: hand-edited files are fenced off from regeneration.
• Shared progress files collide. Two tracks writing one progress file is a race. Fix: one progress file per track, merged read-only by the dashboard.

Why this matters beyond playbooks

The pattern is the point: decompose to the smallest verifiable unit, gate with deterministic checks, supervise with boring scripts, cap concurrency, track state on disk, rebuild only failures. We have since reused it to generate a 10-chapter strategy document set with per-chapter audit-and-fix chains. Same supervisor, different cargo.

A small team does not out-type a content department. It out-manufactures one.

The invoice

Next in The Engine Room: what it costs to run a company on robots.