Message 30 costs 31x message 1: what it costs to run a company on robots

The Team Talk (Alex's version)

SHOT builds its software with AI helpers, and those helpers charge by the word. Every word we send them, and every word they send back, goes on a bill. That bill is as real as the electricity that keeps the floodlights on.

Here is the catch. The AI has no memory. Every time you ask it something, it re-reads the whole conversation from the start, and you pay for the re-read. Deep into a long conversation, one question costs about 31 times what the first one did. It is like a player who has to rewatch the entire first half before every single touch of the ball. Brilliant on the ball, exhausting to run.

So we did what any good club does with a wage bill: we measured it, then we trimmed it. Shorter conversations. Only the notes the AI actually needs. The expensive star player saved for the big decisions, the squad players doing the everyday running. Our best trimmed session now costs less than a quarter of what it used to, same output.

For the person on the pitch, this is why SHOT can keep the app affordable for grassroots clubs. Every pound we do not burn on robot admin is a pound that stays on making the badge mean something for the kid on the far pitch.

The Deep Dive

SHOT is a small product-engineering operation running agent fleets, agentic CI, and multi-model orchestration to ship a consumer sports platform. The human core is deliberately split: CTO for architecture and data, chief engineer for product and platform, product-led CEO for the sport infrastructure we are trying to build. When your engineering department is mostly software, the token bill stops being a curiosity and becomes a line item, the same category as wages and hosting. So we treated it the way Silicon Valley treats cloud spend: measure it, understand the unit economics, engineer it down. All figures below are our measured data unless flagged.

Why the meter runs hot

Four mechanics drive the bill, and none of them are obvious from the pricing page.

The re-reading tax. The model has no memory between messages. Every turn re-processes the entire conversation history from scratch. Message 1 costs 1x; by message 30 the cumulative cost is roughly 31x the first message. Long conversations are not linearly more expensive, they compound.

Output costs about 5x input. Tokens the model writes are priced around five times the tokens it reads. Verbose answers, chatty progress narration, and full-file rewrites are the expensive direction of travel.

Thinking is billed as output. Extended reasoning is invisible in the transcript but charged like any other output. A single hard-thinking turn can burn 30,000+ tokens before a line of visible work appears. Reasoning depth is a dial, and leaving it on maximum for routine work is paying architect rates for photocopying.

Caching changes the shape of the problem. Static content at the front of the conversation (system prompt, project instructions, tool definitions) is cached and re-reads at roughly 10% of full price, but only while it stays byte-identical. Anything volatile near the top of the prompt breaks the cache and puts you back on full fare.

There is a quality mechanic hiding in the same place as the cost mechanic: past roughly 60% of the context window, output quality degrades as the model drowns in its own history. Cutting conversation length is not a trade-off between cost and quality. It improves both, which is the rare kind of optimisation you should do twice.

The levers, in order

We worked through these in sequence. The order matters because each lever tells you where the next one pays.

1. Observe first. Before touching anything, instrument. Per-session token counts, per-project burndown, a statusline that shows spend as it happens. If you are not tracking usage you are guessing, and every optimisation you make while guessing is a coin flip. Our first dashboards showed the truth immediately: a handful of long-running sessions accounted for the bulk of the bill, not the volume of small ones.

2. Compress what reaches the model. Most of what tools return is noise the model pays to read. Test runs do not need the passing lines, only the failures and the summary. Large JSON does not need its values, only its schema. Git status does not need the full listing, only "3 modified, 1 untracked". Applying failures-only and schema-over-values filtering at the shell layer took one representative 30-minute session from ~118k tokens to ~24k, same work delivered. That is not a rounding error, that is a 79% cut on the session's biggest cost centre.

3. Route by model tier. Premium models should hold judgement, not do grunt work. The pattern that stuck for us is the advisor: a cheap model drives the work end to end and consults the expensive model only on the hard calls, which in practice is around one decision in ten. Boilerplate, wiring, log triage, scaffolding, test execution: cheap tier. Architecture, security, the calls that are expensive to get wrong: premium tier. The premium model sees findings, not noise.

4. Cap conversation length. Given the 31x compounding and the ~60% quality cliff, we cut sessions off early and hand context to a fresh one via a structured handover note. A handover costs a few hundred tokens; the alternative is paying compound interest on every message that follows, at declining quality. Batching related requests into one message instead of drip-feeding them helps for the same reason: one context load instead of five.

5. Mind the tool-schema overhead. Every connected tool server dumps its full schema into every message, whether or not the tool is used. One browser-automation server alone was costing us around 15,000 tokens per message as pure standing overhead. We audited the standing toolset, cut to a daily-driver set, and load the rest on demand. This is the silent lever: nothing about a session looks wasteful until you see what it carries before the first word of your actual request.

The scars

Two honest failures, because the misses taught us more than the wins.

First, we ran unmonitored polling loops. An agent checking a condition every few minutes sounds cheap until you multiply a full context load by 288 cycles a day. The fix was structural, not behavioural: repeated work runs on schedules that fire on signal, not loops that poll on a timer, and headless jobs carry hard turn caps and budget caps so a stuck retry loop dies instead of billing all night.

Second, we tried codebase graph indexing to help agents navigate, on the theory that a smarter map means fewer exploratory reads. On a codebase our size it gave minimal gain over the AST and search navigation the tools already have, and the index went stale fast enough to become an integrity risk. We parked it. Not every clever idea survives contact with the meter.

Baked into the CI

This discipline is not a manual habit, it is in the pipeline. When we rebuilt our agentic PR-review flow (covered in our review-pipeline article), we rewrote it from 7 AI sessions per review to 4, worth roughly a 40% token reduction per PR. Every PR that lands now pays the optimised rate by default. That is the end state to aim for: token economics as a property of the system, not a virtue of the operator.

The wider point is the one that keeps us honest. Tokens are heading the way of RAM, a commodity nobody optimises once it feels cheap, and the vendors have no incentive to trim your bill for you. For a small operation running its engineering on agents, that discipline is the difference between a cost line that scales with ambition and one that scales with carelessness. We would rather it scaled with ambition.

Next in The Engine Room: two robots, one broken signup.