The prototype was never the hard part
Building something that works once is now genuinely easy. An afternoon gets you an agent that reads a request, picks a tool, and produces something impressive enough to demo. The mistake is reading that afternoon as evidence that the remaining work is incremental.
It is not incremental, because the prototype and the production system are solving different problems. The prototype answers: can this be done? Production answers: can this be done ten thousand times, by different people, under load, while a dependency is down, without anyone watching, and without the bill surprising anyone. Almost none of the work that answers the second question looks like prompt engineering.
First: the tools were never as reliable as the demo suggested
In a demo, every tool call succeeds. The API is up, the token is fresh, the schema matches, the network is fast, and the response is shaped exactly as expected. In production none of these hold reliably, and the failures are uncorrelated, so at volume something is always failing somewhere.
The instinct is to treat this in the prompt — to tell the model to handle errors gracefully. That does not work, because the model's idea of handling an error is to produce text about the error. What is needed is ordinary integration engineering: schema validation on the way in and out, explicit timeouts, retries with backoff, and an intermediary layer so the agent's logic does not have to know that a given vendor returns 200 with an error body.
Treating tool use as a first-class software integration rather than a prompt affordance is the single highest-value structural change most agent codebases need.
Second: state turns out to be real
A prototype holds state in memory for the duration of one run, which is fine, because a demo is one run. Production has concurrent runs, runs that die halfway, runs that need resuming, and runs whose intermediate results someone will later want to inspect.
The failure mode is not dramatic. It is a run that stopped somewhere in the middle, having already performed two of its five side effects, with no record of which two. Recovering from that manually is expensive and, at any volume, constant.
Durable, explicit state is not premature engineering for agent systems. It is the thing that makes an incident survivable.
Third: you cannot see what happened
Agent observability is genuinely different from application observability, and teams are repeatedly surprised by this. In ordinary software the same input produces the same code path, so a stack trace localises a problem. In an agent system, two identical inputs can take different paths depending on tool selection and intermediate results.
That means logging inputs and outputs tells you almost nothing about why a run went wrong. What you need is the trace: which tools were considered and selected, what they returned, how long each step took, what it cost, and where the run diverged from what you expected. Without that, every investigation starts from scratch and most end in a shrug.
Fourth: the economics change shape
Token consumption that is invisible in a demo becomes material at production volume, and it does not scale linearly with anything intuitive. Retries multiply it. Long-running agents accumulate context. A tool that returns more data than expected inflates every subsequent step in the run.
The specific unpleasant surprise is that cost is often driven by the failure paths rather than the happy path — a run that retries four times costs five times as much as one that succeeds, and the runs that retry are exactly the ones nobody is watching.
Fifth: a model change is a product change
This is the one that catches teams who did everything else right. A provider updates a model, or you switch tiers to save money, and behaviour shifts underneath a system that nobody edited. Without evals running on every change, the detection mechanism is a customer noticing, and the diagnosis is archaeology through logs you probably were not keeping.
This is why evaluation belongs in CI rather than in a notebook. Not because measurement is virtuous, but because it is the only thing standing between you and a silent regression you find out about from a support ticket.
What to do about it
Scope the production work explicitly rather than treating it as polish on the prototype. Concretely: tools as validated integrations, durable state, run traces including cost and timing, evals gating deploys, and irreversible actions behind validation.
None of this is exotic. It is mostly ordinary distributed-systems discipline applied to a component that happens to be non-deterministic. The reason it gets skipped is that the prototype worked, and the prototype working is genuinely misleading evidence.