Why the Second AI Deployment Is the Real Product

DEV Community

The Second Deployment Is the Product

Originally published at Trida AI. The most expensive sentence in our line of work is "we've done this before." Said loosely, it means somebody on the team remembers a similar project. Said properly, it should mean something you can point to: a shelf of artifacts left over from the last deployment, which the next one starts from. The distance between those two meanings is where forward deployed engineering either becomes a business or stays a run of heroic one-offs. A first deployment is discovery at full price. The team learns what the client's data actually looks like, which approvals really matter, where the workflow breaks on a bad Tuesday. That learning is the most valuable thing the engagement produces, and the most perishable. If it lives in the heads of the engineers who did the work, it depreciates on a schedule nobody controls: people rotate, context fades, someone leaves. The second deployment starts close to zero, and the client pays again for lessons already bought once. Our position is simple. Judge a deployment team by what the first deployment leaves behind. The residue is the product. But residue does not curate itself, and most of what a project leaves behind is not reusable. Knowing which parts are, and doing the unglamorous work of making them so, is the discipline this post is about. What carries over Failure catalogues carry furthest. The ways an AI system fails in production repeat across clients because they come from the machinery, not from the business: a source replaced after indexing, quoted as if current; an ambiguous instruction resolved silently instead of asked about; a tool result trusted after the thing it described had changed; an approval requested after the action, not before. A catalogue entry names the shape, says how the system detects it, and says what happens instead. The detection and the response transfer almost intact. The examples inside the entry do not — they get redrawn from the new client's own data, because the old client's data is not ours to carry, and because a failure seen only in someone else's examples is one you will not recognise in yours. Eval sets transfer as skeletons. The question classes, the scoring rubric, the taxonomy of hard cases: those carry. The gold answers do not — what counts as correct is a business judgment, and it belongs to the business being judged. Starting from a skeleton still changes the second deployment's shape: a regression gate exists from the first week, in form, waiting for the new client's labels, instead of being invented under deadline pressure near the end, which is when eval sets usually get built, and why they so often get built badly. Connector patterns transfer; mappings do not. How a class of systems behaves — its authentication flow, its pagination, how it signals rate limits, whether its writes are idempotent — is knowledge that survives the move to the next client running the same class of system. What a field means inside one company's estate does not survive, and pretending otherwise is how a second deployment imports the first client's data model by accident. Review and tracing shapes transfer for a duller reason: the accountability questions are the same everywhere. What did the system do, on what evidence, who approved it, how do we undo it. The records that answer those questions can be rebuilt from a pattern, at a fraction of the cost of designing them from a blank page. What the first deployment hands over. Artifacts move through a curated library; judgments stay in the room. One catalogue entry, carried across. The check and the hold behaviour moved unchanged; the stamp and the staleness window were rewritten. Example content. What has to stay bespoke Data semantics stay. What "active customer" means, when a ticket counts as closed, which of three conflicting records wins: no library knows this, because it is not knowledge about systems. It is knowledge about one business, and it has to be learned in the room, every...

Powered by Readybot.io