Getting AI to Production: Why the Hardest Step Is Deciding Not to Build

Convincing demos are easy. Useful systems are rare. The difference is rarely the model. It is the nerve to kill the wrong idea before you build it, and the discipline to carry the right one into production.

A product we worked on set out to solve a real problem, rewarding people for the exact value their training data adds. On paper it did almost everything right. Contributors uploaded private or synthetic data. The system logged each contribution's provenance on a blockchain, fine-tuned a model on it, measured how much each one improved the result, and paid the contributor automatically, but only when the model got better. The architecture was clean, the demo convincing, the logic hard to argue with. Not one real user ever tried it. 

That gap, between a system that looks sound and one a real person can use, is where most AI dies. The reflex is to blame the technology, the model, the vendor, the next platform. The evidence points elsewhere. A 2025 MIT study, Project NANDA, found that 95% of organisations had seen no measurable return from generative AI, measured against the bottom line. S&P Global recorded the share abandoning most of their AI work climbing from 17% to 42% in a year. The cause is rarely the algorithm. It is the process around it, and the discipline most organisations skip is the hardest one, deciding before they build whether the thing can work at all.

Why the elegant product was doomed 

Look closely at why that product failed, because it is the larger failure in miniature. Its whole value depended on measuring how much a single contributor's data improved a model, then paying them for exactly that. That measurement is one of the hard, still-unsolved problems in applied machine learning. The methods that exist are costly, unstable, and cannot produce a per-contributor figure anyone would defend. The payment at the heart of the product rested on a number no one could trust. 

The rest held up no better. Paying only when the model improved handed the judgement of improvement to the buyer, the one party with every reason to say it had not. Serious AI teams could never fine-tune on outside data inside someone else's app, because the legal and security exposure rules it out on its own. The blockchain, the most visible piece, proved only that a file existed at a given moment, nothing about quality, legality, or trust, which were the real problems. 

None of these are model failures. They are decisions, or the absence of them, made long before the technology became the bottleneck. The problem was worth wanting; the mechanism could not be built or trusted, and no delivery discipline rescues a product from that. Process improves the odds for the right ideas and does nothing for the wrong ones. That is why the first discipline is honest assessment, not delivery. 

The 5-A discipline, and the move that kills 

Honest assessment is the first move, and at Kreeda it sits inside a wider discipline that decides which AI to build, in what order, and when to stop. We call it the 5-A, and the five moves do not carry equal weight. Assess decides whether to build at all. Augment and Automate are stages of handover, one proven task before any platform that promises ten. Assure gates every launch. Accelerate loops back to the next one. The order they apply in is the whole argument. 

Assess comes first, and its real job is to kill. The data product would have died here, in a week, against two blunt questions. Can the contribution be measured and trusted? Will a real AI team ever run this inside their own stack? Sometimes the problem is wrong; more often it is worth solving but the mechanism cannot be built, and the assessment must catch both. This is the hard part, because a convincing demo and a doomed mechanism can look identical for months. The most expensive AI mistake is solving the wrong problem well, so the discipline is to find where AI removes a real bottleneck and, more often than anyone admits, to decide it does not and stop. The bottleneck is rarely glamorous. Most budgets chase the visible work, sales and marketing, while the steadier returns sit in back-office tasks no one demos. When agentic projects get cancelled over the next couple of years, and many will, it will rarely be that the agents failed, but that no one assessed hard enough before they began. 

Assess carries the most weight of the five, because it is the only one that can end a project before it has cost anything. 

The unglamorous path to production 

What that effort looks like barely changes from one system to the next, and almost none of it is the model. It begins by framing a problem statement and a measurable definition of success before anyone builds, the workflow redesigned before the model is chosen. Then a prototype built to learn rather than impress, then the gate most teams skip, evaluation, where a defined test decides whether the output is good and safe enough before a single user sees it. As Andrew Ng has long argued, disciplined evaluation and error analysis move a project forward more than the newest tool. 

Only then integration and a staged rollout to a narrow slice of real use, a person in the loop, because the gap between a prototype and a system that survives live traffic, drifting data, and legacy plumbing is far longer than any demo suggests. Then production proper, monitoring, a service-level commitment, and incident response that works at two in the morning, which on one platform we maintain means a health matrix across cloud, database, and identity providers, every incident answered against an SLA. And finally the loop most programmes never build, feeding real results back so the system keeps improving, without which even a strong launch quietly decays. A tool can speed up any of these steps. It cannot replace one, and it cannot run them in the right order for you. 

The tools-first trap, and the objection that almost holds 

A tool is an amplifier. Point it at a broken process and it does not fix the process. It produces impressive things that cannot ship, only faster. A platform chosen before the goal is clear, automation on requirements no one pinned down, AI standing in for the expertise that ships work. The tooling is rarely the weakest link, and the process beneath it almost always is. 

The strongest objection is not the one people usually raise. It is that a good tool often carries a better process inside it, and adopting it can be how a team discovers the process it never had. That is true, and worth conceding. A well-built platform encodes the sequence its makers learned the hard way, gate, rollout, monitoring, and a team that adopts it inherits a discipline it could not have written itself. We see this often, and recommend buying far more than building. 

The limit is precise. A tool encodes a process only for the problems its makers already understood. It cannot tell you whether your problem is one of them, whether your data can support it, or whether anyone will use the result. It forces a process, never the judgement about whether this is the right thing to build. It accelerates a team that has already done the assessment, and quietly entrenches a bad direction for one that has not. Buying does not skip the assessment; it refuses to rebuild what someone has already solved. The trap was never the tool. It is reaching for one before the outcome, the workflow, and the gate are defined. 

What to change this quarter 

Most of this can begin this quarter, and none of it needs a new platform. Ask the killing question of every idea, whether it can be built and whether a real person will use it. Name one person accountable for the route from experiment to production. Require a quantified definition of success before any pilot is funded, and cancel the ones that cannot show it. Put the evaluation gate in front of every pilot, no exceptions for the impressive ones. Change the scoreboard, so success is measured by what reached production and held, not how many experiments are running. The hardest of these is the nerve to stop, and it frees the people and the budget for the work that deserves to ship. 

The choice underneath it is stark. One path keeps buying tools, trusting demos, and accumulating experiments that never ship, the budget already gone. The other assesses before it builds, kills early, fixes the process, and ships AI that runs. The difference will not show up in the next demo. It shows up a year later, in what is still running. 

The most useful first step costs nothing. Take the last AI idea everyone loved and ask, before another line is written, whether it can be made to work and whether a real person will ever try it. The answer is rarely about the technology. It is almost always about a problem no one had yet agreed to solve.