Most AI projects fail before the model is the problem
A demo won the room in twenty minutes. The pilot had cut inbound calls by thirty percent in a controlled test, the number was on the screen, and everyone in the room agreed that number was the point. The budget got approved on a Tuesday. Then the same project spent six months forgetting why the room said yes. By spring the team was arguing about which prompt framework to standardize on, and by summer the argument had moved to vector databases. Nobody in either meeting said the words “thirty percent.” The goal was never cancelled. It just stopped showing up. I have sat inside that drift, and it kills AI projects with nothing wrong with the model.
The headline is that 80% of them fail to deliver the value they promised, and people cite that number as proof the technology is overhyped or the tooling isn’t ready. The real breakdown is more specific. 33.8% get abandoned before they ever reach production, and another 28.4% reach production and produce nothing the business uses. The failure concentrates in the organizational layer, and it starts early.
Executive sponsorship is the clearest tell. In 56% of failed projects the executive support evaporated within six months of the funding approval. I can describe the sequence because I have lived inside it. The demo creates a moment where the problem is sharp and success is one number everyone agrees on. Then the work moves into implementation, where the visible output stops being “calls went down” and becomes “we picked this retrieval strategy.” The sponsor who approved a business outcome starts hearing about architecture, stops being able to judge what they hear, and quietly checks out. The team keeps making consequential choices with no one left in the room to hold those choices against the original number.
Use-case drift does not announce itself. It accumulates through a series of reasonable pivots, each one a real question that deserves a real answer. The trap is that answering the infrastructure question displaces the original question instead of serving it. A team that can tell you exactly which vector database they chose and why, but cannot tell you their last score against the thirty-percent target, has drifted. The engineering is real. The anchor is gone. I wrote about a smaller version of this in where verification belongs in an agent harness, where an agent that codes and judges its own work in the same context sees what it intended, not what is on the page. Teams do the exact same thing one level up.
The teams that ship treat the original metric as a load-bearing wall. Every architectural decision gets pushed against it. “Does this retrieval approach move our score on the support-call benchmark” is a question you can answer this week. “Is this the right framework” is a question that can swallow a quarter. The discipline is keeping the first question in the room, which means keeping a person in the room whose only job is to ask it.
The second failure mode is quieter, and it arrives after launch. The system reaches production, the monitoring shows no incidents, and the adoption numbers do not move. People don’t trust the output, they route around it, or they use it in a way that erases the benefit it was built to deliver. The system works. The behavior didn’t change. This is the same lie I keep chasing in my harness writing, where a green test suite sits on top of a broken page. Here the lie is a clean dashboard sitting on top of a tool nobody opens.
This happens when a team treats deployment as the finish line. A working demo proves the model can do the task under controlled conditions. It does not prove that people will rebuild their workflow around it, trust it when the stakes are high, or choose it over what they already do. The teams that avoid this ask about adoption before they ask about architecture. They build their pilots to measure behavior change, not output quality, and they treat trust as something earned over weeks rather than granted on launch day.
The fix for both failure modes is the same, and it is boring. Build the evaluation pipeline before anything ships. Not a testing phase bolted on at the end, but a set of labeled cases that score real output against the real task, written before the first model version is chosen. Once that exists, every later decision has something to check against. The moment the demo works, the question turns from “can this do the task” into “what does the eval score on these fifty edge cases, and how does that connect to the thirty-percent target.”
Teams that put the full set of governance gates in place before shipping cut delivery effort by 30 to 50 percent. The gates do not slow the project down. They prevent the rework that comes from a system that won the demo and broke three weeks into production. I made the same argument about context window economics: the cheap discipline you pay for up front is what buys you room to move later.
The model is the least interesting variable in most of these failures. Context windows are longer, inference is cheaper, and the frontier models I can reach today are better than anything from a year ago, and none of it touches the pattern. It is the same point I keep coming back to about what you build while the tools get faster: a better model does not solve the problem you actually have. Projects fail because they lose the question. Every AI project you are running has one number that justified it, and one person whose job is to keep showing up and measure against it. If you cannot name both today, the project is already drifting, and no model on the roadmap will pull it back.
Be the first to comment
Thank you! Your comment will appear shortly, usually within a couple of minutes.
Something went wrong. Try again or contact me directly.