Soundings
Our Perspectives and Insights
AI Made Prototyping Cheap, Not the Path to Production
Part two of a series. Part one, Most AI Pilots Aren’t Killed. They’re Never Allowed to Die asked whether every pilot deserves to reach production. This one starts with the ones that do.
The prototype worked and the demo went great. It was built with AI, did the thing it was supposed to, and everyone is impressed because this type of effort used to take an entire quarter and required a budget. Then comes the sentence that starts the actual project. Great. Let’s put it into production.
In part one of this series, we argued that an AI pilot is a hypothesis and many more should fail than are currently allowed to. This one is about the AI pilots that were right, that proved real value, and that still never show up in production, or showed up and quietly fell over. IDC counts four of every 33 proof-of-concepts making it. The reason it gives is not the technology. It is production readiness: data, process, infrastructure. Which is a different way of saying the hard part is not the build.
Long before AI, and well before spreadsheets lived in a browser, I led a team building grant and clinical trial administration software for clinical research. At the center of it was a browser-based spreadsheet we built ourselves. The prototype demoed beautifully to leadership and to clients. Then we used it for real clinical trials, with an order of magnitude more rows and columns, and it broke right in front of us. There was simply too much to put on a page, and nothing about the way we had built it survived contact with that. The idea was right. The prototype had proved the idea, and it had proved nothing else.
A Prototype Is a Question. Production Is a Promise.
A prototype answers one question: is there something here? That is the whole job, and answering it has gotten dramatically cheaper with AI.
Production is not a bigger answer to the same question. It is a promise, made to people who are going to depend on the thing whether or not you are awake. Does it hold at 2am when the person who built it is asleep? Does it hold at four hundred times the volume it was tested at? What happens when the data is malformed, when the provider has an outage, when somebody feeds it something they should not have? Who gets paged? What does it cost to run when nobody is watching the bill?
None of those are questions about whether the idea is any good. The prototype answered that one and stopped.
Everybody in the Room Is in the Dark
A client once described it to me as walking through a room with the lights off, stumbling into the furniture, because you have never seen what the room is supposed to look like. That is exactly what it feels like from the inside, and I’ve been there holding the flashlight and banging my shins.
The obvious read is that the customer is the one in the dark. That is only half of it, and it is the less useful half.
The team that told you the prototype was ready is usually in the same room. Building prototypes is what they are good at, and they are fast at it. But plenty of them have never operated anything, never been on call for it, never watched something they built get hit by traffic it was not designed for. They assume AI collapsed the distance between prototype and production because it collapsed the distance between idea and prototype. Those are two different distances, and only one of them moved.
The tell is not that they are in the dark. Everyone is at the start of something new. The tell is the one who will not say so.
So you get a client discovering the room in the dark and an expert discovering it at the same time, side by side, both moving with more confidence than the situation supports. The tell is not that they are in the dark. Everyone is at the start of something new. The tell is the one who will not say so.
The Enterprise Has the Light Switch and Never Aimed It at AI
Large companies already know what is in the room. They have security review, threat modeling, load testing, disaster recovery, change control, support models, runbooks. Every one of those rails exists because at some point something fell over and it cost real money.
Werner Vogels, Amazon’s CTO, has been repeating the same six words for most of two decades: “Everything fails all the time.” Enterprise release process is what an organization builds once it finally believes him.
And those same companies, holding every one of those rails, are not yet pointing them at the AI work. The prototype comes out of an innovation team, or a vendor, or somebody’s side project, and it arrives at the production door having never been asked to meet standards the company would apply without hesitation to a far less consequential piece of software. The light switch is on the wall, but nobody’s reaching for it.
The light switch is on the wall, but nobody’s reaching for it.
Then the Pendulum Swings Too Far
Once this gets noticed, the reflex is to swing all the way back. Every AI prototype goes through the full enterprise release cycle, a year or more of it, no exceptions, and you’ve now killed the advantage AI gave you.
That is the wrong correction, and it helps to remember why the cycle got that long in the first place. Those processes were sized for a world where building was slow and expensive. When the build itself took a year, a two month approval track was a rounding error. The process was proportional to the build.
AI changed the build and did nothing to the process. Run every prototype through a cycle designed for the old ratio and you lose the people too, because nobody stays excited about an idea that long.
So the answer is not the prototype team’s answer (ship it, we will find out), and it is not the enterprise reflex (everything through the gate). You keep the standards and you change when they get applied. Not a gate at the end. A constraint at the start.
How to Build Production In From the First Prompt
Every one of these works the same way. Your production standard is a document you already own, and the thing writing your software can read it.
- Put the security standard in the build instruction, not the review queue. Your security policy is already written down. Put it in the prompt on day one, so nothing gets built that has to be unbuilt in week six.
- Design the load test at the same time as the thing it tests. Same session, real numbers: expected volume, peak volume, acceptable response time. Find out in an afternoon instead of in front of users.
- Decide how it fails before you find out. Failover, graceful degradation, and a rollback you have actually run once. AI writes this well when you ask. Almost nobody asks.
- Give it somewhere to be watched from. Logging, alerting, and a name attached to both. Knowing a system is running is not the same as knowing it is right, and you cannot ask the second question without the first.
- Hand it over with a runbook, not a demo. Who operates it, what breaks, what to do at 2am. Write it while you build, because the context is never richer than the moment the thing was made.
If You Have Never Seen the Room With the Lights On
All of that assumes a company that has the production rails and is not aiming them. Plenty of companies do not have any.
Small and mid-size businesses are going to consume more AI than anyone, and most of them have never run a security review, a load test, or a runbook, because they have never built software at a scale that demanded one. They are not refusing to turn the light on. They don’t have lights and have never been in the room.
For them the five moves above are not a compression of an existing process. They are the process, and this is the first moment in the history of the industry when it has been cheap enough to have one. The same tool that writes the feature writes the test, the failover, and the runbook. A five person company can now ship something with more discipline behind it than a lot of enterprise software had a decade ago. We’ll cover this aspect in a future article.
The Bottom Line
Zero to one used to be the expensive part. It got cheaper and faster than most of us expected, and that is real, and it is worth celebrating for about a day.
The prototype was the question. Everything after it is the promise.
One to sixty did not move. Making something robust, secure, observable, and survivable is the same work it always was, except now it is the whole job instead of the tail end of it. The prototype was the question. Everything after it is the promise.
And if somebody tells you those two are the same distance, they are in the dark room with you. They are just walking more confidently into the dark furniture.
A Pilot & Rutter principle.