The app worked perfectly in the demo. Clean UI, fast clicks, the client nodded. Then real data showed up — a few thousand records instead of a few dozen — and the whole thing started to crawl. Opening a single task modal took four seconds. The cloud bill for the month arrived looking like a typo.
None of this showed up in the demo, because the demo had ten rows. Production had ten thousand.
This is the pattern we see over and over now that MSPs are shipping apps with AI. The tools are genuinely good — they get you to a working app faster than ever. But there’s a class of failure they’re structurally blind to, and it’s exactly the class that only surfaces in production.
AI writes code that runs. It doesn’t feel the fifty requests.
We recently looked at an AI-built task app where opening one modal fired more than fifty separate requests to the database. Fifty. To show one task.
The AI that wrote it did nothing “wrong” by its own lights — every line was syntactically fine and the feature worked. But it had no sense that fetching a task, then its comments one by one, then each commenter’s profile one by one, then each attachment one by one, adds up to a request storm. On ten rows it’s invisible. On a real workload it’s a four-second modal and a runaway bill.
AI models are trained to produce code that works. They have no felt sense of request count, data growth, or cost. That gap is where AI-built apps quietly die.
The failures that hide until production
The same blind spots show up again and again:
- N+1 and over-fetching. Data pulled one row at a time, or whole tables fetched to show a handful of fields. Fine on 10 rows, fatal on 10,000.
- Leaky security. This is the scariest one. AI secures the front door — login works — and then leaves the rooms unlocked. With Supabase especially, row-level security policies are subtly wrong constantly: one tenant can read another’s data, or an ID in a URL can be swapped to pull records that aren’t yours. It passes every test the AI thought to write.
- Runaway cost. Inefficient queries and per-item function calls don’t just run slow — they run up egress and invocation bills that can 10x what the app should cost.
- The missing unhappy paths. AI builds the success case and skips the rest: no loading states, no empty states, no error handling. The first time something fails in front of a client, the screen just freezes.
Why do AI-built apps work in the demo but break in production?
Because a demo tests the happy path on tiny data, and production tests everything else. AI is excellent at the happy path — that’s most of what it has seen. It is weak at the conditions that only appear at scale, under concurrency, in a security review, or when something goes wrong. Those aren’t edge cases in production; they’re Tuesday. The demo simply never exercised them.
The gap is a skill, not a tool
Here’s the part that matters: this isn’t an argument against building with AI. Use it — it’s fast, and speed is the whole point. AI gets you to a demo in days instead of months.
But the distance between that demo and something you can put in front of a paying client is a specific, senior-engineering skill: knowing to count the requests, to read the row-level security policy, to check what happens on ten thousand rows and on a failed network call. It’s the pass AI can’t run on itself, because it can’t see its own blind spot.
If you’re shipping AI-built apps to your clients, get that pass before they do. That’s exactly what our Redline audit is for — senior engineers hunt the N+1 queries, the leaky security, and the broken UX, and hand you a scored report with a prioritized fix list. AI gets you to a demo. We get you to production.
Building the app in the first place, or want it hardened as it’s built? That’s custom AI app development. Either way, let’s talk before the fifty-request modal reaches a client.
Scale your MSP with us.
Expert RMM engineers and dedicated technicians, ready to work inside your stack — without the hiring overhead.