Your code passed the linter. Your app failed a 2-minute curl test.

Igor's checklist is genuinely good

A few weeks ago, a FastroAI post hit the top of r/vibecoding. The hook was the Abrahub story — a Brazilian founder celebrating $100k/month in revenue, the whole thing built with AI. A user named Tiago exfiltrated the platform's personal data in two minutes. No brute force. No sophisticated exploit. The routes just answered every request, authenticated or not. Then came 17 checklist items: RLS off by default, missing auth middleware, IDOR on object IDs, Stripe webhooks that accept any payload.

People screenshot-saved it. The list traveled. It deserved to — every item maps to a real class of bug that appears in AI-generated code, over and over.

Go read it. This article starts where that checklist ends.

Three categories, two blind spots

When builders start thinking about security, they reach for one of three tools. Each one is solving a real problem. Each one has a hard limit. Understanding where each stops is the whole point.

Secure templates (FastroAI and its cousins) ship with safe defaults baked in. Auth middleware wired to every route. RLS enabled. Webhook signature checking from day one. The pitch is honest: start from a clean foundation.

SAST scanners (xploitscan's "300 apps" dataset lives here) read your source code and flag known-bad patterns. SQL injection, hardcoded secrets, dangerous function calls. Genuinely useful for catching things that are visibly wrong.

Runtime scanners send actual HTTP requests to your running app and watch what comes back.

The first two categories operate on source code. The third operates on behavior. And behavior is what matters when someone is actually using your app.

The boilerplate expires the moment you edit

FastroAI protects you at t=0. That guarantee is real — on the day you deploy the template, the auth middleware is there, RLS is on, webhooks are verified.

Then you ship a feature.

You ask Claude or Lovable to add a /api/reports endpoint. The AI doesn't know what the template promised. It pattern-matches to the task in front of it: "write a route that returns report data by ID." Here's what it often produces:

// app/api/reports/[id]/route.ts
export async function GET(
  request: Request,
  { params }: { params: { id: string } }
) {
  const report = await db.query(
    'SELECT * FROM reports WHERE id = $1',
    [params.id]
  );
  return Response.json(report.rows[0]);
}

The route works. The app builds. The template's auth middleware does not automatically protect routes you add after the fact — you have to wire it yourself, and the AI skipped that step.

The boilerplate protected you at deploy. Every new route after that was on you. And the AI generating those routes has no memory of the template's guarantees.

SAST can't find lines that aren't there

Static analysis reads your source code and matches it against known-bad patterns. What it cannot do is find a bug that is the absence of a line.

Look at that reports query again:

const data = await db.query(
  'SELECT * FROM orders WHERE id = $1',
  [orderId]
);
return Response.json(data.rows[0]);

A SAST scanner sees a parameterized query. No SQL injection. No hardcoded secret. No dangerous call. It reads as clean.

But there's no userId check. Any authenticated user can request any order by changing the ID in the URL. Swap orderId=100 to orderId=99 and you get someone else's data. That's IDOR — Insecure Direct Object Reference — and it's the most common class of bug in AI-generated APIs.

IDOR isn't broken code. It's absent code. SAST can only match patterns that exist. It cannot grep for a line that was never written.

This is the core structural limitation. xploitscan's "300 apps" dataset found what the source code showed them. What it couldn't find was what the running apps would return to a request they weren't supposed to accept.

What each tool sees

Three runtime tests — copy these and run them

No tooling required. Swap in your own URL.

Test 1: Forged Stripe webhook (no signature)

curl -X POST https://yourapp.com/api/webhooks/stripe \
  -H "Content-Type: application/json" \
  -d '{
    "type": "payment_intent.succeeded",
    "data": {
      "object": {
        "amount": 9900,
        "metadata": { "user_id": "your-user-id" }
      }
    }
  }'

If the endpoint returns 200 and processes the event, your webhook is forgeable. No Stripe involved. Just a JSON blob you typed. The fix is stripe.webhooks.constructEvent — six lines that verify the signature before you act on anything.

Test 2: IDOR — change the ID in the URL

# Log in as yourself. Note a resource ID you own — say, order 100.
# Try order 99.
curl https://yourapp.com/api/orders/99 \
  -H "Authorization: Bearer YOUR_SESSION_TOKEN"

If you get back data that belongs to a different user, the ownership check is missing. Authentication passed. But authentication only proves who you are — it doesn't prove you own the thing you're asking for.

Test 3: Supabase anon query

curl 'https://yourproject.supabase.co/rest/v1/orders?select=*' \
  -H "apikey: YOUR_ANON_KEY" \
  -H "Authorization: Bearer YOUR_ANON_KEY"

The anon key is public by design — it's in your frontend JavaScript. What matters is what your RLS policies say about it. If this returns rows without an authenticated user session, RLS is off on that table. Any visitor to your app has read access to every order in your database. No code analysis will catch this. The database decides at query time.

Checklists and templates plus runtime testing equals full coverage

Igor's list is right about what to look for. Secure boilerplates are right about starting clean. SAST is right about flagging known-bad patterns. None of them replace sending a real request to your running app and watching what comes back.

A checklist is a statement of intent. "I intended to put auth middleware here." A runtime test is a statement of fact. "This endpoint accepted the request or it didn't."

The Abrahub breach didn't happen because Tiago had advanced tools. It happened because the routes answered every request, regardless of who sent them. A 17-item checklist that says "add auth middleware" can't tell you whether the route you added last Thursday actually has it.

Here's your action list:

1. Call your most recently added API endpoint without authentication. Open an incognito window and hit it directly with no Authorization header. If it returns data, the check is missing — regardless of what the checklist says you did.

2. Change a resource ID in a URL by one. If your app has /api/orders/42, try /api/orders/41 while logged in as yourself. If you see someone else's data, that's IDOR. Add AND user_id = $2 to the query and pass the authenticated user's ID.

3. Run the Supabase anon-key query against every table that holds user data. For any table that returns rows, check whether that's intentional. If not, go to Authentication → Policies and add an RLS rule that requires auth.uid() = user_id.

Checklists help you build the right thing. Runtime tests tell you whether you built it. Do both.

The Abrahub breach and FastroAI checklist are from Igor Barroso's April 2026 r/vibecoding post. The Base44 authentication bypass is documented by Wiz Research (July 2025).