Stop Paying for AI Renders in Test Orders

Resumen

Pixel’s announcement explains how internal test orders were accidentally triggering paid AI video renders costing just over $1 each. It details a server-side dry-run guard that keeps tests useful without wasting money, while preserving approved runs for measuring real render times.

DEV Community

Our test orders were paying for real AI renders, so we made them dry runs

We run a small pay-per-render web app: you upload a selfie, pay once, and a few minutes later you get a 15-second meme video with your face in it. It is one Cloudflare Worker, Stripe Checkout, R2 for files, and a paid video-generation job per order. That last part is what this post is about. Every order starts two paid jobs (a main pass and a short segment pass that gets spliced in at the end). Together they cost a little over one US dollar per order. That is fine for a customer order. It is not fine for the orders we create ourselves. Three kinds of orders Once the site was live, orders came from three places: Kind Who creates it Pays? Starts a render? real customers via Stripe Checkout yes yes qa a headless browser check that opens checkout every day and stops there no no (stays awaiting_payment) test us, with an admin token that skips Stripe to test the whole flow no yes The qa orders were harmless: they never get past checkout, and they carry an @example.com address so reports can filter them out. The test orders were the problem. In launch week they were genuinely useful. Our honest "how long does it take" numbers come from them: 5:11 and 5:20 for one face, 6:57 for all five seats. But the admin path called the same start() as a paid order, so every "let's just check the order page after this deploy" also bought a real render. With scheduled jobs and post-deploy checks that each like to "test the whole flow", that adds up quietly. The guard The fix sits on the server, in the two places that start a render for an order. Simplified: // Internal test orders never start a paid render unless a human approved that one run.
const paidRunApproved = (req) => req.headers.get('x-allow-paid-run') === 'approved-by-owner';

async function dryRun(env, tpl, order) {
order.dry = true; // reports and the order page can tell it apart
order.status = 'done'; // same terminal state as a real order
order.video = tpl.exampleVideo // the template's published example clip
order.error = order.video ? null : 'dry run (no example video)';
await putOrder(env, order);
}

// 1) new admin test order (skips Stripe)
if (isAdmin) {
order.status = 'paid'; order.test = true;
if (!paidRunApproved(req)) { await dryRun(env, tpl, order); return json({ url: /o/${order.id}, dry: true }); }
await start(env, tpl, order);
}

// 2) re-using a credit from a multi-video pack
if (order.test && !paidRunApproved(req)) { await dryRun(env, tpl, order); return json({ url: /o/${order.id}, dry: true }); }
Thirteen lines in the real diff. A few decisions behind them: Guard where the money is spent, not where the tests live. We could have told every test script to "please don't render". That works until the next script, or the next person, forgets. The Worker is the only place that can start a render, so the rule lives there and every caller gets it for free. Make the expensive path opt-in, and make the opt-in loud. A boolean like ?real=1 is easy to paste into a script and forget. A header whose value reads like a sentence (approved-by-owner) is hard to add by accident and easy to grep for in logs and code review. Our rule is that it is only sent after the owner says yes to that single run, for example when the generation pipeline itself changed. Keep the state machine identical. A dry order ends in done with a playable video URL, exactly like a real one. So the order page, the video player, the download button and the video_ready analytics event still run end to end. Only the render is skipped. The video is the example clip already shown on the homepage, so nobody can mistake it for a fresh result. Flag it. dry: true on the order lets the daily order report count these separately from real and paid orders, so a dry run can never inflate the numbers we judge the site by. Find every entry point. The first version only guarded new test orders. The second path, re-using a credit from a 3- or 5-video pack, also starts a render, and a test ...
Stop Paying for AI Renders in Test Orders
Powered by Readybot.io

Lo último de Pixel

Sandblasting Grit Guide: 120 vs 220 vs 320

Pixel’s announcement explains how 120, 220, and 320 grit affect aluminium’s matte finish and reveal die-cast porosity. It also covers how pressure, blast distance, and …

NVIDIA Cosmos 3 Edge: Physical AI on Jetson

Pixel shares the first article in CAVEDU's Cosmos series, introducing NVIDIA Cosmos 3 Edge, an approximately 4-billion-parameter world model for Jetson devices. It explores multimodal …

Laravel Semantic Search with OpenAI and Pinecone

Pixel shares a practical guide to building semantic search in Laravel using OpenAI embeddings and Pinecone. It covers 1536-dimensional vectors, cosine similarity, index setup, environment …