Write Test Plans Before AI Agents Code
Summary
Pixel’s announcement explores why teams should define and approve tests before AI coding agents implement software. It highlights a reported workload of up to 6.5 billion tokens and explains how tests can serve as review gates when code output outpaces human review.
DEV Community
Write the test plan before your coding agent writes code
When a coding agent writes code faster than you can read it, reading every line stops working as review, and approved tests have to carry most of that load. So define the tests before the agent writes any implementation, read them yourself, and require every commit to pass them. Your attention moves from reading diffs to deciding what "done" means. Anush Elangovan is Corporate VP of AI Software at AMD. Before we recorded his April 2026 episode, he told me he was generating 250,000 lines of code a week with AI agents. Lines of code isn't a measure of success, but it shows the throughput problem: no human reviews that volume line by line. By the time we talked, he said a week had earlier come to about 4.2 billion tokens, a check that morning showed 6.5 billion consumed, and he could no longer keep track of lines. Why write tests before the agent writes code? Because the tests are the only fixed point while the implementation churns. Anush held this view before agents existed: Yeah, so my philosophy is, even before agentic coding, I used to advise people and friends and colleagues, 80% of your time should be spent on tests and writing tests. Even before you write your first line of code, you should have a test framework. Anush Elangovan, Corporate VP of AI Software at AMD, on Chain of Thought ep 56 250,000 Lines of Code/Week: In… - Chain of Thought | AI Agents, Infrastructure & Engineering - Apple Podcasts Podcast Episode · Chain of Thought | AI Agents, Infrastructure & Engineering · April 22 · 51m podcasts.apple.com With agents, the tests also do the job a reviewer used to do. When he doesn't like something about the output, he turns it into a stronger gate instead of arguing with the agent: Yep, testing is the code review and testing is kind of the gate of your sandbox. If you have a complaint about anything in the box, fix the sandbox, right? Anush Elangovan, Corporate VP of AI Software at AMD, on Chain of Thought ep 56 That includes non-functional requirements. For his virtual GPU simulator he set a performance metric, each dispatch executing in 50 microseconds of simulator time. When a change fails that test, the agent has to find another approach. How do you check the tests an agent proposes? Read them before you approve them. Anush's example: an agent wanted to implement GPU demand paging in one go. He stopped it and told it to write 100 tests covering combinations and corner cases first. Some of the tests were repetitive, with parameters changed but not the thing being tested. After reading them, he was satisfied that if the set passed, the feature was good. Things to look for in an agent-written test list: The same assertion with only literal values changed. Missing negative cases, or platforms you named in the plan but no test covers. Mocks that remove the behavior you care about, which make passing cheap. His reason for the scrutiny is that agents optimize for the gate: they try to just reward hack their way towards that test. So all you have to think of is like, they cannot reward hack that, right? Like you're putting up gates that they have to pass through. Anush Elangovan, Corporate VP of AI Software at AMD, on Chain of Thought ep 56 A minimal sketch to adapt. Test cases are plain records with a content hash, so an agent can't keep an approved test ID while weakening its assertions. def near_duplicates(cases):
"""cases: list of dicts with 'id', 'target' and 'params' keys."""
seen, dups = {}, []
for c in cases:
# same target, same parameter names: probably only values changed
key = (c["target"], tuple(sorted(c["params"])))
if key in seen:
dups.append((seen[key], c["id"]))
else:
seen[key] = c["id"]
return dups
def may_implement(proposed, approved):
"""approved: {test_id: sha256 of the test's source}, frozen at human approval."""
if not approved:
return False, "a human has not approved the test list yet"
current = {c["id"]:...

Powered by Readybot.io