pytest-queryguard: Detecting N+1 Queries Without Noise

Краткое содержание

Pixel shares a deep dive into pytest-queryguard, a pytest plugin that detects N+1 database queries such as repeatedly lazy-loading Order.customer. Testing it against Faraday’s 2,112-test suite produced 551 failures, mostly noise from a user factory that runs only in tests. The findings highlight why practical N+1 detection needs to distinguish production issues from test-only behavior.

DEV Community

My N+1 test plugin failed 551 tests on someone else's project. Nearly all of them were noise.

I maintain pytest-queryguard, a pytest plugin that fails a test when the code it runs sends one query per row. The problem it goes after is old: a test database holds a handful of rows, so a loop that lazy-loads a relationship costs a few extra queries and nobody notices. In production the same loop runs over hundreds of rows. This test passes: def test_list_orders(session):
assert len(list_orders(session)) == 50
and list_orders looks fine in review: def list_orders(session: Session) -> list[dict[str, object]]:
rows = []
for order in session.scalars(select(Order)).all():
rows.append({"id": order.id, "customer": order.customer.name})
return rows
With the plugin installed, the same test fails: queryguard: N+1 detected

Order.customer was lazy-loaded 50 times
first at app/orders.py:10 in list_orders
called from tests/test_orders.py:7 in test_list_orders
parents loaded at app/orders.py:9 in list_orders
kind: relationship-load label: Order.customer
docs: https://maciejziel.github.io/pytest-queryguard/how-detection-works.html#relationship-load
sql: SELECT customer.id, customer.name FROM customer WHERE customer.id = ?
fix: load it with the parent query, e.g.
select(Order).options(joinedload(Order.customer))
Nothing to import or configure. Add the joinedload and it passes. Then I ran it on a suite I didn't write My own corpus of scenarios said the rules were accurate. Then I ran the plugin on Faraday, a Flask-SQLAlchemy app with 2112 tests on PostgreSQL. With the defaults of the code I had at the time, 551 tests failed. I went through them, and nearly all were noise. The biggest group, 422 findings, came from Faraday's user factory: its post_generation hook looks up the role for every user it creates. That is an N+1, technically. It also never runs in production, and fixing it changes nothing for anyone using Faraday. About 480 more came from a statistics helper called on six separate lines of a test. The plugin saw the same counts repeat and called it a loop. The rest were mostly tests sending several requests through one Session that the whole suite shares. A tool that fails a quarter of your suite on the first run gets uninstalled on the first run. So 0.10.0 changed two things. Test code is its own category now If every frame of your code on a statement's stack is test code (the test module, conftest.py, files under tests/, any module that defines a factory_boy or polyfactory factory), its N+1 no longer fails the test. It is counted, and the run ends with a section like this (the example from the docs): ============================ queryguard: test code =============================
422 N+1 findings in test code left out (299 tests): the statements ran only in test modules, conftest.py, factories or helpers under tests/, not in the application.
To see them: pytest -o queryguard_test_code=warn, or set queryguard_test_code = warn (or fail) in the pytest configuration.
Why: https://maciejziel.github.io/pytest-queryguard/how-detection-works.html#test-code
tests/test_accounts.py::test_create_account
tests/test_accounts.py::test_delete_account
tests/test_billing.py::test_invoice_lines
tests/test_billing.py::test_refund
tests/test_orders.py::test_list_orders
and 294 more
One frame of the application on the stack is enough to make the statement the application's, so a loop in your code that a test helper calls is still reported. The rule is by file, which means a function written inside a test module is test code too. If that's how you reproduce bugs, set queryguard_test_code = fail. The second change: every statement now carries its call path, the lines of your code from the statement up to the test. Two calls on separate lines have different paths and count apart; a loop has one path. That stack walk isn't free. In 0.10.0, on my in-memory SQLite micro-benchmark, it added about 3 µs per Core statement and 8 per ORM statement. On the 300-test suite I use for comparisons,...
pytest-queryguard: Detecting N+1 Queries Without Noise
Powered by Readybot.io

Последнее от Pixel

Incident Response: Look Beyond the Red Alert

Pixel’s DEV Community announcement explores how teams can fixate on one failing service while overlooking the real cause of an outage, such as a full …

Sach: AI Fact-Checks WhatsApp Forwards in India

Pixel’s announcement introduces Sach, a tool that checks forwarded messages in English, Hindi, Telugu, and 14 other Indian languages against official sources. It gives cited …

Java 25 Removes SecurityManager: What to Fix

Pixel’s announcement explains how Java 24’s JEP 486 permanently disabled the SecurityManager, breaking test workarounds that intercept System.exit. It recommends returning exit codes or throwing …

Why a Java HPA Got Stuck at 20 Kubernetes Pods

Pixel shares how a Spring Boot pricing service autoscaled to 20 Kubernetes pods and stayed there because the JVM retained committed heap memory after traffic …