Cache LLM Answers with Python and SQLite

Zusammenfassung

Pixel shares a practical guide to reducing LLM calls in small-business FAQ bots using Python and SQLite. It covers normalizing questions, hashing cache keys, refreshing answers weekly, and avoiding sensitive customer data—helping teams control AI costs without adding unnecessary complexity.

DEV Community

Cache Repeated Answers to Cut LLM Calls in a Small-Business FAQ Bot (Python + SQLite)

Small businesses get the same ten questions all day: opening hours, delivery areas, how to book. Sending each one to a language model is wasted money. A small cache is one of the simplest ways to control AI automation costs for a small business. Normalise, then hash import hashlib, re, sqlite3, time

db = sqlite3.connect("answers.db")
db.execute("""CREATE TABLE IF NOT EXISTS cache(
key TEXT PRIMARY KEY, question TEXT, answer TEXT, created REAL, hits INTEGER DEFAULT 0)""")

def normalise(q: str) -> str:
q = q.lower()
q = re.sub(r"[^a-z0-9 ]+", " ", q)
return " ".join(q.split())

def key(q: str) -> str:
return hashlib.sha256(normalise(q).encode()).hexdigest()
Read through the cache MAX_AGE = 7 * 24 * 3600 # refresh weekly so answers do not go stale

def answer(q, call_model):
k = key(q)
row = db.execute("SELECT answer, created FROM cache WHERE key=?", (k,)).fetchone()
if row and time.time() - row[1] < MAX_AGE:
db.execute("UPDATE cache SET hits = hits + 1 WHERE key=?", (k,))
db.commit()
return row[0], "cache"
a = call_model(q)
db.execute("REPLACE INTO cache(key, question, answer, created) VALUES (?,?,?,?)",
(k, q, a, time.time()))
db.commit()
return a, "model"
Rules that keep it safe Cache only general questions. Never cache answers that include a customer's name, order or account details. Expire entries when prices, hours or policies change (clear the table on any website update). Log cache versus model so you can see what the cache actually saves. Going further Exact matching catches more than you expect once text is normalised. Fuzzy or embedding matching adds complexity; measure first. Why we often prefer small custom pieces like this over packaged bots: why small custom pieces often beat packaged bots. Disclosure: I co-founded NxFlowAI, an automation agency.
Cache LLM Answers with Python and SQLite
Powered by Readybot.io

Das Neueste von Pixel

Incident Response: Look Beyond the Red Alert

Pixel’s DEV Community announcement explores how teams can fixate on one failing service while overlooking the real cause of an outage, such as a full …

Sach: AI Fact-Checks WhatsApp Forwards in India

Pixel’s announcement introduces Sach, a tool that checks forwarded messages in English, Hindi, Telugu, and 14 other Indian languages against official sources. It gives cited …

Java 25 Removes SecurityManager: What to Fix

Pixel’s announcement explains how Java 24’s JEP 486 permanently disabled the SecurityManager, breaking test workarounds that intercept System.exit. It recommends returning exit codes or throwing …