Cache LLM Answers with Python and SQLite
Zusammenfassung
Pixel shares a practical guide to reducing LLM calls in small-business FAQ bots using Python and SQLite. It covers normalizing questions, hashing cache keys, refreshing answers weekly, and avoiding sensitive customer data—helping teams control AI costs without adding unnecessary complexity.
DEV Community
Cache Repeated Answers to Cut LLM Calls in a Small-Business FAQ Bot (Python + SQLite)
Small businesses get the same ten questions all day: opening hours, delivery areas, how to book. Sending each one to a language model is wasted money. A small cache is one of the simplest ways to control AI automation costs for a small business. Normalise, then hash import hashlib, re, sqlite3, time
db = sqlite3.connect("answers.db")
db.execute("""CREATE TABLE IF NOT EXISTS cache(
key TEXT PRIMARY KEY, question TEXT, answer TEXT, created REAL, hits INTEGER DEFAULT 0)""")
def normalise(q: str) -> str:
q = q.lower()
q = re.sub(r"[^a-z0-9 ]+", " ", q)
return " ".join(q.split())
def key(q: str) -> str:
return hashlib.sha256(normalise(q).encode()).hexdigest()
Read through the cache MAX_AGE = 7 * 24 * 3600 # refresh weekly so answers do not go stale
def answer(q, call_model):
k = key(q)
row = db.execute("SELECT answer, created FROM cache WHERE key=?", (k,)).fetchone()
if row and time.time() - row[1] < MAX_AGE:
db.execute("UPDATE cache SET hits = hits + 1 WHERE key=?", (k,))
db.commit()
return row[0], "cache"
a = call_model(q)
db.execute("REPLACE INTO cache(key, question, answer, created) VALUES (?,?,?,?)",
(k, q, a, time.time()))
db.commit()
return a, "model"
Rules that keep it safe Cache only general questions. Never cache answers that include a customer's name, order or account details. Expire entries when prices, hours or policies change (clear the table on any website update). Log cache versus model so you can see what the cache actually saves. Going further Exact matching catches more than you expect once text is normalised. Fuzzy or embedding matching adds complexity; measure first. Why we often prefer small custom pieces like this over packaged bots: why small custom pieces often beat packaged bots. Disclosure: I co-founded NxFlowAI, an automation agency.

Powered by Readybot.io