DEV Community
Crypto-shredding Agent Sessions: Deleting What Your Backups Remember
Deleting data is one of the first things any compliance work asks of you. GDPR calls it storage limitation, ISO 27001 has a control just for it (8.10, information deletion), and most B2B contracts say some version of the same thing: data has an expiry date, and when a customer leaves, everything that belongs to them goes with them. Deleting their rows from the live database is the easy part. What I never had a good answer for was everything else, because if a customer is gone but their data is still sitting in thirty days of snapshots, there's no way to open a snapshot and take one tenant out of it, since the whole point of a backup is that nobody can change it. The answer I ended up with is to stop trying to delete the data and make it unreadable instead, by destroying the key it was encrypted with. To check that it really works, I built a small demo for this post with two made-up tenants, acme and globex, took a real backup of their database and then deleted globex. When I restore that old backup now, every globex session comes back as bytes nobody can read, including the password one of their tools had read from a .env file, while acme's sessions are still there in clear. This post is how I got there, and the questions I had to answer along the way. What the frameworks fixed The same question came back when I started looking at agent sessions. Last month I wrote that the frameworks store every turn of a conversation, tool output included, and that giving us a way to delete it was their job. Since then part of that has moved. My fix for ADK's SQLite delete was merged on November 5, so when ADK deletes a session now its events go with it. On Genkit, a maintainer answered my issue that you can already delete through the store, but that they had no objection to an API for it, and since then two PRs that add Agent.DeleteSession have been waiting for review, one for JS and one for Go. What caught my eye in those PRs is what they leave out on purpose. The author writes that "Retention/TTL isn't in here" and that "Traces, logs and streams live outside the store and aren't touched", and I think they're right to draw the line there, because a framework can only delete what's in its own store. How long your data lives, and the copies your infrastructure makes of it, were never something a framework could reach. So I owe last month's post a correction: deleting a session is the framework's job, and the frameworks are doing it, while retention and copies belong to whoever runs the infrastructure. Where a deleted session still lives If I follow a single agent session through a normal production setup on Postgres, the live table is only the first place it lands. Cloud SQL takes a daily backup of it, or, on Kubernetes, CloudNativePG ships base backups and the WAL archive to a bucket through Barman, and point-in-time recovery exists precisely so you can go back to a moment when that row was still there. Then there's the export somebody set up to BigQuery for analytics, and the dump a colleague took before a risky migration and forgot in a bucket. A DELETE reaches the first of these and none of the others, and some of them can't be modified even if you go looking for them. Encrypt per tenant, then destroy the key Crypto-shredding solves this by turning it around. If the content is never stored in clear, then every copy of the database, backups and dumps and exports included, only ever carries ciphertext. Each tenant gets its own key, everything they write is encrypted with it before it reaches Postgres, and when the tenant has to go I don't chase the copies at all: I destroy the key, and every copy becomes unreadable at the same moment, wherever it ended up. The keys live in Cloud KMS, which never hands them out. Calling KMS for every single message would be slow and would cost money on every read, so I use envelope encryption. Each tenant has a data key (a DEK) that does the actual work locally with AES-256-GCM, and KMS only ever encryp...
Powered by Readybot.io
External Link
Platform
You are about to visit:
https://example.comThis will open in a new tab. Make sure you trust this link.