GDPR-Safe Private File Uploads for US and EU Apps
Résumé
Pixel shares an architecture for GDPR-conscious private file workflows using private object storage and short-lived signed upload and download URLs. It explains how database states and deletion deadlines help ensure files, extracted text, vector indexes, and other copies are removed on time.
DEV Community
Private File Upload Architecture for GDPR: Signed Download Retention in US/EU Apps
Short answer: For a beginner building a private file workflow, keep object storage private, let the application issue short-lived signed upload and download URLs, and make the deletion deadline a database state transition rather than a property hidden in a bucket. That architecture gives a browser a narrow, temporary capability instead of a storage credential. The application remains responsible for identity, authorization, retention, and deletion. Object storage handles bytes. This split is especially useful when a document will later enter a Python RAG or agent pipeline: ingestion can be evaluated independently from upload mechanics and prompt cost. The interesting failure is not usually a failed upload. It is a file that uploaded successfully, lost its database owner, and then survived past the deadline because nobody could prove who should delete it. Start with the deletion contract, not the bucket The contract is simple: every stored document has an owner, a region, a purpose, a current state, and a deletion deadline. A download is allowed only while the record says it is allowed. A deletion request must reach the object, extracted text, vector index, cache, thumbnail, and any other copy that the product created. That is the contract. The bucket is only one participant. This framing is useful for a Python team moving from notebook-to-prod because it keeps retrieval evaluation honest: a document cannot remain eligible for search merely because a background cleanup worker forgot one table. For a concrete timeline, imagine an upload intent at 09:00, a 30-minute signed upload capability, and a deletion deadline at 09:00 the next day. The browser may finish at 09:04, while the parser starts at 09:06 and the user revokes access at 10:00. At 09:00 on the following day, the deletion worker must stop new downloads immediately, remove the object, enqueue index cleanup, and leave an audit record even if one downstream operation needs a retry. A lifecycle rule may eventually remove bytes, but it cannot explain which authorization decision allowed a retrieval at 08:59 or whether the vector index was also cleared. That evidence belongs in the application record and its event trail. What breaks when a signed URL becomes the whole design? The first failure mode is confusing possession with permission. A signed URL is a temporary capability; it does not know that a user changed tenant, that a file was revoked, or that the retention deadline passed. The application must make those decisions before issuing it. The second is treating a browser callback as proof that the object is valid. An upload can be incomplete, oversized, mislabeled, or a valid container with an unsafe payload. Use an opaque server-generated object key, check the stored object's metadata and size, validate its type, and apply scanning rules appropriate to the product. OWASP recommends an extension allow-list, file-type validation rather than trust in a supplied Content-Type, size limits, restricted upload permissions, and malware scanning where needed. The third is deleting only the original bytes. For a RAG or agent feature, extracted text and embeddings are copies with their own retention behavior. If the source disappears but its chunk remains retrievable, the system has not fulfilled the user's deletion expectation. Don't hide that work behind a single “delete” button. How can beginners protect private files with signed upload and download URLs in object storage? Use two paths with different jobs. An authenticated application endpoint creates an upload intent. It checks the user and tenant, generates an opaque object key, records the expected size and media type, and returns a short-lived signed upload URL. The browser sends bytes directly to the private bucket. A completion request then verifies the object and moves the row from pending to ready. Downloads repeat the authorization step. The application looks up the file by its database ID, confirms that the requ...

Powered by Readybot.io