S3 Multipart Uploads with Presigned URLs and CDK

DEV Community

S3 multipart upload with presigned URLs and AWS CDK

An S3 multipart upload is the answer as soon as files get too big for a single request. In my article on the S3 presigned URL I showed how the browser pushes files straight into a bucket without them passing through the Lambda function. That works great for profile pictures and PDFs. With a 3 GB audiobook, a video recording or a data export, things look different: a single PUT can be at most 5 GB, and if the connection drops at 90 percent, everything starts over. This topic keeps me busy because most guides stop exactly where it gets tricky. They show how to sign one URL per part and leave open who actually controls the file size, what happens to half-finished uploads, and why the browser suddenly can’t see an ETag. So here we build the complete pattern with AWS CDK and TypeScript: four small endpoints, a DynamoDB table, a lifecycle rule, and a browser client that uploads parts in parallel and retries failures on its own. In short With an S3 multipart upload, the client splits a large file into parts, uploads each part through its own presigned URL straight to S3, and your backend assembles the parts at the end. The key insight: if the server signs the size of every part and counts the parts in S3 itself before completing, the client can neither cheat nor forget anything. The biggest pitfall is incomplete uploads that sit invisibly in the bucket and cost money until a lifecycle rule removes them. What you will learn in this article When you need an S3 multipart upload and which limits S3 sets How to calculate a sensible part size and part count How Lambda signs fixed-size part URLs and completes the upload safely How a browser client uploads parts in parallel and retries errors Which errors are common and how a lifecycle rule prevents costs When do you need an S3 multipart upload? AWS recommends multipart uploads from around 100 MB. That’s a rule of thumb, not a hard limit. From this size on, the benefits become obvious. A failed part is retried on its own, several parts run in parallel and make better use of the bandwidth, and you can resume an upload later because S3 remembers every finished part. From 5 GB you no longer have a choice anyway, since a single PUT can’t be any larger. The limits you have to plan for are manageable. These values come from the S3 documentation (as of November 2026): Limit Value Largest possible object 48.8 TiB Parts per upload up to 10,000, numbered 1 to 10,000 Size of a part 5 MiB to 5 GiB, only the last part may be smaller Parts per ListParts response at most 1,000 Single PUT without multipart at most 5 GB For small files, the simple approach with a single presigned URL remains the better choice. It needs less code and fewer requests. In practice I would draw the line at 100 MB and stick with the simple PUT below that. How an S3 multipart upload works The flow consists of three S3 calls. With CreateMultipartUpload you register the upload and get an upload ID back. After that, you upload any number of parts with UploadPart, each with a part number. For every part, S3 returns an ETag, a kind of fingerprint of that part. Finally, CompleteMultipartUpload sends the list of all part numbers with their ETags, and only then does S3 build an object from them. Until that point, there is no file in the bucket that anyone could download. The architecture: Lambda only manages plan and status, the bytes go straight from the browser to S3. Here’s the trick with presigned URLs: the parts don’t have to go through your backend. The Lambda function signs a separate URL for each part, and the browser sends the bytes directly to S3. A presigned URL is a link that allows one specific S3 action for a short time, without the client needing AWS credentials of its own. Your function only handles a few small JSON requests while gigabytes flow past it. One quirk you should know before you start: parts of an unfinished upload stay in the bucket until someone completes or aborts the upload. You pay for that storage, but...

Powered by Readybot.io