Skip to content
S3 Viewer

How to upload large files to S3 (multipart upload)

By Aleem Rehmtulla, May Inc.

TL;DR

S3 Viewer streams large files through its server into multipart uploads without bucket CORS setup, on AWS S3, Cloudflare R2, Backblaze B2, and other S3-compatible endpoints alike. Uploads protect existing files: AWS S3 and R2 enforce the condition directly, other endpoints get a HEAD check performed by S3 Viewer immediately before the write. Replacements require confirmation and readable original permissions and settings. New files may retry network failures; uncertain replacements require fresh review. The default app upload limit is 5 GiB and can be configured when self-hosting.

Steps

Step-by-step.

  1. 01

    In S3 Viewer: Upload → Upload files, or drag files onto the list

    Open the Upload menu and choose Upload files (or Upload folder), or drag files straight onto the file list. Your browser sends each file through the S3 Viewer server into the bucket without writing it to disk. This works on AWS S3, R2, B2, and other S3-compatible endpoints. No bucket CORS policy is needed. Existing files require confirmation and readable settings before replacement; some MinIO setups cannot provide those settings, in which case replacement is refused rather than guessed at.
  2. 02

    Review uncertain results before replacing a file

    New-file uploads can retry a lost browser connection while still protecting existing objects. Replacements are not automatically retried after an uncertain result: refresh and review first. Interrupted browser multipart uploads retain their parts for resumption. Failed single-request uploads and cancellations attempt to remove unfinished parts, but cleanup can fail.
  3. 03

    AWS CLI: aws s3 cp

    The CLI uses multipart automatically for large files. No flags needed for normal files; tune multipart_chunksize in your AWS config for very large ones.
    aws s3 cp big.zip s3://bucket/big.zip
  4. 04

    AWS SDK: Upload from @aws-sdk/lib-storage

    Handles part splitting, parallelism, and retries for you. The right pattern for streaming uploads from a backend.
    import { Upload } from '@aws-sdk/lib-storage';

const u = new Upload({
  client: s3,
  params: { Bucket: 'b', Key: 'big.zip', Body: stream },
  queueSize: 4,    // parts in flight
  partSize: 8 * 1024 * 1024,
});
await u.done();
  5. 05

    Tune part size for very large files

    Choose a partSize that stays within AWS S3's 10,000-part limit. For a fixed part size, partSize × 10,000 is the part-count ceiling; the effective upload limit is the lowest applicable part-count, operator, application, or provider limit.
  6. 06

    Verify with HeadObject

    After the upload completes, confirm the size and ETag match what you intended to upload.
    aws s3api head-object --bucket b --key big.zip
  7. 07

    Set a lifecycle rule to abort failed uploads

    Incomplete multipart uploads sit in the bucket and you pay for them until aborted. Add an S3 lifecycle rule to auto-abort uploads older than a few days — AWS recommends this in the well-architected framework.

Under the hood

What's actually happening.

S3's PutObject sends an object in a single request that cannot be resumed — a network interruption restarts the upload from byte zero. Multipart upload (CreateMultipartUpload → many UploadPart → CompleteMultipartUpload) splits the file into parts (at least 5 MiB each except the final part, up to 10,000 parts on AWS S3), uploads them in parallel, and lets you retry only the failed parts. S3 Viewer uses multipart upload for large files and a single upload for small files. Both travel through the server without writing file contents to disk or requiring bucket CORS setup, on every connected provider. AWS S3 and R2 enforce the app's overwrite protections directly; B2 and other endpoints get the same check performed by S3 Viewer with a HEAD request immediately before the write. Replacement requires readable original permissions and settings, so unsupported encryption or unavailable settings stop it before the existing file is changed.

FAQ

Common questions.

What's the max file size I can upload to S3?

S3 Viewer's default upload limit is 5 GiB per file. A self-hosted instance can change MAX_UPLOAD_BYTES, subject to the app's multipart and provider limits. Large files stream in parts on every connected provider, including B2.

What is multipart upload in S3?

A three-step S3 API flow — CreateMultipartUpload, UploadPart (in parallel, at least 5 MiB per part except the final part, up to 10,000 parts on AWS S3), and CompleteMultipartUpload. It splits a large file into pieces, uploads them in parallel, and lets you retry only the failed parts. It is useful for large files and uploads that need retries.

What happens if my upload fails halfway?

S3 Viewer may retry a new-file upload after a browser network error, while keeping protection against overwriting an existing file. An uncertain replacement requires fresh review. Interrupted browser multipart uploads retain their parts and saved session so you can resume them. Failed single-request uploads and cancellations attempt cleanup; a bucket lifecycle rule can remove abandoned parts when cleanup fails or an upload is never resumed.

Do I get charged for failed multipart uploads?

Yes — incomplete multipart uploads are stored and billed at standard storage rates until you abort them. Set an S3 lifecycle rule to auto-abort incomplete uploads after a few days.

Does Cloudflare R2 support multipart upload?

Yes. S3 Viewer supports multipart uploads to R2 and protects against overwriting an unreviewed file. Replacements require readable original settings, and R2 object rename and deletion both work the same way they do on AWS S3.

How do I upload large files to S3 from a browser safely?

Keep the credential on a server. S3 Viewer sends smaller files in a single API request to the server's streaming uploader. For larger files, the browser coordinates multipart uploads and sends each part through the API into the bucket. Neither path exposes credentials or requires bucket CORS changes. In your own app, presigned URLs per part are another option, but direct browser requests to S3 need a CORS rule allowing your origin.

Use S3 Viewer for this

Skip the CLI. Try it in the browser.

S3 Viewer turns the steps above into a single click. Open source, self-hostable, free.

Open appStar on GitHub