You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
The multipart upload API is designed to improve the upload experience for larger objects. You can upload an object in parts. These object parts can be uploaded independently, in any order, and in parallel. You can use a multipart upload for objects from 5 MB to 5 TB in size.
Tasks
The following tasks will need to be carried out:
First create a multipart upload with CreateMultipartUploadRequest and get the upload id.
Upload all the different parts of the object with the help of UploadPartRequest and CompletedPart.
Finally call completeMultipartUpload operation with CompletedMultipartUpload request to tell S3 to merge all uploaded parts and finish the multipart operation.
ptirador
changed the title
Use Multipart upload to upload files larger than 5 GB.
Use Multipart upload API to upload files larger than 5 GB.
Oct 30, 2020
carlspring
changed the title
Use Multipart upload API to upload files larger than 5 GB.
Use Multipart upload API to upload files larger than 5 GB
Nov 23, 2020
There are 2 kinds of APIs to perform this multi-part operation: high-level API and low-level API:
We should use the low-level API when we need to pause and resume multipart uploads, vary part sizes during the upload, or do not know the size of the upload data in advance. When we don't have these requirements, we should use the high-level API.
If we upload files in the S3 bucket using TransferManager(high-level API), which it's easier to integrate, we can upload an InputStream or File, and also we can use multipart upload in TransferManager by configuring MultipartUploadThreshold in TransferManager client. But for this client we need to provide the content-length of the file size in the PutObjectRequest before uploading the file to the S3 bucket. And when dealing with InputStream of a large file we might not get the content-length in the HTTP Response, so in that case, we cannot use TransferManager (high-level API).
So, in summary, I think low-level API seems more flexible and manageable, although it may be not so easy to integrate, but in the long run it is worth it.
Thanks for having a preliminary look, @ptirador! I don't know, if you've seen the existing pull requests in the Upplication upstream and whether there's anything we could use as inspiration from there?
Task Description
The multipart upload API is designed to improve the upload experience for larger objects. You can upload an object in parts. These object parts can be uploaded independently, in any order, and in parallel. You can use a multipart upload for objects from 5 MB to 5 TB in size.
Tasks
The following tasks will need to be carried out:
CreateMultipartUploadRequest
and get the upload id.UploadPartRequest
andCompletedPart
.completeMultipartUpload
operation withCompletedMultipartUpload
request to tell S3 to merge all uploaded parts and finish the multipart operation.Task Relationships
This task:
Useful Links
Help
The text was updated successfully, but these errors were encountered: