AWS S3 TransferManager multipart upload time difference in File upload vs InputStream

[fykidwai]

Is there any difference in performance when using InputStream instead of file in uploading to S3 using TransferManager multipart upload?

[debora-ito]

We don't have benchmarks on the difference in performance for TransferManager v1, sorry.

We do, though, have some benchmarks comparing performances of TransferManager v1 and TransferManager v2 and v2 seems to have better results due to the underlying Common Runtime HTTP client - some references: https://aws.amazon.com/blogs/developer/introducing-aws-common-runtime-http-client-in-the-aws-sdk-for-java-2-x/
https://aws.amazon.com/blogs/developer/introducing-amazon-s3-transfer-manager-in-the-aws-sdk-for-java-2-x/

If you're writing a new application consider using the Java SDK v2, but please note that TransferManager v2 is still in Developer Preview (as of June 2022) and we don't recommend using in production environments yet. We are actively working on the GA release.

[fykidwai]

It is a new application but since S3TransferManager v2 is still in preview and I only need to upload files to S3 so going for aws-sdk-java v1 for now.. BTW when can we expect the GA release for that..

What I have noticed with transfer Manager v1 is that InputStream is always slightly slower than using a file..I couldn't find anything about the benchmark or performance difference, hence the question..!

[fykidwai]

Found something..
There is this call

aws-sdk-java/aws-java-sdk-s3/src/main/java/com/amazonaws/services/s3/transfer/internal/UploadCallable.java

Line 221 in 08aa338

if (TransferManagerUtils.isUploadParallelizable(origReq, isUsingEncryption)) {

which basically returns true only for files and not for InputStream

aws-sdk-java/aws-java-sdk-s3/src/main/java/com/amazonaws/services/s3/transfer/internal/TransferManagerUtils.java

Lines 75 to 82 in 08aa338

	public static boolean isUploadParallelizable(final PutObjectRequest putObjectRequest, final boolean isUsingEncryption) {
	// Each uploaded part in an encrypted upload depends on the encryption context
	// from the previous upload, so we cannot parallelize encrypted upload parts.
	if (isUsingEncryption) return false;
	// Otherwise, if there's a file, we can process the uploads concurrently.
	return (getRequestFile(putObjectRequest) != null);
	}

Probably this is the reason why upload using InputStream is slower..