Missing Content-Length header on empty payloads with CrtHttpEngine

[abhimanyuchugh]

Describe the bug

S3 API seems to always expect Content-Length header in the http request, however when the http request is generated by the CrtHttpEngine, it explicitly excludes this header for empty payloads. We use empty payloads to upload a placeholder file to S3, but with the CrtHttpEngine, we are not able to do this. This works fine with the default OkHttpEngine which does include the zero Content-Length header for empty payloads.

This behaviour was explicitly added in this commit (ed43245) however the linked PR doesn't match the commit change, so not sure of the context behind the change.

Expected Behavior

Include Content-Length=0 HTTP header for empty payloads in CrtHttpRequests

Current Behavior

Content-Length header is explicitly excluded from the CrtHttpRequest per ed43245

Stack trace:

2023-03-06T15:39:47,840+00:00 [pool-4-thread-1] DEBUG httpTraceMiddleware - sdkRequestId: 71776c3b-e991-4e56-a9cf-ec7417d59de7; service: S3; operation: PutObject; - HttpRequest:
PUT /s3-path?x-id=PutObject
Host: s3.eu-west-1.amazonaws.com
amz-sdk-invocation-id: ...
amz-sdk-request: attempt=1; max=3
User-Agent: aws-sdk-kotlin/0.17.12-beta os/linux/5.4.228-131.415.amzn2.x86_64 lang/kotlin/1.8.10 
x-amz-user-agent: aws-sdk-kotlin/0.17.12-beta
X-Amz-Content-Sha256: ...
X-Amz-Date: 20230306T153947Z
X-Amz-Security-Token: ...
Authorization: ...

2023-03-06T15:39:47,909+00:00 [pool-4-thread-1] DEBUG httpTraceMiddleware - sdkRequestId: 71776c3b-e991-4e56-a9cf-ec7417d59de7; service: S3; operation: PutObject; - HttpResponse:
HTTP 411: Length Required
x-amz-request-id: ...
x-amz-id-2: ...
Content-Type: application/xml
Transfer-Encoding: chunked
Date: Mon, 06 Mar 2023 15:39:47 GMT
Server: AmazonS3
Connection: close
2023-03-06T15:39:47,913+00:00 [pool-4-thread-1] DEBUG Retry - sdkRequestId: 71776c3b-e991-4e56-a9cf-ec7417d59de7; service: S3; operation: PutObject; - request failed with non-retryable error

aws.sdk.kotlin.services.s3.model.S3Exception: You must provide the Content-Length HTTP header.
	at aws.sdk.kotlin.services.s3.transform.PutObjectOperationDeserializerKt.throwPutObjectError(PutObjectOperationDeserializer.kt:66) ~[s3-jvm-0.17.12-beta.jar:?]
	at aws.sdk.kotlin.services.s3.transform.PutObjectOperationDeserializerKt.access$throwPutObjectError(PutObjectOperationDeserializer.kt:1) ~[s3-jvm-0.17.12-beta.jar:?]
	at aws.sdk.kotlin.services.s3.transform.PutObjectOperationDeserializer.deserialize(PutObjectOperationDeserializer.kt:25) ~[s3-jvm-0.17.12-beta.jar:?]
	at aws.smithy.kotlin.runtime.http.operation.SdkOperationExecutionKt$decorate$3.invoke(SdkOperationExecution.kt:92) ~[http-jvm-0.12.13.jar:?]
	at aws.smithy.kotlin.runtime.http.operation.SdkOperationExecutionKt$decorate$3.invoke(SdkOperationExecution.kt:92) ~[http-jvm-0.12.13.jar:?]
	at aws.smithy.kotlin.runtime.http.operation.DeserializeHandler.call(SdkOperationExecution.kt:148) ~[http-jvm-0.12.13.jar:?]
	at aws.smithy.kotlin.runtime.http.operation.DeserializeHandler$call$1.invokeSuspend(SdkOperationExecution.kt) ~[http-jvm-0.12.13.jar:?]
	at kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(ContinuationImpl.kt:33) ~[kotlin-stdlib-1.8.10.jar:1.8.10-release-430(1.8.10)]
	at kotlinx.coroutines.UndispatchedCoroutine.afterResume(CoroutineContext.kt:233) ~[kotlinx-coroutines-core-jvm-1.6.4.jar:?]
	at kotlinx.coroutines.AbstractCoroutine.resumeWith(AbstractCoroutine.kt:102) ~[kotlinx-coroutines-core-jvm-1.6.4.jar:?]
	at kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(ContinuationImpl.kt:46) ~[kotlin-stdlib-1.8.10.jar:1.8.10-release-430(1.8.10)]
	at kotlinx.coroutines.DispatchedTask.run(DispatchedTask.kt:106) ~[kotlinx-coroutines-core-jvm-1.6.4.jar:?]
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136) ~[?:?]
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635) ~[?:?]
	at java.lang.Thread.run(Thread.java:833) ~[?:?]

Steps to Reproduce

S3Client {
    sdkLogMode = SdkLogMode.LogRequest + SdkLogMode.LogResponse
    region = awsRegion
    httpClientEngine = CrtHttpEngine { }
}.use { s3Client ->
    s3Client.putObject(
        PutObjectRequest {
            bucket = s3Bucket
            key = s3Key
        }
    )
}

Possible Solution

Revert the change in linked commit

Context

We use empty payloads to upload a placeholder file to S3, but with the CrtHttpEngine, we are not able to do this. This works fine with the default OkHttpEngine which does include the zero Content-Length header for empty payloads.

Your Environment

• Smithy Kotlin version used: 0.12.13 (although the issue is present in later versions too)
• Platform (JVM/JS/Native): JVM

[ianbotsf]

This issue has been fixed since #819 was merged.