Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: Support generating presigned url for UploadPart operation #1809

Open
wants to merge 16 commits into
base: main
Choose a base branch
from

Conversation

plu
Copy link

@plu plu commented Oct 30, 2024

It adds methods to generate a presigned url for the UploadPart operation.

Issue

Fixes #1808, #723

Description of changes

It generates two new methods:

  • UploadPartInput.presignURL(config:expiration:)
  • S3Client.presignedURLForUploadPart(input:expiration:)

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.

@plu plu closed this Oct 30, 2024
@plu plu reopened this Oct 31, 2024
@plu
Copy link
Author

plu commented Oct 31, 2024

After 9e3244e the generated URL's are correct and uploading works :).

Will look into writing tests for this now.

Comment on lines +32 to +39
if let partNumber = input.partNumber {
let queryItem = Smithy.URIQueryItem(name: "partNumber".urlPercentEncoding(), value: Swift.String(partNumber).urlPercentEncoding())
builder.withQueryItem(queryItem)
}
if let uploadId = input.uploadId {
let queryItem = Smithy.URIQueryItem(name: "uploadId".urlPercentEncoding(), value: Swift.String(uploadId).urlPercentEncoding())
builder.withQueryItem(queryItem)
}
Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Wondering if there is a way to generate this.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Query item provider is already generated here. It handles adding input values with @httpQuery trait in Smithy model for the service to the request at runtime.

The UploadPartInput.queryItemProvider gets used here for normal request flow, and to add QueryItemMiddleware for the presignURL method, only change that needs to be made is probably adding the middleware codegen back for UploadPart in here. I.e., this custom middleware is not necessary.

@plu plu changed the title feat: Add UploadPart to list of presignable url operations feat: Support presigning url for UploadPart operation Nov 1, 2024
@plu plu changed the title feat: Support presigning url for UploadPart operation feat: Support generating presigned url for UploadPart operation Nov 1, 2024
Copy link
Contributor

@sichanyoo sichanyoo left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you for submitting this PR. I added couple pointers & comments.

Feel free to lmk if you'd rather have me take up the PR. I'm happy to do either (review your PR and have you drive it through to merge, or take up the work from you going forward).

Comment on lines +32 to +39
if let partNumber = input.partNumber {
let queryItem = Smithy.URIQueryItem(name: "partNumber".urlPercentEncoding(), value: Swift.String(partNumber).urlPercentEncoding())
builder.withQueryItem(queryItem)
}
if let uploadId = input.uploadId {
let queryItem = Smithy.URIQueryItem(name: "uploadId".urlPercentEncoding(), value: Swift.String(uploadId).urlPercentEncoding())
builder.withQueryItem(queryItem)
}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Query item provider is already generated here. It handles adding input values with @httpQuery trait in Smithy model for the service to the request at runtime.

The UploadPartInput.queryItemProvider gets used here for normal request flow, and to add QueryItemMiddleware for the presignURL method, only change that needs to be made is probably adding the middleware codegen back for UploadPart in here. I.e., this custom middleware is not necessary.

@@ -86,6 +89,9 @@ class PresignableUrlIntegration(private val presignedOperations: Map<String, Set
"com.amazonaws.s3#PutObject" -> {
renderMiddlewareClassForPutObject(ctx, delegator, op)
}
"com.amazonaws.s3#UploadPart" -> {
renderMiddlewareClassForUploadPart(ctx, delegator, op)
}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As per previous comment, to use the already generated queryItemProvider for UploadPartInput, you won't need to add this custom middleware here anymore.

Comment on lines +213 to +216
"com.amazonaws.s3#UploadPart" -> {
operationMiddlewareCopy.removeMiddleware(op, "OperationInputBodyMiddleware")
operationMiddlewareCopy.appendMiddleware(op, UploadPartPresignedURLMiddlewareRenderable())
}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is where you could just add back the QueryItemMiddleware which has the name OperationInputQueryItemMiddleware; it's located here.

Comment on lines +280 to +307
private fun renderMiddlewareClassForUploadPart(codegenContext: SwiftCodegenContext, delegator: SwiftDelegator, op: OperationShape) {

val serviceShape = codegenContext.model.expectShape<ServiceShape>(codegenContext.settings.service)
val ctx = codegenContext.toProtocolGenerationContext(serviceShape, delegator)?.let { it } ?: run { return }

val opIndex = OperationIndex.of(ctx.model)
val inputShape = opIndex.getInput(op).get()
val outputShape = opIndex.getOutput(op).get()
val operationErrorName = MiddlewareShapeUtils.outputErrorSymbolName(op)
val inputSymbol = ctx.symbolProvider.toSymbol(inputShape)
val outputSymbol = ctx.symbolProvider.toSymbol(outputShape)
val outputErrorSymbol = Symbol.builder().name(operationErrorName).build()
val filename = ModelFileUtils.filename(ctx.settings, "${inputSymbol.name}+QueryItemMiddlewareForPresignUrl")
val headerMiddlewareSymbol = Symbol.builder()
.definitionFile(filename)
.name(inputSymbol.name)
.build()
delegator.useShapeWriter(headerMiddlewareSymbol) { writer ->
val queryItemMiddleware = UploadPartPresignedURLMiddleware(
inputSymbol,
outputSymbol,
outputErrorSymbol,
writer
)
MiddlewareGenerator(writer, queryItemMiddleware).generate()
}
}

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not needed if using preexisting codegen'd query item provider & runtime library's query item middleware

Comment on lines +1 to +31
package software.amazon.smithy.aws.swift.codegen.middleware

import software.amazon.smithy.model.shapes.OperationShape
import software.amazon.smithy.swift.codegen.SwiftWriter
import software.amazon.smithy.swift.codegen.integration.ProtocolGenerator
import software.amazon.smithy.swift.codegen.middleware.MiddlewareRenderable

// This middleware renderer inserts a custom middleware named `UploadPartPresignedURLMiddleware`
// into the operation stack. It is only intended for use with S3 `UploadPart` and only when
// generating a pre-signed URL.
class UploadPartPresignedURLMiddlewareRenderable : MiddlewareRenderable {

override val name = "UploadPartPresignedURLMiddleware"

override fun render(
ctx: ProtocolGenerator.GenerationContext,
writer: SwiftWriter,
op: OperationShape,
operationStackName: String
) {
super.renderSpecific(ctx, writer, op, operationStackName, "serialize")
}

override fun renderMiddlewareInit(
ctx: ProtocolGenerator.GenerationContext,
writer: SwiftWriter,
op: OperationShape
) {
writer.write("\$L()", name)
}
}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not needed if using preexisting codegen'd query item provider & runtime library's query item middleware

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Unable to get presigned URL for UploadPart with all mandatory query parameters
2 participants