Announcement: S3 default integrity change

[aBurmeseDev]

In AWS SDK for Rust v1.69.0, we released changes to the S3 client that adopts new default integrity protections. For more information on default integrity behavior, please refer to the official SDK documentation. In SDK releases from this version on, clients default to enabling an additional checksum on all Put calls and enabling validation on Get calls.

You can disable default integrity protections for S3. We do not recommend this because checksums are important to S3 integrity posture. Integrity protections can be disabled by setting the config flag to WHEN_REQUIRED, or by using the related AWS shared config file settings or environment variables.

Disclaimer: the AWS SDKs and CLI are designed for usage with official AWS services. We may introduce and enable new features by default, such as these new default integrity protections, prior to them being supported or otherwise handled by third-party service implementations. You can disable the new behavior with the WHEN_REQUIRED value for the request_checksum_calculation and response_checksum_validation configuration options covered in Data Integrity Protections for Amazon S3.

[danprueitt]

Seems changing the default breaks AWS S3 Presigned PUTs if you don't explicitly set a checksum? I can see a checksum is added to the headers and when the client attempts an upload it gets:

Error><Code>BadDigest</Code><Message>The CRC32 you specified did not match the calculated checksum.</Message>
...

The issue was resolved by setting the config to WHEN_REQUIRED, but this feels like a breaking behavior change in this scenario. Wouldn’t it be better to manage this change behind a behavior_version in the AWS config instead?

[ysaito1001]

Hi @danprueitt, thank you for bringing that to our attention. Sounds like it's a regular S3 presigned PUT requests that are affected? Can you provide us with a reproducer so we can work backwards from it?

[landonxjames]

@danprueitt I was able to replicate your issue and working on a fix now, thanks for bringing this to our attention.

[aaronmondal]

It looks like this change added the crc64fast-nvme dependency which contains bindgen code as of awesomized/crc64fast-nvme@1ec317f. This is somewhat hard to work with in sandboxed builds (like Bazel) as it tries to write to read-only directories. IIUC the aws sdk doesn't actually make use of the bindings.

See also: awesomized/crc64fast-nvme#5

[landonxjames]

Fix was merged in smithy-lang/smithy-rs#3971, we are working to release the codegen now and the actual fix should be released in version 1.70.0 of the aws-sdk-s3 crate on Monday