listObjectVersions not returning all existing versions

[RodrasSilva]

Describe the bug

After calling deleteObject for a file without specifying a version (thus creating a 'deleted version'), subsequent calls to listObjectVersions fail to display versions of files with a lower alphabetical order. This inconsistency poses a challenge in accurately managing file versions.

Expected behavior

When I call listObjectVersions I expect to see all versions of all objects except for deleted versions.

Current behavior

When I call listObjectVersions some versions are not returned, on objects with no deleted versions

Steps to Reproduce

Example 1:

1. First, we create file-1, file-2, and file-3 on S3.
2. Then, we generate a delete version for file-2 using the deleteObject method, specifying only the bucket and the filename (no version is specified).
3. Upon calling listObjectVersions, we observe that it returns versions for file-2 and file-3 exclusively. It's noteworthy that the alphabetical order of filenames is crucial; had file-3 been the one deleted, listObjectVersions would solely present versions for file-3.

Example 2:

1. Initially, we create file-3 and file-2.
2. We proceed to create a delete version for file-3 using the deleteObject method, specifying only the bucket and the filename (no version is specified).
3. Subsequently, we create file-1 and file-4 on S3.
4. Upon invoking listObjectVersions, we notice that it exclusively returns versions for file-3 and file-4, despite the fact that file-1 was generated subsequent to the invocation of deleteObject for file-3.

Possible Solution

No response

Context

Migration from Java to Kotlin SDK.
I've encountered an unusual bug while attempting to delete multiple files from my MinIO instance, which we use in our local environment as our S3 instance. To delete all objects along with their versions, I utilize the listObjectVersions method to gather information and subsequently invoke the deleteObject function with the respective file key and version.

AWS Kotlin SDK version used

1.0.54

Platform (JVM/JS/Native)

JVM

Operating System and version

macOS 14.3.1 (Sonoma)

[aajtodd]

Thanks for the report, I was able to recreate this.

It seems related to the use of xmlFlattened on the model:

                "Versions": {
                    "target": "com.amazonaws.s3#ObjectVersionList",
                    "traits": {
                        "smithy.api#documentation": "<p>Container for version information.</p>",
                        "smithy.api#xmlFlattened": {},
                        "smithy.api#xmlName": "Version"
                    }
                },
                "DeleteMarkers": {
                    "target": "com.amazonaws.s3#DeleteMarkers",
                    "traits": {
                        "smithy.api#documentation": "<p>Container for an object that is a delete marker.</p>",
                        "smithy.api#xmlFlattened": {},
                        "smithy.api#xmlName": "DeleteMarker"
                    }
                },

The response I'm seeing has version and delete marker elements intermixed:

<ListVersionsResult ...>
    <DeleteMarker> ... </DeleteMarker>
    <Version> ... </Version>
    <DeleteMarker> ... </DeleteMarker>
    <Version> ... </Version>
    ....
</ListVersionsResult>

This is of course valid XML but wasn't expected by the XML deserializer for the given trait.

We'll have to investigate how to best rectify this in the short term and in the longer term clarify with Smithy team on whether this is expected service behavior.

[RodrasSilva]

Hello @aajtodd. Do you have any updates from the Smithy team? I don't think this is expected service behavior since this is a different behavior from the one present in the Java SDK.

[aajtodd]

It's a bug in the SDK. I'm working on a fix.

[github-actions]

⚠️COMMENT VISIBILITY WARNING⚠️

Comments on closed issues are hard for our team to see.
If you need more assistance, please either tag a team member or open a new issue that references this one.
If you wish to keep having a conversation with other community members under this issue feel free to do so.

[aajtodd]

Fix for this has been merged to main, should be available in a release either today or tomorrow.

[RodrasSilva]

Fix for this has been merged to main, should be available in a release either today or tomorrow.

Thank you for your help @aajtodd 🦾