SecretsManager rotate_secret() strips the "secret_string" value from the "AWSCURRENT" version, rather than creating an empty secret version for "AWSPENDING"

[mcintoac-aws]

As part of release 5.0.27, this PR was merged: #8504

The PR was meant to create an "AWSPENDING" version with no string value.

Instead, the behavior mistakingly removes the "secret_string" value, which is a key on "secret_version", for the "AWSCURRENT" version.

This isn't the correct behavior, and results in errors when attempting to call get_secret_value(...) for the "AWSCURRENT" VersionStage after performing a rotate_secret(...) call.

The error can be more specifically diagnosed by viewing these lines in the CR which removed the old secret string value, which was sent as an argument to _addSecret. To allow for a new secret version creation to be detected, a new boolean was also added: https://github.com/getmoto/moto/pull/8504/files#diff-88f9a429095123bea6fa9447524fa1ff91beb2f59a94ceea839b157855dc22eaL771-L778

However now, reset_default_version is called with "AWSCURRENT" as the version stage, but there is no "secret_string" key within the version:

moto/moto/secretsmanager/models.py

Line 179 in 786a8ad

def reset_default_version(

[bblommers]

Hi @mcintoac-aws, can you share a small reproducible test to show the problem?

Our own test specifically verifies that this value exists, so I'd be curious to see what the difference is/what we're missing:

moto/tests/test_secretsmanager/test_secretsmanager.py

Lines 1304 to 1314 in 786a8ad

	secret_not_updated = True
	while secret_not_updated:
	updated_secret = secrets_conn.get_secret_value(
	SecretId=secret["ARN"], VersionStage="AWSCURRENT"
	)
	if updated_secret["SecretString"] == "UpdatedValue":
	secret_not_updated = False
	else:
	from time import sleep
	sleep(5)

[mcintoac-aws]

Certainly, thank you for looking into this @bblommers . The code I pointed out in the original post I still believe is the root cause (where secret_string was removed):

@mock_aws
def test_rotate_secret_example(
    custom_resource_load_or_create_iot_credentials_event: Dict[str, Any]
) -> None:
    print("\nRunning example test...\n")
    # The event fixture mocks an AWS lambda function for rotation so we can call the rotate_secret function
    # and sets up a mocked IAM role for that lambda as well.
    # The secret arn is then stored in an event object we can get here
    rotate_secret_lambda_arn = custom_resource_load_or_create_iot_credentials_event["ResourceProperties"]["RotateSecretLambdaARN"]

    test_secret = "test_secret"
    secrets_manager_client = boto3.client("secretsmanager")

    # assert that secret does not exist
    with pytest.raises(ClientError):
        secrets_manager_client.get_secret_value(SecretId=test_secret)

    # create secret, store in variable for later use when rotating
    secret = secrets_manager_client.create_secret(
        Name=test_secret,
        ClientRequestToken=str(uuid.uuid4()),
        Description="Example secret.",
        SecretString="Example secret value.",
    )

    # confirm that created secret now exists (no error is thrown)
    secrets_manager_client.get_secret_value(SecretId=test_secret)
    print("Initial get_secret_value call passes before rotation.")

    # rotate secret
    secrets_manager_client.rotate_secret(
        SecretId=secret["ARN"],
        ClientRequestToken=str(uuid.uuid4()),
        RotationLambdaARN=rotate_secret_lambda_arn,
        RotationRules={
            "AutomaticallyAfterDays": 90,
        },
        RotateImmediately=False,
    )

    # Now try to retrieve secret and assert an error is thrown
    with pytest.raises(Exception) as resource_not_found_error:
        secrets_manager_client.get_secret_value(SecretId=test_secret)

    if resource_not_found_error:
        print("\nERROR:\n")
        print(resource_not_found_error)
    else:
        print("NO ERROR.")

Here is the test output:

pytest -s -k test_rotate_secret_example

> Running example test...

> Initial get_secret_value call passes before rotation.

> ERROR:

> <ExceptionInfo ResourceNotFoundException("An error occurred (ResourceNotFoundException) when calling the GetSecretValue operation: Secrets Manager can't find the specified secret value for staging label: AWSCURRENT") tblen=3>

Creating a similar test with further debugging calls to the secret versions can reveal that the secret_string value for the AWSCURRENT version is being removed, hence get_secret_string(...) fails.

Note that this is different than the version not existing, which will throw a separate error and can be seen by specifying a non-existent version rather than AWSCURRENT.