Prevent override of region when using endpoint_url

[kyleknap]

This ensures that user's region is not overwritten by the credentialScope if the user supplies an endpoint url. This is needed to be able to make signed calls to STS regionalized endpoints, without getting auth errors due to incorrect region . So for example, in the CLI, to make a GetSessionToken call to us-west-2 you would do:

$ aws sts get-session-token --region us-west-2 --endpoint-url https://sts.us-west-2.amazonaws.com

Before, this would cause a signing error if signed with sigv4 because the credentialScope was set to us-east-1 and that overrode the region that you provided.

cc @jamesls @danielgtaylor

[coveralls]

Coverage increased (+0.0%) to 94.51% when pulling 1a25222 on kyleknap:endpoint-override-region into 6a63e18 on boto:develop.

[kyleknap]

Coveralls is wrong. Coverage should have went up.

[danielgtaylor]

LGTM 🚢-it!

[jamesls]

Region is only required for sigv4, so you could previously specify an endpoint-url without a region provided you weren't using sigv4. For example, this is a breaking change for commands such as this:

$ aws configure list | grep region
    region                <not set>             None    None
$ aws s3api list-buckets --endpoint-url https://s3.amazonaws.com/

[kyleknap]

Good point so how about instead?
if endpoint_url is None and region_name is None:

[jamesls]

I don't think that will fix the issue. In the example above endpoint_url is not None so we don't pull in the region override from the heuristics file.

[coveralls]

Coverage increased (+0.0%) to 94.51% when pulling 68a4c9d on kyleknap:endpoint-override-region into 6a63e18 on boto:develop.

[kyleknap]

@jamesls
I think I got the s3 case covered now. The credentialScope region is only used now if it is specified in the endpoint JSON, an endpoint url was not specified, and then, if a region was not specified by the user.

[jamesls]

Is there another change that's needed before the calls to us-west-2 work? I'm seeing a signing error with the latest changes:

$ aws sts get-session-token --region us-west-2 --endpoint-url https://sts.us-west-2.amazonaws.com

A client error (SignatureDoesNotMatch) occurred when calling the GetSessionToken operation: Credential should be scoped to a valid region, not 'us-east-1'.

[coveralls]

Coverage remained the same at 94.5% when pulling 57a3240 on kyleknap:endpoint-override-region into 6a63e18 on boto:develop.

[kyleknap]

To make it clear what I did in the latest commits, here is a summary:

1. Separate all of the region determining logic into a helper method _determine_region_name() in the endpoints class. The goal is that this method can be directly ported to clients when we deprecate operation objects and endpoint.region_name. Note that now both the clients and operations rely on this helper method for region name logic. This consolidation will ensure that clients and operation objects will use the same codepath when choosing a region name.

2. Removed some of the signature version arguments from endpoint methods. We were not using them at all.

3. Added integration test to test that sts regionalized calls can be made.

[danielgtaylor]

This is a backward-incompatible change of a public function. It's possible we might break someone using Botocore.

[danielgtaylor]

I think this looks good. We just need to decide if we really want to make that breaking change. 🚢-it.

[jamesls]

You can't call sts when you're using session credentials (say on a CI box using an IAM role), so we should put a check where we only run these tests if we have long term credentials.

[coveralls]

Coverage increased (+0.0%) to 94.51% when pulling 2d215bc on kyleknap:endpoint-override-region into 6a63e18 on boto:develop.

[kyleknap]

@jamesls
I made the if statement in the setUp that if a session token is being used then skip the test.

[jamesls]