JwtAccess Credentials Feature

[anthmgoogle]

Tracking completion of implementation of JwtAccess Credentials for service accounts for Python. This is a service account that passes a JWT directly as an a bearer token instead of exchanging for an access token.

For reference, the Java implementation:
https://github.com/google/google-auth-library-java/blob/master/oauth2_http/java/com/google/auth/oauth2/ServiceAccountJwtAccessCredentials.java

[dhermes]

Cool. See #211 as well. May be able to do them all with one class.

[dhermes]

@nathanielmanistaatgoogle This "should be" sufficient:

diff --git a/oauth2client/service_account.py b/oauth2client/service_account.py
index f18f192..5c8ec9d 100644
--- a/oauth2client/service_account.py
+++ b/oauth2client/service_account.py
@@ -71,6 +71,8 @@ class ServiceAccountCredentials(AssertionCredentials):
                         header of a JWT token assertion.
         client_id: string, (Optional) Client ID for the project that owns the
                    service account.
+        audience: string, (Optional) An audience to use for scopeless
+                  credentials.
         user_agent: string, (Optional) User agent to use when sending
                     request.
         kwargs: dict, Extra key-value pairs (both strings) to send in the
@@ -97,6 +99,7 @@ class ServiceAccountCredentials(AssertionCredentials):
                  scopes='',
                  private_key_id=None,
                  client_id=None,
+                 audience=None,
                  user_agent=None,
                  **kwargs):

@@ -109,6 +112,7 @@ class ServiceAccountCredentials(AssertionCredentials):
         self._private_key_id = private_key_id
         self.client_id = client_id
         self._user_agent = user_agent
+        self._audience = audience
         self._kwargs = kwargs

     def _to_json(self, strip, to_serialize=None):
@@ -137,7 +141,7 @@ class ServiceAccountCredentials(AssertionCredentials):
             strip, to_serialize=to_serialize)

     @classmethod
-    def _from_parsed_json_keyfile(cls, keyfile_dict, scopes):
+    def _from_parsed_json_keyfile(cls, keyfile_dict, scopes, audience):
         """Helper for factory constructors from JSON keyfile.

         Args:
@@ -145,6 +149,8 @@ class ServiceAccountCredentials(AssertionCredentials):
                           containing the contents of the JSON keyfile.
             scopes: List or string, Scopes to use when acquiring an
                     access token.
+            audience: string, (Optional) An audience to use for scopeless
+                      credentials.

         Returns:
             ServiceAccountCredentials, a credentials object created from
@@ -168,18 +174,21 @@ class ServiceAccountCredentials(AssertionCredentials):
         signer = crypt.Signer.from_string(private_key_pkcs8_pem)
         credentials = cls(service_account_email, signer, scopes=scopes,
                           private_key_id=private_key_id,
-                          client_id=client_id)
+                          client_id=client_id,
+                          audience=audience)
         credentials._private_key_pkcs8_pem = private_key_pkcs8_pem
         return credentials

     @classmethod
-    def from_json_keyfile_name(cls, filename, scopes=''):
+    def from_json_keyfile_name(cls, filename, scopes='', audience=None):
         """Factory constructor from JSON keyfile by name.

         Args:
             filename: string, The location of the keyfile.
             scopes: List or string, (Optional) Scopes to use when acquiring an
                     access token.
+            audience: string, (Optional) An audience to use for scopeless
+                      credentials.

         Returns:
             ServiceAccountCredentials, a credentials object created from
@@ -192,10 +201,11 @@ class ServiceAccountCredentials(AssertionCredentials):
         """
         with open(filename, 'r') as file_obj:
             client_credentials = json.load(file_obj)
-        return cls._from_parsed_json_keyfile(client_credentials, scopes)
+        return cls._from_parsed_json_keyfile(client_credentials, scopes,
+                                             audience)

     @classmethod
-    def from_json_keyfile_dict(cls, keyfile_dict, scopes=''):
+    def from_json_keyfile_dict(cls, keyfile_dict, scopes='', audience=None):
         """Factory constructor from parsed JSON keyfile.

         Args:
@@ -203,6 +213,8 @@ class ServiceAccountCredentials(AssertionCredentials):
                           containing the contents of the JSON keyfile.
             scopes: List or string, (Optional) Scopes to use when acquiring an
                     access token.
+            audience: string, (Optional) An audience to use for scopeless
+                      credentials.

         Returns:
             ServiceAccountCredentials, a credentials object created from
@@ -217,7 +229,8 @@ class ServiceAccountCredentials(AssertionCredentials):

     @classmethod
     def from_p12_keyfile(cls, service_account_email, filename,
-                         private_key_password=None, scopes=''):
+                         private_key_password=None, scopes='',
+                         audience=None):
         """Factory constructor from JSON keyfile.

         Args:
@@ -228,6 +241,8 @@ class ServiceAccountCredentials(AssertionCredentials):
                                   private key. Defaults to ``notasecret``.
             scopes: List or string, (Optional) Scopes to use when acquiring an
                     access token.
+            audience: string, (Optional) An audience to use for scopeless
+                      credentials.

         Returns:
             ServiceAccountCredentials, a credentials object created from
@@ -243,7 +258,8 @@ class ServiceAccountCredentials(AssertionCredentials):
             private_key_password = _PASSWORD_DEFAULT
         signer = crypt.Signer.from_string(private_key_pkcs12,
                                           private_key_password)
-        credentials = cls(service_account_email, signer, scopes=scopes)
+        credentials = cls(service_account_email, signer, scopes=scopes,
+                          audience=audience)
         credentials._private_key_pkcs12 = private_key_pkcs12
         credentials._private_key_password = private_key_password
         return credentials
@@ -252,12 +268,14 @@ class ServiceAccountCredentials(AssertionCredentials):
         """Generate the assertion that will be used in the request."""
         now = int(time.time())
         payload = {
-            'aud': self.token_uri,
+            'aud': self._audience or self.token_uri,
             'scope': self._scopes,
             'iat': now,
             'exp': now + self.MAX_TOKEN_LIFETIME_SECS,
             'iss': self._service_account_email,
         }
+        if self._audience is not None:  # Scope-less
+            payload['sub'] = self._service_account_email
         payload.update(self._kwargs)
         return crypt.make_signed_jwt(self._signer, payload,
                                      key_id=self._private_key_id)
@@ -316,6 +334,7 @@ class ServiceAccountCredentials(AssertionCredentials):
             scopes=json_data['_scopes'],
             private_key_id=json_data['_private_key_id'],
             client_id=json_data['client_id'],
+            audience=json_data['_audience'],
             user_agent=json_data['_user_agent'],
             **json_data['_kwargs']
         )
@@ -344,6 +363,7 @@ class ServiceAccountCredentials(AssertionCredentials):
                                 scopes=scopes,
                                 private_key_id=self._private_key_id,
                                 client_id=self.client_id,
+                                audience=self._audience,
                                 user_agent=self._user_agent,
                                 **self._kwargs)
         result.token_uri = self.token_uri

though I tried to use the system tests with this and couldn't get it to work (I tried to use an audience of the form foo.apps.googleusercontent.com)

[nathanielmanistaatgoogle]

@anthmgoogle I know you've pointed us at the Java and Ruby implementations of this but is there any RFC-like technical specification to which we might refer? Something from which we might write our test coverage?

Also @anthmgoogle what do you make of @dhermes' draft implementation in the previous comment?

[dhermes]

@anthmgoogle Bump.

Also, is azp the field intended rather than sub?

UPDATE: Never mind, RE: azp, I verified the fields that Java uses

[anthmgoogle]

Apologies. There is an internal document that I've emailed. If you copy one of the other implementations and verify and example end to end it should be good, since the server verification is pretty exacting. The most counter intuitive part is that both "iss" and "sub" need to have the same service account email address.

[dhermes]

@anthmgoogle You should check out the snippet above, where iss and sub have the same value. I think the key issue was using a client ID for that value rather than a service account email address. I'm going to give it a spin and will report back here.

[anthmgoogle]

Yes, the email rather than the client ID should be used here.

[dhermes]

@anthmgoogle I'm having some issues with the correct values. Here is a (failing) snippet that I have been tweaking to try to get it to work (requires oauth2client installed): https://gist.github.com/dhermes/a88f95b8fd9807e15c26

It does the bare minimum to mint a token and does it correctly in the scoped case but not in the scopeless case.

Things I think might be the issue:

• bad audience value (the one in the internal doc does not work)
• wrong token URI (should it be the usual https://www.googleapis.com/oauth2/v4/token or something service specific, it seems like the latter)
• bad payload on the POST (current it is a URL encoded

grant_type=urn:ietf:params:oauth:grant-type:jwt-bearer
&
assertion=BASE64_JWT_PAYLOAD

but maybe the payload should just be the JWT?

• wrong grant type

[anthmgoogle]

I think the issue is that the whole JWT should go in the request header's authorization field instead of the access token. You should not be hitting the token server to get an access token at all with this mode of authorization.

[dhermes]

Ahhhh I see, there is no token exchange at all? Makes more sense 😀

[anthmgoogle]

Also, this only works with APIs that also support gRPC, so use an API like Pubsub or DataStore to test it.

[dhermes]

I figured. Thanks for the heads up.

[dhermes]

OK we officially have a proof of concept.

I verified this works by editing the way gcloud-python-bigtable adds an OAuth 2.0 access token to a request to instead add the JWT directly:

diff --git a/gcloud_bigtable/client.py b/gcloud_bigtable/client.py
index 7e12455..b3506d9 100644
--- a/gcloud_bigtable/client.py
+++ b/gcloud_bigtable/client.py
@@ -433,7 +433,24 @@ class _MetadataPlugin(object):

     def __call__(self, unused_context, callback):
         """Adds authorization header to request metadata."""
-        access_token = self._credentials.get_access_token().access_token
+        import time
+        from oauth2client.crypt import make_signed_jwt
+        now = int(time.time())
+        signer = self._credentials._signer
+        service_uri = (
+            'https://bigtableclusteradmin.googleapis.com/'
+            'google.bigtable.admin.cluster.v1.BigtableClusterService')
+        payload = {
+            'aud': service_uri,
+            'exp': now + 3600,
+            'iat': now,
+            'iss': self._credentials.service_account_email,
+            'sub': self._credentials.service_account_email,
+        }
+        access_token = make_signed_jwt(
+            signer, payload, key_id=self._credentials._private_key_id)
         headers = [
             ('Authorization', 'Bearer ' + access_token),
             ('User-agent', self._user_agent),

and it works just fine talking to the BigtableClusterService:

>>> import gcloud_bigtable
>>> client = gcloud_bigtable.Client(project='MY-PROJECT', admin=True)
>>> client.start()
>>> client.list_zones()
[u'asia-east1-b', u'europe-west1-c', u'us-central1-c', u'us-central1-b']
>>> client.list_clusters()
([], [])
>>> client.stop()

[anthmgoogle]

Cool. It might be worth changing the variable "access_token" to "signed_jwt" since an access token is a different thing.

[dhermes]

That isn't an implementation, just a proof of concept. We at least have a jumping off point here in oauth2client now.

[theacodes]

Thank you for creating this issue, however, this project is deprecatedand we will only be addressing critical security issues. You can read moreabout this deprecation here.

If you need support or help using this library, we recommend that you ask yourquestion on StackOverflow.

If you still think this issue is relevant and should be addressed, pleasecomment and let us know!