Support for new v2 OAuth flow

[jookyboi]

Any plans to migrate over to use Slack's v2 OAuth flow?
https://api.slack.com/authentication/migration#update_oauth

[jasonrudolph]

In case it helps, I've been successfully using this patch to get things working with Slack's V2 OAuth flow.

That patch might be able to serve as a starting point for a pull request to add support for the V2 OAuth flow, but if it's important to retain support for the original OAuth flow at the same time, then more work would be needed.

[kwent]

Using with devise as

  config.omniauth :slack, ENV['SLACK_APP_ID'], ENV['SLACK_APP_SECRET'], {
    ## specify options for your oauth provider here, e.g.:
    scope: 'identity.basic,identity.email',
  }

is generating

https://workspace.slack.com/oauth?client_id=<redacted>&redirect_uri=http%3A%2F%2Flocalhost%3A3000%2Fusers%2Fauth%2Fslack%2Fcallback&state<redacted>&scope=identity.basic%2identity.email&user_scope=&granular_bot_scope=1&install_redirect=&single_channel=0&team=1&tracked=1

Note the scope=identity.basic%2identity.email and empty user_scope.

And slack is not happy

But if i modified in the url with user_scope=identity.basic%2identity.email slack is happy.

Modifying

  config.omniauth :slack, ENV['SLACK_APP_ID'], ENV['SLACK_APP_SECRET'], {
    ## specify options for your oauth provider here, e.g.:
    user_scope: 'identity.basic,identity.email',
  }

does not have any effect and user_scope in url is still empty and so i get

So i think the branch needs a bit more work to support this attribute otherwise Sign In With slack feature is not possible at all.

Regards

[kwent]

After further debugging and research i found this post https://stackoverflow.com/questions/61150208/sign-in-with-slack-invalid-scopes-identity-basic-identity-avatar

So using /oauth/v2/authorize and /api/oauth.v2.access for add to slack and /oauth/authorize and /api/oauth.access for signin with slack

Sign in with slack is using different endpoints and does not work with /v2 ones. Yes i know crazy... Slack documentation does not mention this anywhere

[kwent]

I opened a PR for user_scope still jasonrudolph#1

[ginjo]

Hi all,

I'll work these patches into the gem. I'd like to keep it backwards compatible with Slack's original (V1?) oauth flow, so I'll probably add a strategy option for flow_version, or something similar. I'll also make sure that the new user_scope is handled.

Parsing the authorization response AuthHash, tokens, info, etc., will also require some tweaking. Hopefully, my previous addition of the DataMethods layer will make that less tedious now.

Thanks everyone for the sleuthing and the code!

[ginjo]

Here's an update on my progress with Slack's v2 oauth flow.

The Good

I've incorporated the changes posted by @jasonrudolph and @kwent into the gem. I've also added an option flow_version to client_options, which defaults to 'v2'. The authorization will use the Slack v2 endpoints by default. You can set client_options: {flow_version: 'v1'} in the omniauth builder block to force the v1 flow endpoints. user_scope is now supported in the v2 flow and is required by Slack for user scopes like identity.user, identity.team, etc.

So this all works well. The gem handles the v2 authorization request (redirect) and the api/oauth.v2.access request just fine. But then things turn sour when receiving the api/oauth.v2.access response.

The Bad

If you are authenticating with only user_scope scopes, as you would for a sign-in-with-slack authorization, Slack's api/oauth.v2.access response does NOT conform to the OAUTH2 spec, which causes the Oauth2 gem to raise a generic invalid_credentials error. This happens even when Slack returns a successful response containing a valid user token.

The OAUTH2 Spec states that a successful get-token API response MUST contain the access_token field at the top level of the JSON data.

And here's what Slack's api/oauth.v2.access response looks like, when using only user_scope scopes.

{
  "enterprise" : null,
  "ok" : true,
  "team" : {
    "id" : "T0BXXXXXX"
  },
  "authed_user" : {
    "scope" : "identity.basic,identity.email,identity.avatar,identity.team",
    "id" : "U0BXXXXXX",
    "token_type" : "user",
    "access_token" : "xoxp-111111111111-22222222222-3333333333333-fa39d45841fa1daab3a98f945a133d02"
  },
  "app_id" : "A0XXXXXXXXX"
}

To temporarily get past the above mess, I've overridden the Oauth2 Client#get_token method to skip validation of the api/oauth.v2.access response data (I'll make this an option if we need to keep this hack long-term). This allows the Oauth2 gem to package the response into an OmniAuth AuthHash object...

But only sometimes: About half of my passes through Slack's v2 oauth flow return oauth_authorization_url_mismatch from the api/oauth.v2.access request. According to Slack Documentation, this means that the authorization url used to get the current auth-code was the wrong url. According to my logs and testing, I am NOT using the wrong authorization URL.

The Ugly

Even if there were no errors in the oauth flow, Slack's change in the access-token response object, and the addition of a second level of scopes, means that not only our libraries, but also our applications, will need to be updated/changed/tweaked, possibly to significant extent.

Summary

1. Slack's api/oauth.v2.access response does not always adhere to the OAUTH2 spec, causing errors in the Oauth2 gem, even with a successful response containing a valid access-token.

2. Slack's api/oauth.v2.access API call seems to randomly return oauth_authorization_url_mismatch, even when the correct authorize-url was used to initiate the oauth flow.

3. Slack's api/oauth.v2.access response and the hard split between regular and user scopes makes for increasingly complex downstream session/token/user management. In some cases, this may require significant refactoring in Application code.

It is possible that I've missed something important that could reduce all of this to a simple misunderstanding. I hope that's the case. I will update when I find out more info.

[bearyjon]

Thanks for digging into this @ginjo. I'm following along closely and I'd love to try out the branch. I can't make any guarantees that I'll be able to track down the random authorize-url error, but maybe another person poking at it might help.

Are you open to pushing the branch to GitHub?

[kwent]

@ginjo haha i wanted to do a comment about the sign in with slack use case but you did it before me ! Yes i have been struggling all week to know what was going on til i figure out the OAUTH2 spec spec was not conform.

Here is my ugly fix https://github.com/kwent/omniauth-slack/blob/use-v2-oauth-api-user-scope-providers/lib/omniauth/strategies/sign_in_slack.rb

I created a all new provider OmniAuth::Strategies::SignInSlack and implemented a new SignInSlackClient which inherits OAuth2::Client and read the nested access_token properly...

I don't have any randomize error on my side with this patch but i used to meet oauth_authorization_url_mismatch for another falsy reason. (Confirmed by a slack engineer). Is that if you have a redirect_uri in your authorizecall and don't call api/oauth.v2.access with this same redirect_uri (which is mandatory if specified in the authorize call). You might meet this error. They basically raise the wrong error type.

[ginjo]

After digging around in the code, building test apps, comparing different API requests/responses, and having a few back-and-forths with Slack support, I've a better handle on the current state of OmniAuthSlack-vs-Slack-API. And while the request/response/build-token cycle is pretty straightforward (aside from a few gotchas I'll mention shortly), the messy part of squeezing Slack's OAUTH response into OmniAuth's AuthHash object is... well, messier than ever.

Here are some general observations regarding Slack's v2 API. These observations update any that I documented in previous posts. If I'm wrong about any of these, please call them out so we can nail them down.

Or, you can skip to the botton, where I summarize the current state of OmniAuthSlack and the basics of what it can do with Slack's v2 API.

Authorization request /oauth/v2/authorize

• Slack's v2 API uses two scope fields in the authorization request. user_scope only accepts scopes associated with user tokens. scope only accepts scopes associated with bot tokens.

• Either-or-both of the v2 scope fields can be presented during a single authorization request.

• Presenting the user_scope field with identity scopes only, and with no scope field, will invoke the process we know as sign-in-with-slack.

• Presenting the user_scope field with identity scopes and other user-token scopes will raise an error from slack about not being able to mix the two types of user scopes.

• Attempting to run the sign-in-with-slack scenario where the Slack confirmation page is skipped, by passing a team field containg team-id, will raise a oauth_authorization_url_mismatch error in the oauth.v2.access phase.

Token request and response /api/oauth.v2.access

• The Slack v2 API token response is quite different from the v1 API workspace-app token response.

• The parsed response object from Slack is a two-tiered object, with the top-level being a bot-token hash, and with the sub section authed_user being a user-token hash.

• Both a bot and a user-token can be returned in a single response.

• In a sign-in-with-slack scenario, where no scope field was presented in the authorization request,
  no bot-token will be present in the access-token response. However, the user-token will still be in a sub section authed_user of the JSON object. This is non-conformant with the Oauth2 spec, which states that all successful token responses must have an access_token field, with data, at the top level of the JSON object.

• The OAuth2 gem will raise a generic invalid-credentials error upon receiving the non-conformant token response from Slack. The current version of the OAuth2 gem provides no easy way to adjust the validation of the token response object. The only way to avoid the error is to include raise_errors: false in the client_options hash within the provider :slack call in the OmniAuth builder block. This workaround only helps in OAuth2 gem version 1.4.4+.

Slack's API spec

• https://api.slack.com/specs/openapi/v2/slack_web.json
• https://github.com/slackapi/slack-api-specs

Slack's hard split of scopes between scope and user_scope during the OAUTH authorization request require that we know which scope field to use for every one of Slack's API scopes.

• The information that relates methods and scopes to token types (and thus which scope field to use) is available in Slack's general API documentation but not in Slack's API spec, making programatic handling of the dual scope fields currently impossible (unless you want to create and maintain your own reference table from Slack's documentation pages).

• The current Slack API spec data is incorrect with regard to what scopes are required for some methods.

OmniAuthSlack

In my dev branch (will push to public soon), authorization and access-token request/response works to the extent that you can obtain a fully functional OAuth2::AccessToken object for both the bot-token and the user-token.

# In your callback action:

@bot_token = env['omniauth.strategy'].access_token

This top-level access token will be a bot-token, or an invalid OAUTH response object (if you didn't include any scopes in the scope authorization field).

The user-token, if included, will be in a authed_user hash within this top-level token, regardless of whether the top-level is a legitimate bot-token or not. As an OmniAuthSlack convenience, the user-token can be accessed like this:

@bot_token = env['omniauth.strategy'].access_token
@user_token = @bot_token.user_token

The returned user_token is a full-fledged OAuth2::AccessToken object built from the authed_user hash. It uses the same OAuth2::Client instance as the main (bot) access-token.

You can use both of these AccessToken instances to query the Slack API.

@bot_token.get("/api/conversations.list", ...).parsed
# --> response data hash from Slack

@user_token.get("/api/users.identity", ...).parsed
# --> response data hash from Slack

Not-quite-ready-for-prime-time are the AuthHash object from env['omniauth.auth'] and the AccessToken#has_scope? method. As mentioned above, this is the messy part. I won't get into why it's messy, as you can see for yourself by comparing the AuthHash objects with the actual response data contained in the AccessToken objects.

The OmniAuth::AuthHash object has some really useful breakdowns of the data retrieved from Slack API, but you don't strictly need the AuthHash object, unless you're building an application that relies on the (mostly) uniform structure of the AuthHash across different providers.

• Disclaimer: The Client, AccessToken, and AuthHash objects are actually subclassed within the OmniAuthSlack gem, allowing us to work around some of the idiosyncrasies of the OmniAuth and OAuth2 gems.

• More disclaimer: No disrespect intended. Omniauth, oauth2, and omniauth-oauth2 are awesome libraries!

Slack workspace-app tokens, classic tokens, and legacy tokens

Prior to Slack's v2 API, most of my work on this gem was focused on supporting the workspace-app tokens. I've attempted to keep that functionality while developing the v2 API, even though the two are rather incompatible. I have not yet attempted to maintain OmniAuthSlack compatibility with tokens or functionality from before the workspace-app tokens.

[martingjaldbaek]

That's an awesome piece of research and debugging. Thank you so much for taking the time to figure all this out, and for maintaining this gem in general - it's very much appreciated!

[ginjo]

Thanks everyone for the feedback! I've pushed my latest changes to the master branch. The basic functionality of the gem should work, however a bunch of the tests are currently broken (or obsolete), and the AuthHash object is still in flux. Here's how it currently works with the v2 API:

• The info hash is populated with user (person) data from the AccessToken instance, if possible. Otherwise, it pulls bot data, if possible.

However, I'm considering changing this to:

• The info hash always pulls user (person) data, if possible. All bot data is stored in the extra['bot_info'] hash, if possible.

I'll release the gem when I get these things cleared up and after I've updated the documentation. A good portion of the README is currently geared toward workspace-app tokens.

---

Meanwhile, I just heard back from a Slack tech regarding the conversations.join method inaccuracies in the spec document (mentioned in my previous post). Turns out that method can accept multiple scopes, depending on which token is being requested:

Regarding the "conversations.join" method, it actually accepts both scopes,
but this will depend on whether the token is a user or bot token.

The tech said the spec document just hasn't caught up with the newer features and changes yet.

[ream88]

This is the custom strategy we rolled with as I always felt omniauth-slack being a little bit bloated – no offense here, thank you to all the maintainers of this gem, but we had to move on 😉:

# inspired by
# https://dev.to/vvo/devise-create-a-local-omniauth-strategy-for-slack-4066

require 'omniauth-oauth2'

module OmniAuth
  module Strategies
    class Slack < OmniAuth::Strategies::OAuth2
      option :client_options, site: 'https://slack.com', authorize_url: 'oauth/v2/authorize', token_url: 'api/oauth.v2.access'
      option :authorize_options, %i[scope state user_scope]

      uid do
        "#{raw_info.dig('authed_user', 'id')}-#{raw_info.dig('team', 'id')}"
      end

      info do
        raw_info
      end

      def raw_info
        @raw_info ||= access_token.params
      end

      def callback_url
        full_host + script_name + callback_path
      end
    end
  end
end

[ginjo]

No offense taken @ream88, I appreciate your input and the code example. I am solely responsible for any bloating in this fork of the gem 😬 The extra code in this branch serves two main purposes: 1. Create a consistent AuthHash object, regardless of what kind of token is requested and returned. 2. Provide a handful of extra features that should be helpful to users of the gem in a variety of situations.

But I have to admit that in trying to keep up with the evolution of Slack's API while maintaining features and behavior of the gem, I've sometimes been tempted to throw the whole thing out and start over with something bare-bones simple, as you posted above.

This all brings up a good discussion point: What is most important to users of omniauth-slack? Here are some vectors to consider.

• simplicity
• extra features
• behavioral consistency
• data structure consistency
• size
• speed
• robustness
• other things I'm forgetting to mention

Opinions and input welcome...

[bearyjon]

The most important feature to me right now is to get sign-in-via-Slack and add-to-slack capabilities in my app via the most modern available Slack API.

At least in our case, most of our Slack integration is with the Events API and other such APIs, and the process of getting a token is the least interesting part of what we're working on. I imagine I feel similarly to @ream88 with "we had to move on" being a primary goal for me. I've personally spent an embarrassingly large portion of my time working with various Slack auth gems or mini-scripts like @ream88's. I'm feeling fairly guilty about picking Ruby/Rails for this part of the project when there are official libraries available for other languages and frameworks :)

[kwent]

I wouldn't blame this library but i would blame Slack who doesn't properly implements OAuth 2.0 specification properly on their v2 api. If they were, it would have worked out of the box. Other libraries and framework must face the same issue working with v2.

[bearyjon]

Agreed @kwent - hope I didn't come across as too critical of this library and all the work that @ginjo has put into this. I definitely feel like Slack could have been a bit better here.

Thanks for everything you've done to make things work @ginjo. Without this library, I would probably still be digging around trying to get things to work.

[ginjo]

Thanks for the feedback @bearyjon and @kwent . I think the frustration we're all feeling comes from a combination of things that individually aren't much of a problem, however as technology evolves, we're starting to feel the boundaries of the individual components... And they don't always line up the way we want.

• The OAUTH2 spec for the token-response strictly states that a token string must be present. Beyond that, however, it's wide open... anything goes.

• Slack has a product that has grown in complexity beyond what the OAUTH2 spec comfortably handles. I think what Slack really wants is an OAUTH2 response that contains an array of tokens.

• OmniAuth appears to have been originally designed to normalize the authorization flow and response data, regardless of how the authorization was obtained or how its data is structured. Yet normalization requires that the data to be normalized adheres to at least some kind of agreed-upon structure. OAUTH2 spec does not specify that structure.

• OAuth2 (gem) and OmniAuth-OAuth2 further project the boundaries (and vagaries) of the OAUTH2 spec and wrap them up in the the OmniAuth model. Breaking out of these boundaries requires customization, hacking, and/or overriding the OAUTH2 spec.

Please don't take this as a rant. These are all wonderful tools, and I'm not trying to point fingers or lay blame. Just attempting to shed some light on the issues. We could build our own authorization tools from scratch using net/http, and it would not be that difficult. Or we could build a simple slack-oauth2 gem that is focused solely on Slack's OAUTH2 implementation (already exists I think). But the OmniAuth, OAuth2, and OmniAuth-OAuth2 gems bring a lot to the table, and I'm hoping we can find that sweet spot where we make good use of them, without becoming mired in the complexities.

Moving forward, I hope the latest version of this gem will solve (or at least make solvable) some of the issues people are having. I will release v2.5.0 soon. Here are some highlights:

• Compatible with all Slack API versions and tokens (that are currently available and supported by Slack).

• Only make one API call to Slack: oauth.[v2.]access (others can be added by developers).

• Stop trying to map ALL Slack data specifics to the AuthHash (will only map the "required" fields).

• Make the AccessToken available and functional as an API gateway and make it easy to serialize for storage. Also make it easy to reconstitute the serialized data into a fully functional AccessToken (OAuth2 gem makes this all possible, and OmniAuthSlack makes it more accessible and tailored to Slack).

• (experimental) Provide a VerifySlackRequest middleware module to verify incoming Slack requests (events) against the Slack signing-secret (super helpful if you're receiving Slack events on your application's API).

I'll post an update when I release the gem. Meanwhile, feel free to try out your apps against the master branch. Functionally, it's all there. Just needs a little more cleanup before releasing.

[fayeroske]

First, I've found this thread incredibly helpful as we work though our Granular Scopes / v2 migration. Thank you @ginjo .

Fun finding, we just came across this -- any chance including the OpenID stuff in omniauth-slack is in the cards?

[kwent]

@fayeroske https://github.com/amkisko/omniauth-slack-openid.rb is working well here