Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support dynamic domains in auth backends #3106

Merged
merged 28 commits into from
May 21, 2021

Conversation

leszke
Copy link
Contributor

@leszke leszke commented Apr 29, 2021

Rework the auth backends to support host types and dynamic domains.

  • New mongoose_gen_auth behaviour is introduced with updated and simplified callbacks and a new API.
  • The backend-handling logic in mongoose_auth is unified and updated.
  • Backend modules support multiple domains now.
  • Unit tests use host types.

The following is intentionally not done yet, but it can be done in upcoming PR's:

  • Handling remove_domain in backends.
  • Rework of the ejabberd_auth API functions e.g. get_vh_* to make them more consistent with the backend API.
  • Similar rework of admin commands that are using 'vhosts'.
  • Rework of error handling in ejabberd_auth, which differs between API functions and can be unified. After unification, add more tests for ejabberd_auth.
  • Unit tests for backends including ejabberd_auth_anonymous, ejabberd_auth_external
  • Migration guide for RDBMS users.
  • Warnings in docs that users number estimates don't support multiple domains and LDAP registration works only for a single domain
  • Experimented with, but not done (not convinced): putting the LUser, LServer, HostType triplet in a map, maybe extended with optional password data.
  • Remove supported_features/0 - why have it when all backends support dynamic domains?

For details, see individual commits.

@codecov
Copy link

codecov bot commented Apr 29, 2021

Codecov Report

Merging #3106 (f4fc9b4) into master (8c597bc) will increase coverage by 0.15%.
The diff coverage is 79.16%.

Impacted file tree graph

@@            Coverage Diff             @@
##           master    #3106      +/-   ##
==========================================
+ Coverage   79.22%   79.38%   +0.15%     
==========================================
  Files         389      390       +1     
  Lines       31959    31936      -23     
==========================================
+ Hits        25321    25352      +31     
+ Misses       6638     6584      -54     
Impacted Files Coverage Δ
src/auth/ejabberd_auth_dummy.erl 63.15% <ø> (+4.82%) ⬆️
src/auth/ejabberd_auth_jwt.erl 77.50% <0.00%> (-6.18%) ⬇️
src/auth/ejabberd_auth_pki.erl 50.00% <0.00%> (+32.35%) ⬆️
src/ejabberd_config.erl 76.84% <ø> (-0.25%) ⬇️
src/mongoose_credentials.erl 93.33% <ø> (ø)
src/mongoose_hooks.erl 91.71% <0.00%> (-0.59%) ⬇️
src/auth/ejabberd_auth_external.erl 29.00% <35.13%> (-0.13%) ⬇️
src/auth/ejabberd_auth_anonymous.erl 53.84% <36.36%> (+5.09%) ⬆️
src/auth/ejabberd_auth_rdbms.erl 55.69% <75.55%> (-2.43%) ⬇️
src/sasl/cyrsasl_digest.erl 74.02% <80.00%> (+2.59%) ⬆️
... and 25 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 8c597bc...f4fc9b4. Read the comment docs.

@leszke leszke force-pushed the multi-tenancy-dynamic-domains-in-auth branch from ee60b1b to aebbbe1 Compare May 5, 2021 14:16
@chrzaszcz chrzaszcz force-pushed the multi-tenancy-dynamic-domains-in-auth branch 4 times, most recently from da50419 to 4ff3765 Compare May 17, 2021 13:18
@chrzaszcz chrzaszcz force-pushed the multi-tenancy-dynamic-domains-in-auth branch 9 times, most recently from a8f0f50 to dab4db4 Compare May 19, 2021 13:52
@chrzaszcz chrzaszcz marked this pull request as ready for review May 19, 2021 14:03
Key differences:
  - Callbacks take host type as the first argument
  - More optional callbacks for less cluttered backend code
      and more unified default values.
  - One 'is_exported' helper to be consistent for all functions.
      'ensure_loaded' is called always as it is a very cheap call
      for already loaded modules.
  - Reworked 'get_registered_users*'
    - Less calls for simplicity
    - Removed the "dirty" call for all users as it is not needed
        (it called ejabberd_auth domain-by-domain in a convoluted way).
    - Removed 'vh' as this infix was confusing.
        There are no other versions of these functions,
        so a 'domain' suffix does not seem necessary.
@chrzaszcz chrzaszcz force-pushed the multi-tenancy-dynamic-domains-in-auth branch from dab4db4 to 546401b Compare May 19, 2021 17:13
chrzaszcz added 10 commits May 20, 2021 08:49
This change prepares the removal of 'dirty_get_registered_users',
  which has the following logic:
  - For each auth backend:
    - For each domain using that backend [1]:
      - Get users for that domain

The new logic does the same, but in a simpler way.

[1] Does not apply to 'internal', which is not recommended anyway.
    We could optimize 'get_registered_users' for Mnesia
    if we ever need to do so (not likely).
This commit prepares the removal of dirty_get_registered_users

- Do the user (count) checks for the two configured domains
- Do the carboncopy check (was unused)
Note: this commit starts a sequence of changes that break tests,
  because the backends are not reworked yet.
  The last commit it the sequence will make them pass again.

- Extract looping through auth modules to a separate function
    - makes the logic more consistent
    - allows easier introduction of host types
- Remove dirty_get_registered_users
- Replace 'does_user_exist_in_other_modules' with a more specific
    'does_stored_user_exist' as it was used only for that purpose.

The API functions are not changed if not necessary
  e.g. the 'vh' infix is not removed (yet)
Also:
  - Simplify the is_protocol_enabled logic
Convert extauth to use host types as well
Group user-related arguments in 'params'.
This approach could be used for backend callbacks as well.
- Keep dirty_get_registered_users for its efficiency.
    It can be used in the Erlang shell if needed.
- Simplify names of 'get_*users*' functions
    Main motivation: code readability
Remove unimplemented optional callbacks.
chrzaszcz added 15 commits May 20, 2021 08:50
- Set/get the state for host types
- Use lserver in dn_filter to support dynamic domains
- Remove unimplemented callbacks and unused functions
- User registration still works only for a single domain (as before)
Store the domain in a new column.
  - Make it the first column in the primary key for queries
      by domain name.
  - The name 'server' is used for consistency.
Bucket type needs to be configured differently for each host type
  to avoid conflicts when dynamic domains are used
  with multiple host types.
This might need to be documented later.
Also:
  The errors for 'authorize' were simplified,
    only 'not_authorized' needs to be handled now.
  This was inconsistent across the sasl modules anyway.
Also: remove tests for deleted functions.

This commit fixes the tests. They should pass from now on.
@chrzaszcz chrzaszcz force-pushed the multi-tenancy-dynamic-domains-in-auth branch from 546401b to f4fc9b4 Compare May 20, 2021 06:52
@chrzaszcz chrzaszcz requested a review from DenysGonchar May 20, 2021 06:56
ok.

-spec remove_domain(mongooseim:host_type(), jid:lserver()) -> ok.
Copy link
Contributor

@gustawlippa gustawlippa May 20, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This handler is present only in the dummy module out of the auth modules, right? Is it an oversight?
[edited - sorry, I made a silly error reading the spec]

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this is not a hook handler. this is ejabberd_auth callback. the hook handler is in ejabberd_auth

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh, sorry, for some reason I misread the spec... Obviously this is fine - I'll edit the comment. But still it's a callback that is present only in the dummy module - is it deliberate?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Handling domain removal has to be added to the TODO list. Doing it now.

Copy link
Collaborator

@DenysGonchar DenysGonchar left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Generally looks good, the only thing that make me worrying that we don't have tests for ejabberd_auth module itself. We should definitely create some unit tests for it with multiple mocked auth methods that return different values to ensure that checks are done until the first successful and in the same sequence as methods configured. Also, that would be nice to ensure that mongoose_gen_auth handles properly missing optional callbacks (can be done in the same ejabberd_auth unit tests). Being honest, I would trust such unit tests more than any code review, because it's a bit hard to follow all the changes.

Approving this PR, but please let's create a ticket for ejabberd_auth unit tests.

@chrzaszcz
Copy link
Member

chrzaszcz commented May 21, 2021

Generally looks good, the only thing that make me worrying that we don't have tests for ejabberd_auth module itself. We should definitely create some unit tests for it with multiple mocked auth methods that return different values to ensure that checks are done until the first successful and in the same sequence as methods configured. Also, that would be nice to ensure that mongoose_gen_auth handles properly missing optional callbacks (can be done in the same ejabberd_auth unit tests). Being honest, I would trust such unit tests more than any code review, because it's a bit hard to follow all the changes.

Approving this PR, but please let's create a ticket for ejabberd_auth unit tests.

I think we could add more tests when the unification of error handling in ejabbed_auth is done. We would need to look at the case with multiple backends configured once again, because IMO it does not make much sense for most backends and the behaviour is very confusing, especially when one backend returns an error. The good thing is that most of the functionality is already covered by functional tests, including the optional callbacks. Added to the TODO list above. I will create tasks from that list when this PR is merged.

@DenysGonchar DenysGonchar merged commit 8e05121 into master May 21, 2021
@DenysGonchar DenysGonchar deleted the multi-tenancy-dynamic-domains-in-auth branch May 21, 2021 06:51
fun check_private/0,
fun check_vcard/0,
fun check_roster/0,
fun check_carboncopy/0],
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@chrzaszcz I've just realised, this is checking an ets table called carboncopy that doesn't really exists in mim 🤔

@Premwoik Premwoik added this to the 5.0.0 milestone Oct 5, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

6 participants