Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

CETS backend for mongoose_cluster_id (minimal logic) #4136

Merged
merged 9 commits into from
Oct 6, 2023

Conversation

arcusfelis
Copy link
Contributor

@arcusfelis arcusfelis commented Oct 4, 2023

This PR addresses "MIM-2040 System metrics fail with no_client_id".

This is a copy of #4135 but the difference is that this branch does not try to fix all the tests, that still assume Mnesia is started.

Proposed changes include:

  • CETS backend for mongoose_cluster_id.
  • Added cets:insert_serial/2. See PR for CETS library: Add cets:insert_serial/2 cets#33
  • Actually ensure that Mnesia is stopped after applying preset. This requires some fixes in distributed_helper, because some functions assume Mnesia is always running. - update: we have the code but we disable this ensuring check for now, because it breaks a lot of tests. We will try to fix tests suite-by-suite and enable this check..

@codecov
Copy link

codecov bot commented Oct 4, 2023

Codecov Report

Attention: 2 lines in your changes are missing coverage. Please review.

Comparison is base (c5d0b24) 84.06% compared to head (48f1203) 84.07%.
Report is 34 commits behind head on master.

Additional details and impacted files
@@            Coverage Diff             @@
##           master    #4136      +/-   ##
==========================================
+ Coverage   84.06%   84.07%   +0.01%     
==========================================
  Files         558      559       +1     
  Lines       34016    34035      +19     
==========================================
+ Hits        28595    28616      +21     
+ Misses       5421     5419       -2     
Files Coverage Δ
src/ejabberd_app.erl 89.74% <100.00%> (-0.03%) ⬇️
src/mongoose_cluster_id.erl 78.48% <96.29%> (+1.81%) ⬆️
src/mongoose_internal_databases.erl 90.00% <90.00%> (ø)

... and 9 files with indirect coverage changes

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@arcusfelis arcusfelis force-pushed the cets-for-cluster-id-v3 branch from 359703b to 16e8d99 Compare October 4, 2023 10:55
@esl esl deleted a comment from mongoose-im Oct 4, 2023
@esl esl deleted a comment from mongoose-im Oct 4, 2023
@arcusfelis arcusfelis force-pushed the cets-for-cluster-id-v3 branch from 16e8d99 to 7e3de03 Compare October 4, 2023 12:57
@mongoose-im

This comment was marked as outdated.

@arcusfelis arcusfelis force-pushed the cets-for-cluster-id-v3 branch from 7e3de03 to 3affe01 Compare October 4, 2023 13:23
@mongoose-im

This comment was marked as outdated.

@mongoose-im

This comment was marked as outdated.

@mongoose-im
Copy link
Collaborator

mongoose-im commented Oct 4, 2023

elasticsearch_and_cassandra_25 / elasticsearch_and_cassandra_mnesia / ffc78a8
Reports root/ big
OK: 369 / Failed: 0 / User-skipped: 38 / Auto-skipped: 0


small_tests_24 / small_tests / ffc78a8
Reports root / small


small_tests_25 / small_tests / ffc78a8
Reports root / small


small_tests_25_arm64 / small_tests / ffc78a8
Reports root / small


ldap_mnesia_24 / ldap_mnesia / ffc78a8
Reports root/ big
OK: 2303 / Failed: 0 / User-skipped: 865 / Auto-skipped: 0


dynamic_domains_mysql_redis_25 / mysql_redis / ffc78a8
Reports root/ big
OK: 4236 / Failed: 0 / User-skipped: 144 / Auto-skipped: 0


ldap_mnesia_25 / ldap_mnesia / ffc78a8
Reports root/ big
OK: 2303 / Failed: 0 / User-skipped: 865 / Auto-skipped: 0


dynamic_domains_pgsql_mnesia_24 / pgsql_mnesia / ffc78a8
Reports root/ big
OK: 4268 / Failed: 0 / User-skipped: 112 / Auto-skipped: 0


dynamic_domains_mssql_mnesia_25 / odbc_mssql_mnesia / ffc78a8
Reports root/ big
OK: 4265 / Failed: 0 / User-skipped: 115 / Auto-skipped: 0


pgsql_cets_25 / pgsql_cets / ffc78a8
Reports root/ big
OK: 4655 / Failed: 0 / User-skipped: 121 / Auto-skipped: 0


internal_mnesia_25 / internal_mnesia / ffc78a8
Reports root/ big
OK: 2453 / Failed: 0 / User-skipped: 715 / Auto-skipped: 0


mysql_redis_25 / mysql_redis / ffc78a8
Reports root/ big
OK: 4637 / Failed: 0 / User-skipped: 139 / Auto-skipped: 0


dynamic_domains_pgsql_mnesia_25 / pgsql_mnesia / ffc78a8
Reports root/ big
OK: 4268 / Failed: 0 / User-skipped: 112 / Auto-skipped: 0


pgsql_mnesia_24 / pgsql_mnesia / ffc78a8
Reports root/ big
OK: 4657 / Failed: 0 / User-skipped: 119 / Auto-skipped: 0


pgsql_mnesia_25 / pgsql_mnesia / ffc78a8
Reports root/ big
OK: 4657 / Failed: 0 / User-skipped: 119 / Auto-skipped: 0


mssql_mnesia_25 / odbc_mssql_mnesia / ffc78a8
Reports root/ big
OK: 4653 / Failed: 1 / User-skipped: 122 / Auto-skipped: 0

muc_SUITE:register:user_changes_nick
{error,
  {{assertion_failed,assert,is_iq_result,
     [{xmlel,<<"iq">>,
        [{<<"type">>,<<"set">>},
         {<<"id">>,<<"00f3c54b89751252fc631d7268d6c9ac">>},
         {<<"to">>,<<"muc.localhost">>}],
        [{xmlel,<<"query">>,
           [{<<"xmlns">>,<<"jabber:iq:register">>}],
           [{xmlel,<<"x">>,
            [{<<"xmlns">>,<<"jabber:x:data">>},
             {<<"type">>,<<"submit">>}],
            [{xmlel,<<"field">>,
               [{<<"type">>,<<"hidden">>},
                {<<"var">>,<<"FORM_TYPE">>}],
               [{xmlel,<<"value">>,[],
                  [{xmlcdata,<<"jabber:iq:register">>}]}]},
             {xmlel,<<"field">>,
               [{<<"type">>,<<"text-single">>},
                {<<"var">>,<<"nick">>}],
               [{xmlel,<<"value">>,[],
                  [{xmlcdata,
                     <<"thirdwitch1room-1f225b4026">>}]}]}]}]}]}],
     {xmlel,<<"iq">>,
       [{<<"from">>,<<"muc.localhost">>},
        {<<"to">>,<<"alice_user_changes_nick_2645@localhost/res1">>},
        {<<"type">>,<<"error">>},
        {<<"id">>,<<"00f3c54b89751252fc631d7268d6c9ac">>}],
       [{xmlel,<<"query">>,
          [{<<"xmlns">>,<<"jabber:iq:register">>}],
          [{xmlel,<<"x">>,
             [{<<"xmlns">>,<<"jabber:x:data">>},
            {<<"type">>,<<"submit">>}],
             [{xmlel,<<"field">>,
              [{<<"type">>,<<"hidden">>},
               {<<"var">>,<<"FORM_TYPE">>}],
              [{xmlel,<<"value">>,[],
                 [{xmlcdata,<<"...

Report log


mssql_mnesia_25 / odbc_mssql_mnesia / ffc78a8
Reports root/ big
OK: 4653 / Failed: 1 / User-skipped: 122 / Auto-skipped: 0

graphql_server_SUITE:admin_cli:clustering_tests:join_successful
{error,{#{expected_type => ok,
      response_code => {exit_status,1},
      what => invalid_response_code},
    [{graphql_helper,assert_response_code,2,
             [{file,"/home/circleci/project/big_tests/tests/graphql_helper.erl"},
              {line,256}]},
     {graphql_helper,get_ok_value,2,
             [{file,"/home/circleci/project/big_tests/tests/graphql_helper.erl"},
              {line,239}]},
     {graphql_server_SUITE,join_successful,1,
                 [{file,"/home/circleci/project/big_tests/tests/graphql_server_SUITE.erl"},
                {line,150}]},
     {test_server,ts_tc,3,[{file,"test_server.erl"},{line,1782}]},
     {test_server,run_test_case_eval1,6,
            [{file,"test_server.erl"},{line,1291}]},
     {test_server,run_test_case_eval,9,
            [{file,"test_server.erl"},{line,1223}]}]}}

Report log


mssql_mnesia_25 / odbc_mssql_mnesia / ffc78a8
Reports root/ big
OK: 4654 / Failed: 0 / User-skipped: 122 / Auto-skipped: 0

@arcusfelis arcusfelis marked this pull request as ready for review October 4, 2023 16:07
@arcusfelis arcusfelis requested a review from chrzaszcz October 4, 2023 16:08
Copy link
Member

@chrzaszcz chrzaszcz left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks ok in general, I added some comments.

src/mongoose_internal_databases.erl Outdated Show resolved Hide resolved
src/mongoose_internal_databases.erl Outdated Show resolved Hide resolved
big_tests/tests/mongoose_sanity_checks_SUITE.erl Outdated Show resolved Hide resolved
src/mongoose_internal_databases.erl Outdated Show resolved Hide resolved
@arcusfelis arcusfelis force-pushed the cets-for-cluster-id-v3 branch from ffc78a8 to 48f1203 Compare October 6, 2023 11:04
@mongoose-im
Copy link
Collaborator

mongoose-im commented Oct 6, 2023

small_tests_25_arm64 / small_tests / 48f1203
Reports root / small


elasticsearch_and_cassandra_25 / elasticsearch_and_cassandra_mnesia / 48f1203
Reports root/ big
OK: 369 / Failed: 0 / User-skipped: 38 / Auto-skipped: 0


small_tests_24 / small_tests / 48f1203
Reports root / small


small_tests_25 / small_tests / 48f1203
Reports root / small


ldap_mnesia_24 / ldap_mnesia / 48f1203
Reports root/ big
OK: 2303 / Failed: 0 / User-skipped: 865 / Auto-skipped: 0


dynamic_domains_pgsql_mnesia_24 / pgsql_mnesia / 48f1203
Reports root/ big
OK: 4268 / Failed: 0 / User-skipped: 112 / Auto-skipped: 0


dynamic_domains_mysql_redis_25 / mysql_redis / 48f1203
Reports root/ big
OK: 4236 / Failed: 0 / User-skipped: 144 / Auto-skipped: 0


ldap_mnesia_25 / ldap_mnesia / 48f1203
Reports root/ big
OK: 2303 / Failed: 0 / User-skipped: 865 / Auto-skipped: 0


dynamic_domains_pgsql_mnesia_25 / pgsql_mnesia / 48f1203
Reports root/ big
OK: 4268 / Failed: 0 / User-skipped: 112 / Auto-skipped: 0


pgsql_mnesia_24 / pgsql_mnesia / 48f1203
Reports root/ big
OK: 4657 / Failed: 0 / User-skipped: 119 / Auto-skipped: 0


internal_mnesia_25 / internal_mnesia / 48f1203
Reports root/ big
OK: 2453 / Failed: 0 / User-skipped: 715 / Auto-skipped: 0


dynamic_domains_mssql_mnesia_25 / odbc_mssql_mnesia / 48f1203
Reports root/ big
OK: 4265 / Failed: 0 / User-skipped: 115 / Auto-skipped: 0


pgsql_cets_25 / pgsql_cets / 48f1203
Reports root/ big
OK: 4655 / Failed: 0 / User-skipped: 121 / Auto-skipped: 0


mssql_mnesia_25 / odbc_mssql_mnesia / 48f1203
Reports root/ big
OK: 4654 / Failed: 0 / User-skipped: 122 / Auto-skipped: 0


mysql_redis_25 / mysql_redis / 48f1203
Reports root/ big
OK: 4637 / Failed: 0 / User-skipped: 139 / Auto-skipped: 0


pgsql_mnesia_25 / pgsql_mnesia / 48f1203
Reports root/ big
OK: 4657 / Failed: 0 / User-skipped: 119 / Auto-skipped: 0

@arcusfelis arcusfelis requested a review from chrzaszcz October 6, 2023 12:52
Copy link
Member

@chrzaszcz chrzaszcz left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👌

@chrzaszcz chrzaszcz merged commit 9bbbbe2 into master Oct 6, 2023
@chrzaszcz chrzaszcz deleted the cets-for-cluster-id-v3 branch October 6, 2023 13:04
@chrzaszcz chrzaszcz added this to the 6.2.0 milestone Dec 11, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants