Pools/per host config #4235

NelsonVides · 2024-03-05T14:35:25Z

What I wanted to have was to allow to have a default global pool, and another default for a single_host_type for example. But the current TOML config doesn’t let me have two pools tagged default. Allow for this.

codecov · 2024-03-05T14:45:05Z

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 84.41%. Comparing base (fd0720d) to head (daf4c01).

Additional details and impacted files

@@            Coverage Diff             @@
##           master    #4235      +/-   ##
==========================================
+ Coverage   84.00%   84.41%   +0.40%     
==========================================
  Files         552      552              
  Lines       33531    33538       +7     
==========================================
+ Hits        28169    28312     +143     
+ Misses       5362     5226     -136

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

mongoose-im · 2024-03-06T10:06:46Z

elasticsearch_and_cassandra_26 / elasticsearch_and_cassandra_mnesia / 075d957
Reports root/ big
OK: 429 / Failed: 0 / User-skipped: 43 / Auto-skipped: 0

small_tests_25 / small_tests / 075d957
Reports root / small

small_tests_26 / small_tests / 075d957
Reports root / small

small_tests_26_arm64 / small_tests / 075d957
Reports root / small

ldap_mnesia_25 / ldap_mnesia / 075d957
Reports root/ big
OK: 2274 / Failed: 1 / User-skipped: 895 / Auto-skipped: 0

pubsub_SUITE:tree+basic:publish_with_max_items_test

{error,{{badmatch,false},
    [{pubsub_tools,check_response,2,
             [{file,"/home/circleci/project/big_tests/tests/pubsub_tools.erl"},
            {line,444}]},
     {pubsub_tools,receive_response,3,
             [{file,"/home/circleci/project/big_tests/tests/pubsub_tools.erl"},
            {line,434}]},
     {pubsub_tools,receive_and_check_response,4,
             [{file,"/home/circleci/project/big_tests/tests/pubsub_tools.erl"},
            {line,424}]},
     {pubsub_SUITE,'-publish_with_max_items_test/1-fun-0-',2,
             [{file,"/home/circleci/project/big_tests/tests/pubsub_SUITE.erl"},
            {line,457}]},
     {escalus_story,story,4,
            [{file,"/home/circleci/project/big_tests/_build/default/lib/escalus/src/escalus_story.erl"},
             {line,72}]},
     {test_server,ts_tc,3,[{file,"test_server.erl"},{line,1782}]},
     {test_server,run_test_case_eval1,6,
            [{file,"test_server.erl"},{line,1291}]},
     {test_server,run_test_case_eval,9,
            [{file,"test_server.erl"},{line,1223}]}]}}

Report log

ldap_mnesia_26 / ldap_mnesia / 075d957
Reports root/ big
OK: 2275 / Failed: 0 / User-skipped: 895 / Auto-skipped: 0

dynamic_domains_mysql_redis_26 / mysql_redis / 075d957
Reports root/ big
OK: 4469 / Failed: 0 / User-skipped: 144 / Auto-skipped: 0

dynamic_domains_pgsql_mnesia_26 / pgsql_mnesia / 075d957
Reports root/ big
OK: 4502 / Failed: 0 / User-skipped: 111 / Auto-skipped: 0

internal_mnesia_26 / internal_mnesia / 075d957
Reports root/ big
OK: 2415 / Failed: 0 / User-skipped: 755 / Auto-skipped: 0

pgsql_cets_26 / pgsql_cets / 075d957
Reports root/ big
OK: 4425 / Failed: 0 / User-skipped: 178 / Auto-skipped: 0

dynamic_domains_mssql_mnesia_26 / odbc_mssql_mnesia / 075d957
Reports root/ big
OK: 4499 / Failed: 0 / User-skipped: 114 / Auto-skipped: 0

graphql_muc_light_SUITE:domain_admin:domain_admin_muc_light:end_per_group

{error,
 {{unregistering_failed,
   {amount,1},
   {unregistered_items,
  [{{<<"_domain_admin_invite_user_no_permission_1134">>,
     [{escalus_event_mgr,<0.24794.0>},
    {tc_name,domain_admin_invite_user_no_permission},
    {escalus_cleaner,<0.24793.0>},
    {watchdog,<0.24792.0>},
    {muc_light_host,<<"muclight.domain.example.com">>},
    {secondary_muc_light_host,<<"muclight.domain.example.org">>},
    {protocol,http},
    {domain_admin,{<<"[email protected]">>,<<"bb2c31151bc3f850">>}},
    {schema_endpoint,domain_admin},
    {{ejabberd_cwd,mongooseim@localhost},
     "/home/circleci/project/_build/mim1/rel/mongooseim"},
    {preset,"odbc_mssql_mnesia"},
    {mim_data_dir,
     "/home/circleci/project/big_tests/tests/graphql_muc_light_SUITE_data"},
    {tc_logfile,
     "https://circleci-mim-results.s3.eu-central-1.amazonaws.com/PR/4235/210619/odbc_mssql_mnesia.26.1.2-amd64/big/ct_run.test%40b7b04114ea57.2024-03-06_10.02.16/big_tests.tests.graphql_muc_light_SUITE.logs/run.2024-03-06_10.06.52/graphql_muc_light_suite.domain_admin_invite_user_no_permission.html"},
    {tc_group_properties,[{name,domain_admin_muc_light}]},
    {tc_group_path,[[{name,domain_admin}]]},
    {data_dir,
     "/home/circleci/project/big_tests/_build/default/lib/mongoose_tests/ebin/graphql_muc_light_SUITE_data/"},
    {priv_dir,
     "https://circleci-mim-results.s3.eu-central-1.amazonaws.com/PR/4235/210619/odbc_mssql_mnesia.26.1.2-amd64/big/ct_run.test%40b7b04114ea57.2024-03-06_10.02.16/big_tests.tests.graphql_muc_light_SUITE.logs/run.2024-03-06_10.06.52/log_private/"},
    {{saved_modules,mongooseim@localhost,<<"test type"...

Report log

pgsql_mnesia_25 / pgsql_mnesia / 075d957
Reports root/ big
OK: 4891 / Failed: 0 / User-skipped: 118 / Auto-skipped: 0

mysql_redis_26 / mysql_redis / 075d957
Reports root/ big
OK: 4870 / Failed: 0 / User-skipped: 139 / Auto-skipped: 0

mssql_mnesia_26 / odbc_mssql_mnesia / 075d957
Reports root/ big
OK: 4887 / Failed: 1 / User-skipped: 121 / Auto-skipped: 0

graphql_muc_light_SUITE:admin_http:admin_muc_light:end_per_group

{error,
 {{unregistering_failed,
   {amount,1},
   {unregistered_items,
  [{{<<"_admin_create_room_with_custom_fields_1038">>,
     [{escalus_event_mgr,<0.23673.0>},
    {tc_name,admin_create_room_with_custom_fields},
    {escalus_cleaner,<0.23672.0>},
    {watchdog,<0.23671.0>},
    {muc_light_host,<<"muclight.localhost">>},
    {secondary_muc_light_host,<<"muclight.localhost.bis">>},
    {protocol,http},
    {schema_endpoint,admin},
    {listener_opts,
     #{module => mongoose_graphql_handler,path => "/api/graphql",
       host => "localhost",username => <<"admin">>,
       password => <<"secret">>,schema_endpoint => admin,
       sse_idle_timeout => 3600000}},
    {{ejabberd_cwd,mongooseim@localhost},
     "/home/circleci/project/_build/mim1/rel/mongooseim"},
    {preset,"odbc_mssql_mnesia"},
    {mim_data_dir,
     "/home/circleci/project/big_tests/tests/graphql_muc_light_SUITE_data"},
    {tc_logfile,
     "https://circleci-mim-results.s3.eu-central-1.amazonaws.com/PR/4235/210622/odbc_mssql_mnesia.26.1.2-amd64/big/ct_run.test%40bd21dcf2cd68.2024-03-06_10.02.30/big_tests.tests.graphql_muc_light_SUITE.logs/run.2024-03-06_10.06.58/graphql_muc_light_suite.admin_create_room_with_custom_fields.html"},
    {tc_group_properties,[{name,admin_muc_light}]},
    {tc_group_path,[[{name,admin_http}]]},
    {data_dir,
     "/home/circleci/project/big_tests/_build/default/lib/mongoose_tests/ebin/graphql_muc_light_SUITE_data/"},
    {priv_dir,
     "https://circleci-mim-results.s3.eu-central-1.amazonaws.com/PR/4235/210622/odbc_mssql_mnesia.26.1.2-amd64/big/ct_run.test%40bd21dcf2cd68.2024-03-06_10.02.30/big_tests.tests.g...

Report log

sm_SUITE:ping_timeout

{error,
  {{assertion_failed,assert,is_presence,
     {xmlel,<<"iq">>,
       [{<<"from">>,<<"localhost">>},
        {<<"to">>,
         <<"alice_ping_timeout_3513@localhost/escalus-default-resource">>},
        {<<"type">>,<<"get">>},
        {<<"id">>,<<"a4ff87edcb7e1fcc">>}],
       [{xmlel,<<"ping">>,[{<<"xmlns">>,<<"urn:xmpp:ping">>}],[]}]},
     "<iq from='localhost' to='alice_ping_timeout_3513@localhost/escalus-default-resource' type='get' id='a4ff87edcb7e1fcc'><ping xmlns='urn:xmpp:ping'/></iq>"},
   [{escalus_new_assert,assert_true,2,
      [{file,
         "/home/circleci/project/big_tests/_build/default/lib/escalus/src/escalus_new_assert.erl"},
       {line,84}]},
    {sm_helper,initial_presence_step,2,
      [{file,"/home/circleci/project/big_tests/tests/sm_helper.erl"},
       {line,135}]},
    {escalus_connection,connection_step,2,
      [{file,
         "/home/circleci/project/big_tests/_build/default/lib/escalus/src/escalus_connection.erl"},
       {line,163}]},
    {lists,foldl_1,3,[{file,"lists.erl"},{line,1599}]},
    {escalus_connection,start,2,
      [{file,
         "/home/circleci/project/big_tests/_build/default/lib/escalus/src/escalus_connection.erl"},
       {line,145}]},
    {sm_helper,connect_spec,3,
      [{file,"/home/circleci/project/big_tests/tests/sm_helper.erl"},
       {line,153}]},
    {sm_SUITE,ping_timeout,1,
      [{file,"/home/circleci/project/big_tests/tests/sm_SUITE.erl"},
       {line,625}]},
    {test_server,ts_tc,3,[{file,"test_ser...

Report log

pgsql_mnesia_26 / pgsql_mnesia / 075d957
Reports root/ big
OK: 4891 / Failed: 0 / User-skipped: 118 / Auto-skipped: 0

ldap_mnesia_25 / ldap_mnesia / 075d957
Reports root/ big
OK: 2275 / Failed: 0 / User-skipped: 895 / Auto-skipped: 0

mssql_mnesia_26 / odbc_mssql_mnesia / 075d957
Reports root/ big
OK: 4888 / Failed: 0 / User-skipped: 121 / Auto-skipped: 0

chrzaszcz

It's going in the right direction. I added some comments.

src/config/mongoose_config_spec.erl

src/wpool/mongoose_wpool.erl

test/common/config_parser_helper.erl

test/config_parser_SUITE.erl

mongoose-im · 2024-03-07T14:48:20Z

elasticsearch_and_cassandra_26 / elasticsearch_and_cassandra_mnesia / daf4c01
Reports root/ big
OK: 429 / Failed: 0 / User-skipped: 43 / Auto-skipped: 0

small_tests_25 / small_tests / daf4c01
Reports root / small

small_tests_26 / small_tests / daf4c01
Reports root / small

small_tests_26_arm64 / small_tests / daf4c01
Reports root / small

ldap_mnesia_25 / ldap_mnesia / daf4c01
Reports root/ big
OK: 2275 / Failed: 0 / User-skipped: 895 / Auto-skipped: 0

ldap_mnesia_26 / ldap_mnesia / daf4c01
Reports root/ big
OK: 2275 / Failed: 0 / User-skipped: 895 / Auto-skipped: 0

dynamic_domains_pgsql_mnesia_25 / pgsql_mnesia / daf4c01
Reports root/ big
OK: 4502 / Failed: 0 / User-skipped: 111 / Auto-skipped: 0

dynamic_domains_pgsql_mnesia_26 / pgsql_mnesia / daf4c01
Reports root/ big
OK: 4502 / Failed: 0 / User-skipped: 111 / Auto-skipped: 0

dynamic_domains_mysql_redis_26 / mysql_redis / daf4c01
Reports root/ big
OK: 4469 / Failed: 0 / User-skipped: 144 / Auto-skipped: 0

internal_mnesia_26 / internal_mnesia / daf4c01
Reports root/ big
OK: 2415 / Failed: 0 / User-skipped: 755 / Auto-skipped: 0

dynamic_domains_mssql_mnesia_26 / odbc_mssql_mnesia / daf4c01
Reports root/ big
OK: 4499 / Failed: 0 / User-skipped: 114 / Auto-skipped: 0

graphql_muc_light_SUITE:domain_admin:domain_admin_muc_light:end_per_group

{error,
 {{unregistering_failed,
   {amount,1},
   {unregistered_items,
  [{{<<"_domain_admin_kick_user_no_permission_1154">>,
     [{escalus_event_mgr,<0.24900.0>},
    {tc_name,domain_admin_kick_user_no_permission},
    {escalus_cleaner,<0.24899.0>},
    {watchdog,<0.24898.0>},
    {muc_light_host,<<"muclight.domain.example.com">>},
    {secondary_muc_light_host,<<"muclight.domain.example.org">>},
    {protocol,http},
    {domain_admin,{<<"[email protected]">>,<<"9dbe38b735894f79">>}},
    {schema_endpoint,domain_admin},
    {{ejabberd_cwd,mongooseim@localhost},
     "/home/circleci/project/_build/mim1/rel/mongooseim"},
    {preset,"odbc_mssql_mnesia"},
    {mim_data_dir,
     "/home/circleci/project/big_tests/tests/graphql_muc_light_SUITE_data"},
    {tc_logfile,
     "https://circleci-mim-results.s3.eu-central-1.amazonaws.com/PR/4235/210883/odbc_mssql_mnesia.26.1.2-amd64/big/ct_run.test%40358d8b5e1cf0.2024-03-07_14.44.39/big_tests.tests.graphql_muc_light_SUITE.logs/run.2024-03-07_14.49.04/graphql_muc_light_suite.domain_admin_kick_user_no_permission.html"},
    {tc_group_properties,[{name,domain_admin_muc_light}]},
    {tc_group_path,[[{name,domain_admin}]]},
    {data_dir,
     "/home/circleci/project/big_tests/_build/default/lib/mongoose_tests/ebin/graphql_muc_light_SUITE_data/"},
    {priv_dir,
     "https://circleci-mim-results.s3.eu-central-1.amazonaws.com/PR/4235/210883/odbc_mssql_mnesia.26.1.2-amd64/big/ct_run.test%40358d8b5e1cf0.2024-03-07_14.44.39/big_tests.tests.graphql_muc_light_SUITE.logs/run.2024-03-07_14.49.04/log_private/"},
    {{saved_modules,mongooseim@localhost,<<"test type">>},
 ...

Report log

pgsql_cets_26 / pgsql_cets / daf4c01
Reports root/ big
OK: 4425 / Failed: 0 / User-skipped: 178 / Auto-skipped: 0

pgsql_mnesia_25 / pgsql_mnesia / daf4c01
Reports root/ big
OK: 4891 / Failed: 0 / User-skipped: 118 / Auto-skipped: 0

mysql_redis_26 / mysql_redis / daf4c01
Reports root/ big
OK: 4870 / Failed: 0 / User-skipped: 139 / Auto-skipped: 0

pgsql_mnesia_26 / pgsql_mnesia / daf4c01
Reports root/ big
OK: 4891 / Failed: 0 / User-skipped: 118 / Auto-skipped: 0

mssql_mnesia_26 / odbc_mssql_mnesia / daf4c01
Reports root/ big
OK: 4888 / Failed: 0 / User-skipped: 121 / Auto-skipped: 0

graphql_muc_light_SUITE:domain_admin:domain_admin_muc_light:end_per_group

{error,
 {{unregistering_failed,
   {amount,1},
   {unregistered_items,
  [{{<<"_domain_admin_kick_user_no_permission_1112">>,
     [{escalus_event_mgr,<0.25115.0>},
    {tc_name,domain_admin_kick_user_no_permission},
    {escalus_cleaner,<0.25114.0>},
    {watchdog,<0.25113.0>},
    {muc_light_host,<<"muclight.localhost">>},
    {secondary_muc_light_host,<<"muclight.localhost.bis">>},
    {protocol,http},
    {domain_admin,{<<"admin@localhost">>,<<"f722cbcbf8abfd7d">>}},
    {schema_endpoint,domain_admin},
    {{ejabberd_cwd,mongooseim@localhost},
     "/home/circleci/project/_build/mim1/rel/mongooseim"},
    {preset,"odbc_mssql_mnesia"},
    {mim_data_dir,
     "/home/circleci/project/big_tests/tests/graphql_muc_light_SUITE_data"},
    {tc_logfile,
     "https://circleci-mim-results.s3.eu-central-1.amazonaws.com/PR/4235/210876/odbc_mssql_mnesia.26.1.2-amd64/big/ct_run.test%40d03d93714ba1.2024-03-07_14.44.39/big_tests.tests.graphql_muc_light_SUITE.logs/run.2024-03-07_14.49.12/graphql_muc_light_suite.domain_admin_kick_user_no_permission.html"},
    {tc_group_properties,[{name,domain_admin_muc_light}]},
    {tc_group_path,[[{name,domain_admin}]]},
    {data_dir,
     "/home/circleci/project/big_tests/_build/default/lib/mongoose_tests/ebin/graphql_muc_light_SUITE_data/"},
    {priv_dir,
     "https://circleci-mim-results.s3.eu-central-1.amazonaws.com/PR/4235/210876/odbc_mssql_mnesia.26.1.2-amd64/big/ct_run.test%40d03d93714ba1.2024-03-07_14.44.39/big_tests.tests.graphql_muc_light_SUITE.logs/run.2024-03-07_14.49.12/log_private/"},
    {{saved_modules,mongooseim@localhost,<<"localhost">>},
     #{mod_vcard =>
    ...

Report log

small_tests_25 / small_tests / daf4c01
Reports root / small

chrzaszcz

Looks good 👍

NelsonVides added 3 commits March 4, 2024 12:06

Simplify atom-binary conversions in config_spec listeners

26f9249

Extract list of scopes

dcf0cd7

Cosmetic spacing in mongoose_wpool

aef7a94

This comment was marked as outdated.

Sign in to view

Verify expanding pools before any side-effects are executed

9f45375

NelsonVides force-pushed the pools/per_host_config branch from a964a41 to 80acbf1 Compare March 6, 2024 08:53

This comment was marked as outdated.

Sign in to view

NelsonVides force-pushed the pools/per_host_config branch from 80acbf1 to 191851c Compare March 6, 2024 09:32

This comment was marked as outdated.

Sign in to view

NelsonVides added 3 commits March 6, 2024 10:58

Move pools single_host opt to host_config sections

922092d

Remove single_host(_type) option in favour of host_config

4ca0440

Expand pools taking new host_config ones

075d957

NelsonVides force-pushed the pools/per_host_config branch from 191851c to 075d957 Compare March 6, 2024 09:58

NelsonVides marked this pull request as ready for review March 6, 2024 11:06

NelsonVides mentioned this pull request Mar 6, 2024

Pools/list #4232

Closed

chrzaszcz reviewed Mar 6, 2024

View reviewed changes

Apply review comments

daf4c01

chrzaszcz approved these changes Mar 8, 2024

View reviewed changes

chrzaszcz merged commit e3e9381 into master Mar 8, 2024
4 checks passed

chrzaszcz deleted the pools/per_host_config branch March 8, 2024 08:50

jacekwegr added this to the 6.2.1 milestone Apr 3, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Pools/per host config #4235

Pools/per host config #4235

NelsonVides commented Mar 5, 2024

This comment was marked as outdated.

codecov bot commented Mar 5, 2024 •

edited

Loading

This comment was marked as outdated.

This comment was marked as outdated.

mongoose-im commented Mar 6, 2024 •

edited

Loading

chrzaszcz left a comment

mongoose-im commented Mar 7, 2024 •

edited

Loading

chrzaszcz left a comment

Pools/per host config #4235

Pools/per host config #4235

Conversation

NelsonVides commented Mar 5, 2024

This comment was marked as outdated.

codecov bot commented Mar 5, 2024 • edited Loading

Codecov Report

This comment was marked as outdated.

This comment was marked as outdated.

mongoose-im commented Mar 6, 2024 • edited Loading

chrzaszcz left a comment

Choose a reason for hiding this comment

mongoose-im commented Mar 7, 2024 • edited Loading

chrzaszcz left a comment

Choose a reason for hiding this comment

codecov bot commented Mar 5, 2024 •

edited

Loading

mongoose-im commented Mar 6, 2024 •

edited

Loading

mongoose-im commented Mar 7, 2024 •

edited

Loading