Use maps for mongoose_wpool options #3645
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This comment was marked as outdated.
This comment was marked as outdated.
chrzaszcz
force-pushed
the
pool-config-rework
branch
from
54e69cb to
aa4b9f4
Compare
This comment was marked as outdated.
This comment was marked as outdated.
chrzaszcz
force-pushed
the
pool-config-rework
branch
from
aa4b9f4 to
b75c352
Compare
This comment was marked as outdated.
This comment was marked as outdated.
chrzaszcz
force-pushed
the
pool-config-rework
branch
from
b75c352 to
331b1e8
Compare
This comment was marked as outdated.
This comment was marked as outdated.
Codecov Report
@@ Coverage Diff @@
## master #3645 +/- ##
==========================================
- Coverage 81.07% 81.04% -0.03%
==========================================
Files 428 428
Lines 31942 31912 -30
==========================================
- Hits 25896 25863 -33
- Misses 6046 6049 +3
Continue to review full report at Codecov.
|
This comment was marked as outdated.
This comment was marked as outdated.
This comment was marked as outdated.
This comment was marked as outdated.
chrzaszcz
force-pushed
the
pool-config-rework
branch
from
7628efc to
f1a209c
Compare
This comment was marked as outdated.
This comment was marked as outdated.
chrzaszcz
force-pushed
the
pool-config-rework
branch
from
f1a209c to
d96b115
Compare
This comment was marked as outdated.
This comment was marked as outdated.
chrzaszcz
force-pushed
the
pool-config-rework
branch
from
d96b115 to
ba5df96
Compare
This comment was marked as outdated.
This comment was marked as outdated.
added 3 commits
May 12, 2022 12:33
- Refactor wpool options to use the new utlitity merge_sections/2 - Store connection options in maps - Remove unnecessary conversions and changes of keys - Fix rabbit options - they did not work before, as no 'amqp_' prefixes were expected in the resulting config. - Rename rootdn to root_dn - same as in the resulting options - Add 'max_start_interval' to rdbms - there was code for handling it, but the option was missing from the spec. - Add missing resuired keys for some pools. - Make some implicit defaults explicit. See the following commits for the pool-specific counterparts of this change.
Expect connection options in a map
chrzaszcz
force-pushed
the
pool-config-rework
branch
from
ba5df96 to
5a53ed5
Compare
This comment was marked as outdated.
This comment was marked as outdated.
chrzaszcz
force-pushed
the
pool-config-rework
branch
2 times, most recently
from
118ad94 to
0105e62
Compare
This comment was marked as outdated.
This comment was marked as outdated.
This comment was marked as outdated.
This comment was marked as outdated.
added 4 commits
May 12, 2022 15:07
- Expect connection options and server options in maps - Add type specs: one with generic options and one per backend - Minimize changes in TLS options for now
Expect original keys from the spec: 'host' and 'tls'
Add type specs
- Expect TLS options with the original 'tls' key. - Expect credential in the original format. The previous two-phase processing was too complicated.
added 10 commits
May 12, 2022 15:07
Expect options in the original format (in maps).
Simplify option processing
- Rename 'host' to 'host_type' - Expect options without the 'amqp_' prefixes which actually did not work (there was no conversion)
- Use options directly to initialize state - Get rid of the 'encrypt' option - 'tls' is enough and more consistent with other pools
- Update option format and default values - Replace the custom merging function with a generic recursive one This way any section with subsections can by specified by providing only the extra options.
- Rearrange pool tests to check options from one section in one test case - Add missing tests for elastic - Check wpool options for all pool types (they sometimes change) - Use paths (P ++ [Key]) for all checks - Rework 'compare_nodes' to be more generic and consistent. Only TLS options require unordered list comparisons now.
Use the 'config/2' helper to merge options with defaults.
- Update the format of changed options - Use the config/2 helper to merge options with defaults
- Update the format of changed options - Put ODBS after PGSQL/MySQL, as it is less common - Mention required options explicitly - Add changed options to the migration guide
chrzaszcz
force-pushed
the
pool-config-rework
branch
from
0105e62 to
8316f89
Compare
|
small_tests_24 / small_tests / 8316f89 small_tests_23 / small_tests / 8316f89 dynamic_domains_pgsql_mnesia_24 / pgsql_mnesia / 8316f89 dynamic_domains_pgsql_mnesia_23 / pgsql_mnesia / 8316f89 dynamic_domains_mysql_redis_24 / mysql_redis / 8316f89 muc_SUITE:hibernation:hibernated_room_can_be_queried_for_archive{error,{{assertion_failed,assert,is_groupchat_message,
[<<"Restorable message">>],
undefined,"undefined"},
[{escalus_new_assert,assert_true,2,
[{file,"/home/circleci/project/big_tests/_build/default/lib/escalus/src/escalus_new_assert.erl"},
{line,84}]},
{muc_SUITE,wait_for_mam_result,3,
[{file,"/home/circleci/project/big_tests/tests/muc_SUITE.erl"},
{line,4383}]},
{muc_SUITE,'-hibernated_room_can_be_queried_for_archive/1-fun-0-',3,
[{file,"/home/circleci/project/big_tests/tests/muc_SUITE.erl"},
{line,4124}]},
{escalus_story,story,4,
[{file,"/home/circleci/project/big_tests/_build/default/lib/escalus/src/escalus_story.erl"},
{line,72}]},
{muc_SUITE,hibernated_room_can_be_queried_for_archive,1,
[{file,"/home/circleci/project/big_tests/tests/muc_SUITE.erl"},
{line,4120}]},
{test_server,ts_tc,3,[{file,"test_server.erl"},{line,1783}]},
{test_server,run_test_case_eval1,6,
[{file,"test_server.erl"},{line,1292}]},
{test_server,run_test_case_eval,9,
[{file,"test_server.erl"},{line,1224}]}]}}ldap_mnesia_23 / ldap_mnesia / 8316f89 ldap_mnesia_24 / ldap_mnesia / 8316f89 dynamic_domains_mssql_mnesia_24 / odbc_mssql_mnesia / 8316f89 muc_SUITE:register:user_unregisters_nick_twice{error,
{{assertion_failed,assert,is_iq_result,
[{xmlel,<<"iq">>,
[{<<"type">>,<<"set">>},
{<<"id">>,<<"807e91f4bf603df2fa9e71b1446e62f0">>},
{<<"to">>,<<"groupchats.domain.example.com">>}],
[{xmlel,<<"query">>,
[{<<"xmlns">>,<<"jabber:iq:register">>}],
[{xmlel,<<"x">>,
[{<<"xmlns">>,<<"jabber:x:data">>},
{<<"type">>,<<"submit">>}],
[{xmlel,<<"field">>,
[{<<"type">>,<<"hidden">>},
{<<"var">>,<<"FORM_TYPE">>}],
[{xmlel,<<"value">>,[],
[{xmlcdata,<<"jabber:iq:register">>}]}]},
{xmlel,<<"field">>,
[{<<"type">>,<<"text-single">>},
{<<"var">>,<<"nick">>}],
[{xmlel,<<"value">>,[],
[{xmlcdata,
<<"thirdwitch1room-002f131e00">>}]}]}]}]}]}],
{xmlel,<<"iq">>,
[{<<"from">>,<<"groupchats.domain.example.com">>},
{<<"to">>,
<<"[email protected]/res1">>},
{<<"type">>,<<"error">>},
{<<"xml:lang">>,<<"en">>},
{<<"id">>,<<"807e91f4bf603df2fa9e71b1446e62f0">>}],
[{xmlel,<<"query">>,
[{<<"xmlns">>,<<"jabber:iq:register">>}],
[{xmlel,<<"x">>,
[{<<"xmlns">>,<<"jabber:x:data">>},
{<<"type">>,<<"submit">>}],
[{xmlel,<<"field">>,
[{<<"type">>,<<"hidden">>},
{<<...smart_markers_SUITE:regular:one2one:marker_after_timestamp_for_threadid_can_be_fetched{error,
{timeout_when_waiting_for_stanza,
[{escalus_client,wait_for_stanza,
[{client,
<<"alice_marker_after_timestamp_for_threadid_can_be_fetched_2004@domain.example.com/res1">>,
escalus_tcp,<0.536.2>,
[{event_manager,<0.514.2>},
{server,<<"domain.example.com">>},
{username,
<<"alicE_marker_after_timestamp_for_threadid_can_be_fetched_2004">>},
{resource,<<"res1">>}],
[{event_client,
[{event_manager,<0.514.2>},
{server,<<"domain.example.com">>},
{username,
<<"alicE_marker_after_timestamp_for_threadid_can_be_fetched_2004">>},
{resource,<<"res1">>}]},
{resource,<<"res1">>},
{username,
<<"alice_marker_after_timestamp_for_threadid_can_be_fetched_2004">>},
{server,<<"domain.example.com">>},
{host,<<"localhost">>},
{port,5222},
{auth,{escalus_auth,auth_plain}},
{wspath,undefined},
{username,
<<"alicE_marker_after_timestamp_for_threadid_can_be_fetched_2004">>},
{server,<<"domain.example.com">>},
{host,<<"localhost">>},
{password,<<"matygrysa">>},
{stream_id,<<"718b1a68edb4bef0">>}]},
5000],
[{file,
"/home/circleci/project/big_tests/_build/default/lib/escalus/src/escalus_client.erl"},
{line,136}]},
{smart_markers_SUITE,
'-mark...smart_markers_SUITE:regular:one2one:marker_for_thread_can_be_fetched{error,
{timeout_when_waiting_for_stanza,
[{escalus_client,wait_for_stanza,
[{client,
<<"[email protected]/res1">>,
escalus_tcp,<0.530.2>,
[{event_manager,<0.507.2>},
{server,<<"domain.example.com">>},
{username,
<<"alicE_marker_for_thread_can_be_fetched_2003">>},
{resource,<<"res1">>}],
[{event_client,
[{event_manager,<0.507.2>},
{server,<<"domain.example.com">>},
{username,
<<"alicE_marker_for_thread_can_be_fetched_2003">>},
{resource,<<"res1">>}]},
{resource,<<"res1">>},
{username,
<<"alice_marker_for_thread_can_be_fetched_2003">>},
{server,<<"domain.example.com">>},
{host,<<"localhost">>},
{port,5222},
{auth,{escalus_auth,auth_plain}},
{wspath,undefined},
{username,
<<"alicE_marker_for_thread_can_be_fetched_2003">>},
{server,<<"domain.example.com">>},
{host,<<"localhost">>},
{password,<<"matygrysa">>},
{stream_id,<<"0364204a5e80ee38">>}]},
5000],
[{file,
"/home/circleci/project/big_tests/_build/default/lib/escalus/src/escalus_client.erl"},
{line,136}]},
{smart_markers_SUITE,'-marker_for_thread_can_be_fetched/1-fun-0-',2,
[{file,
"/home/circleci/project/big_tes...internal_mnesia_24 / internal_mnesia / 8316f89 service_mongoose_system_metrics_SUITE:system_metrics_are_reported_to_configurable_google_analytics{error,
{{assertEqual,
[{module,service_mongoose_system_metrics_SUITE},
{line,470},
{expression,"ActualTrackingIds"},
{expected,[<<"UA-151671255-1">>,<<"UA-EXTRA-TRACKING-ID">>]},
{value,[<<"UA-151671255-1">>]}]},
[{service_mongoose_system_metrics_SUITE,
events_are_reported_to_tracking_ids,1,
[{file,
"/home/circleci/project/big_tests/tests/service_mongoose_system_metrics_SUITE.erl"},
{line,470}]},
{service_mongoose_system_metrics_SUITE,
system_metrics_are_reported_to_configurable_google_analytics,1,
[{file,
"/home/circleci/project/big_tests/tests/service_mongoose_system_metrics_SUITE.erl"},
{line,204}]},
{test_server,ts_tc,3,[{file,"test_server.erl"},{line,1783}]},
{test_server,run_test_case_eval1,6,
[{file,"test_server.erl"},{line,1292}]},
{test_server,run_test_case_eval,9,
[{file,"test_server.erl"},{line,1224}]}]}}elasticsearch_and_cassandra_24 / elasticsearch_and_cassandra_mnesia / 8316f89 mysql_redis_24 / mysql_redis / 8316f89 pgsql_mnesia_24 / pgsql_mnesia / 8316f89 pgsql_mnesia_23 / pgsql_mnesia / 8316f89 mssql_mnesia_24 / odbc_mssql_mnesia / 8316f89 pep_SUITE:pep_tests:unsubscribe_after_presence_unsubscription{error,
{{badmatch,
[{xmlel,<<"message">>,
[{<<"from">>,
<<"alice_unsubscribe_after_presence_unsubscription_1929@localhost">>},
{<<"to">>,
<<"bob_unsubscribe_after_presence_unsubscription_1929@localhost/res1">>},
{<<"type">>,<<"headline">>}],
[{xmlel,<<"event">>,
[{<<"xmlns">>,
<<"http://jabber.org/protocol/pubsub#event">>}],
[{xmlel,<<"items">>,
[{<<"node">>,<<"Ue8G5GJXLiy07XpxTCbieA==">>}],
[{xmlel,<<"item">>,
[{<<"id">>,<<"salmon">>}],
[{xmlel,<<"entry">>,
[{<<"xmlns">>,
<<"http://www.w3.org/2005/Atom">>}],
[]}]}]}]},
{xmlel,<<"headers">>,
[{<<"xmlns">>,<<"http://jabber.org/protocol/shim">>}],
[]}]}]},
[{pep_SUITE,'-unsubscribe_after_presence_unsubscription/1-fun-0-',2,
[{file,"/home/circleci/project/big_tests/tests/pep_SUITE.erl"},
{line,384}]},
{escalus_story,story,4,
[{file,
"/home/circleci/project/big_tests/_build/default/lib/escalus/src/escalus_story.erl"},
{line,72}]},
{test_server,ts_tc,3,[{file,"test_server.erl"},{line,1783}]},
{test_server,run_test_case_eval1,6,
[{file,"test_server.erl"},{line,1292}]},
{test_server,run_test_case_eval,9,
[{file,"test_server.erl"},{line,1224}]}]}}pep_SUITE:pep_tests:unsubscribe_after_presence_unsubscription{error,
{{badmatch,
[{xmlel,<<"message">>,
[{<<"from">>,
<<"alice_unsubscribe_after_presence_unsubscription_1943@localhost">>},
{<<"to">>,
<<"bob_unsubscribe_after_presence_unsubscription_1943@localhost/res1">>},
{<<"type">>,<<"headline">>}],
[{xmlel,<<"event">>,
[{<<"xmlns">>,
<<"http://jabber.org/protocol/pubsub#event">>}],
[{xmlel,<<"items">>,
[{<<"node">>,<<"mbJNG6WFpxlSSHpJCZchIA==">>}],
[{xmlel,<<"item">>,
[{<<"id">>,<<"salmon">>}],
[{xmlel,<<"entry">>,
[{<<"xmlns">>,
<<"http://www.w3.org/2005/Atom">>}],
[]}]}]}]},
{xmlel,<<"headers">>,
[{<<"xmlns">>,<<"http://jabber.org/protocol/shim">>}],
[]}]}]},
[{pep_SUITE,'-unsubscribe_after_presence_unsubscription/1-fun-0-',2,
[{file,"/home/circleci/project/big_tests/tests/pep_SUITE.erl"},
{line,384}]},
{escalus_story,story,4,
[{file,
"/home/circleci/project/big_tests/_build/default/lib/escalus/src/escalus_story.erl"},
{line,72}]},
{test_server,ts_tc,3,[{file,"test_server.erl"},{line,1783}]},
{test_server,run_test_case_eval1,6,
[{file,"test_server.erl"},{line,1292}]},
{test_server,run_test_case_eval,9,
[{file,"test_server.erl"},{line,1224}]}]}}riak_mnesia_24 / riak_mnesia / 8316f89 internal_mnesia_24 / internal_mnesia / 8316f89 dynamic_domains_mssql_mnesia_24 / odbc_mssql_mnesia / 8316f89 mssql_mnesia_24 / odbc_mssql_mnesia / 8316f89 |
Premwoik
approved these changes
May 13, 2022
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
The main goal is to keep the configuration options for outgoing pools in maps in the
mongoose_wpool_*modules. The benefit is that they don't have to be converted to an intermediate format and then converted again to the format expected by the specific connection libraries.Other main changes:
amqp_option prefix was not expected bymongoose_amqpcompare_nodeslogic was cleaned up.config/2helper recursive by default, so there is no more need to use custom helpers to merge nested configs with defaults.Omitted on purpose: