Update hook docs

doc/developers-guide/Hooks-and-handlers.md

@@ -29,7 +29,8 @@ To avoid that coupling and also to enable other ([possibly yet to be written](#s
 mongoose_hooks:offline_message_hook(Acc, From, To, Packet);
 ```
 
-`mongoose_hooks` is a module which serves as an API for calling all hooks in the server.
+`mongoose_hooks` is a module which serves as an API for calling hooks in the server. All such modules are placed in `src/hooks`.
+
 For every hook, there needs to be a function in this module written beforehand which accepts the correct arity of arguments and makes the call to actual low-level hooks mechanism.
 This means that there is some degree of coupling still - but this time between the `ejabberd_sm` module and `mongoose_hooks`, and the latter is always available.
 
@@ -46,10 +47,13 @@ This depends on which handlers are registered to process the event.
 
     This was actually the case before this module was introduced, and hooks' names were just atoms provided as an argument to this low-level API.
     However, we discovered it was causing problems and producing bugs, due to the lack of static code analysis.
-    Now we can have some guarantees thanks to Dialyzer, and each hook invocation has correct number of arguments.
+    Now we can have some guarantees thanks to Dialyzer, and each hook invocation has a correct number of arguments.
     Thanks to this, writing handlers is easier - there is a single source of truth about how a hook is run.
     Remember that a given hook can be invoked from many places in many modules.
 
+    With the new `mongoose_c2s` implementation we introduced a new hook API module, `mongoose_c2s_hooks`.
+    All such API modules are placed in the `src/hooks` directory.
+
 ### Getting results from handlers
 
 Hook handlers are called by "folding".
@@ -73,7 +77,7 @@ The initial value of the accumulator being passed through the sequence of handle
 ### Using accumulators
 
 MongooseIM uses a dedicated data structure to accumulate data related to stanza processing (see ["Accumulators"](accumulators.md)).
-It is instantiated with an incoming stanza, passed along throughout the processing chain, supplied to and returned from certain hook calls, and terminated when stanza is leaving MongooseIM.
+It is instantiated with an incoming stanza, passed along throughout the processing chain, supplied to and returned from certain hook calls, and terminated when the stanza is leaving MongooseIM.
 There are some hooks which don't use this data structure.
 
 If a Mongoose accumulator is passed to a hook, handlers should store their return values in one of 3 ways:
@@ -82,10 +86,10 @@ If a Mongoose accumulator is passed to a hook, handlers should store their retur
 * If the value is to be passed on to be reused within the current processing context, use `mongoose_acc:set(Namespace, Key, Value, Acc)`.
 * If the value should be passed on to the recipient's session, pubsub node etc. use `mongoose_acc:set_permanent(Namespace, Key, Value, Acc)`.
 
-A real life example, then, with regard to `mod_offline` is the `resend_offline_messages_hook` run in `ejabberd_c2s`:
+A real life example, then, with regard to `mod_offline` is the `resend_offline_messages_hook` run in `mod_presence`:
 
 ```erlang
-Acc1 = mongoose_hooks:resend_offline_messages_hook(Acc, StateData#state.jid),
+Acc1 = mongoose_hooks:resend_offline_messages_hook(Acc, Jid),
 Rs = mongoose_acc:get(offline, messages, [], Acc1),
 ```
 
@@ -94,7 +98,7 @@ Rs = mongoose_acc:get(offline, messages, [], Acc1),
 Hooks are meant to decouple modules; in other words, the caller signals that some event took place or that it intends to use a certain feature or a set of features, but how and if those features are implemented is beyond its interest.
 For that reason hooks don't use the "let it crash" approach. Instead, it is rather like "fire-and-forget", more similar in principle to the `Pid ! signal` way.
 
-In practical terms: if a handler throws an error the hook machine logs a message and proceeds to the next handler with an unmodified accumulator.
+In practical terms: if a handler throws an error, the hook machine logs a message and proceeds to the next handler with an unmodified accumulator.
 If there are no handlers registered for a given hook, the call simply has no effect.
 
 ### Sidenote: Code yet to be written
@@ -118,7 +122,7 @@ It is decided when creating a hook and can be checked in the `mongoose_hooks` mo
 
 ## Registering hook handlers
 
-In order to store a packet when `ejabberd_sm` runs `offline_message_hook` the relevant module must register a handler for this hook.
+In order to store a packet when `ejabberd_sm` runs `offline_message_hook`, the relevant module must register a handler for this hook.
 To attain the runtime configurability the module should register the handlers when it's loaded and unregister them when
 it's unloaded.
 That's usually done in, respectively, `start/2` and `stop/1` functions.
@@ -199,12 +203,13 @@ in_subscription(Acc, #{to := ToJID, from := FromJID, type := Type}, _) ->
 
 As seen in this example, a handler receives an accumulator, parameters and extra parameters (in this case - ignored).
 Then it matches to the result of `process_subscription/4` and can return 3 different values:
+
 * `{ok, Acc}` - it allows further processing and does not change the accumulator.
 * `{stop, mongoose_acc:set(hook, result, false, Acc)}` - it stops further processing and returns accumulator with a new value in it.
 * `{stop, Acc}` - it stops further processing and does not change the accumulator.
 
 This is an important feature to note: in some cases our handler returns a tuple  `{stop, Acc}`.
-This skips calling the latter actions in the handler sequence, while the hook call returns the `Acc`.
+This skips calling later actions in the handler sequence, while the hook call returns the `Acc`.
 Further processing is only performed if the first element of return tuple is `ok`.
 
 Watch out! Different handlers may be registered for the same hook - the priority mechanism orders their execution.
@@ -218,7 +223,7 @@ Always ensure what handlers are registered for a given hook (`grep` is your frie
 The following command should give you a list of all the hooks available in MongooseIM:
 
 ```bash
-awk '/\-export\(\[/,/\]\)\./' src/mongoose_hooks.erl | grep -oh "\w*\/" | sed 's/.$//' | sort
+awk '/\-export\(\[/,/\]\)\./' src/hooks/*.erl | grep -oh "\w*/" | sed 's/.$//' | sort
 ```
 It returns:
 ```bash
@@ -230,7 +235,7 @@ adhoc_sm_commands
 xmpp_stanza_dropped
 ```
 
-It just extracts the hooks exported from the `mongoose_hooks` module.
+It just extracts the hooks exported from `mongoose_hooks` and other hook API modules.
 Refer to `grep`/`ack` to find where they're used.
 
 ## Creating your own hooks
@@ -328,7 +333,7 @@ stopping_handler(Acc, #{number := Number}, _) ->
 
 never_run_handler(Acc, #{number := Number}, _) ->
     ?LOG_INFO(#{what => never_run_handler,
-                text => <<"This hook won't run as it's registered with a priority bigger "
+                text => <<"This handler won't run as it's registered with a priority bigger "
                           "than that of stopping_handler/2 is. "
                           "This text should never get printed.">>}),
     {ok, Acc * Number}.
@@ -339,8 +344,8 @@ The module is intended to be used from the shell for educational purposes:
 ```erlang
 (mongooseim@localhost)1> gen_mod:is_loaded(<<"localhost">>, mod_hook_example).
 false
-(mongooseim@localhost)2> gen_mod:start_module(<<"localhost">>, mod_hook_example, [no_opts]).
-{ok,ok}
+(mongooseim@localhost)2> mongoose_modules:ensure_started(<<"localhost">>, mod_hook_example, #{}).
+{started,ok}
 (mongooseim@localhost)3> gen_mod:is_loaded(<<"localhost">>, mod_hook_example).
 true
 (mongooseim@localhost)4> mongoose_logs:set_module_loglevel(mod_hook_example, info).

doc/developers-guide/hooks_description.md

@@ -4,26 +4,47 @@ This is a brief documentation for a few selected hooks.
 Though hooks & handlers differ in what they are there to do, it is not necessary to describe them all, because the mechanism is general.
 The following is meant to give you the idea of how the hooks work, what they are used for and the various purposes they can serve.
 
-## `user_send_packet`
+## `user_send_*`
 
 ```erlang
-mongoose_hooks:user_send_packet(Acc, FromJID, ToJID, El),
+mongoose_c2s_hooks:user_send_packet(HostType, Acc, Params)
+mongoose_c2s_hooks:user_send_message(HostType, Acc, Params)
+mongoose_c2s_hooks:user_send_presence(HostType, Acc, Params)
+mongoose_c2s_hooks:user_send_iq(HostType, Acc, Params)
+mongoose_c2s_hooks:user_send_xmlel(HostType, Acc, Params)
 ```
+These hooks are run in `mongoose_c2s` after the C2S process receives an XML element from the client.
+
+The hooks won't be called for stanzas arriving from a user served by a federated server (i.e. on a server-to-server connection handled by `ejabberd_s2s`) intended for a user served by the relevant ejabberd instance.
+
+The logic depends on the C2S state, which changes during the connection, authentication and resource binding process:
+
+### Hooks called for `session_established`
 
-This hook is run in `ejabberd_c2s` after the user sends a packet.
 Some rudimentary verification of the stanza is done once it is received from the socket:
 
 - if present, the `from` attribute of the stanza is checked against the identity of the user whose session the process in question serves;
   if the identity does not match the contents of the attribute, an error is returned,
 - the recipient JID (`to` attribute) format is verified.
 
-The hook is not run for stanzas which do not pass these basic validity checks.
-Neither are such stanzas further processed by the server.
-The hook is not run for stanzas in the `jabber:iq:privacy` or `urn:xmpp:blocking` namespaces.
+After successful checks, the following hooks are called. The first one is `user_send_packet`, which is called for all received XML elements. Next, depending on the type of the element, one of the following hooks is called:
+
+* `user_send_message` for messages,
+* `user_send_presence` for presences,
+* `user_send_iq` for IQ (info/query) stanzas,
+* `user_send_xmlel` for other XML elements.
+
+These type-specific hooks should be used instead of `user_send_packet` when possible.
+
+### Hooks called for other states
+
+If the session is not established (e.g. the client hasn't authenticated or its resource is not bound yet), only the `user_send_xmlel` hook is called regardless of the XML element type. No other `user_send_*` hooks are called, and no stanza checks are performed.
 
-This hook won't be called for stanzas arriving from a user served by a federated server (i.e. on a server-to-server connection handled by `ejabberd_s2s`) intended for a user served by the relevant ejabberd instance.
+### Handler examples
 
-It is handled by the following modules:
+These hooks are handled by the following modules:
+
+* [`mod_blocking`](../modules/mod_blocking.md) - handles IQ requests for blocking lists.
 
 * [`mod_caps`](../modules/mod_caps.md) - detects and caches capability information sent with certain messages for later use.
 
@@ -38,52 +59,80 @@ It is handled by the following modules:
 * [`mod_ping`](../modules/mod_ping.md) - upon reception of every message from the client, this module (re)starts a timer;
  if nothing more is received from the client within 60 seconds, it sends an IQ ping, to which the client should reply - which starts another timer.
 
-* `mod_smart_markers` - checks if the stanza contains chat markers info and stores the update.
+* [`mod_presence`](../modules/mod_presence.md) - handles incoming presence stanzas, updating the user presence state and broadcasting presence updates.
+
+* [`mod_privacy`](../modules/mod_privacy.md) - filters sent stanzas according to privacy lists and handles privacy-related IQ requests.
+
+* [`mod_register`](../modules/mod_register.md) - registers a new user when a registration IQ is received. `user_send_xmlel` is used because the stanza is received while the session is not established.
+
+* [`mod_smart_markers`](../modules/mod_smart_markers.md) - checks if the stanza contains chat markers info and stores the update.
+
+* [`mod_stream_management`](../modules/mod_stream_management.md) - counts stanzas sent by the client and handles special XML elements like `<a>` and `<enable>`.
+
+## `filter_packet` and `filter_local_packet`
+
+```erlang
+mongoose_hooks:filter_packet({From, To, Acc, Packet})
+mongoose_hooks:filter_local_packet({From, To, Acc, Packet})
+```
+
+These hooks are run when the packet is being [routed](../Stanza-routing/#3-message-routing) by `ejaberd_router:route/4`, which is the most general function used to route stanzas across the entire cluster. For example, `mongoose_c2s` calls it after calling the `user_send_message` or `user_send_iq` hook, and multiple modules use it for sending replies and errors.
+
+* `filter_packet` is run by `mongoose_router_global` for all routed packets. It is called at the start of the routing procedure.
+* `filter_local_packet` is run by `mongoose_local_delivery` when the packet is being routed to a domain hosted by the local server.
 
-## `user_receive_packet`
+The handlers expect the `{From, To, Acc, Packet}` accumulator as their first argument.
+The stanza can be filtered out (in case the handler returns `drop`), left unchanged or modified.
+
+!!! note "`filter_packet` is a global hook"
+    Note the hook code inside `mongoose_hooks`:
+    ```erlang
+    filter_packet(Acc) ->
+        run_global_hook(filter_packet, Acc, #{}).
+    ```
+    This hook is run not for a host type, but globally across the whole cluster.
+    Keep that in mind when registering the handlers and appropriately use the atom `global` instead of a host type as the second argument.
+
+## `user_receive_*`
 
 ```erlang
-mongoose_hooks:user_receive_packet(StateData#state.host_type, Acc,
-                                   StateData#state.jid, From, To, FixedEl),
+mongoose_c2s_hooks:user_receive_packet(HostType, Acc, Params)
+mongoose_c2s_hooks:user_receive_message(HostType, Acc, Params)
+mongoose_c2s_hooks:user_receive_presence(HostType, Acc, Params)
+mongoose_c2s_hooks:user_receive_iq(HostType, Acc, Params)
+mongoose_c2s_hooks:user_receive_xmlel(HostType, Acc, Params)
 ```
 
-The hook is run just before a packet received by the server is sent to the user.
+These hooks are run in `mongoose_c2s` after the recipient's  C2S process receives an XML element and before sending it to the user.
+
+The hooks won't run for stanzas which are destined to users of a different XMPP domain served by a federated server, connection to which is handled by `ejabberd_s2s`.
 
-Prior to sending, the packet is verified against any relevant privacy lists (the mechanism is described in [XEP-0016: Privacy Lists][privacy-lists]).
-The privacy list mechanism itself is not mandatory and requires `mod_privacy` to be configured; otherwise all stanzas are allowed to pass.
+The first hook is `user_receive_packet`, which is called for all received XML elements. Next, depending on the type of the stanza, one of the following hooks is called:
 
-This hook won't run for stanzas which are destined to users of a different XMPP domain served by a federated server, connection to which is handled by `ejabberd_s2s`.
+* `user_receive_message` for messages,
+* `user_receive_presence` for presences,
+* `user_receive_iq` for IQ (info/query) stanzas,
+* `user_receive_xmlel` for other XML elements.
 
-It is handled by the following modules:
+These type-specific hooks should be used instead of `user_receive_packet` when possible.
+
+### Handler examples
+
+These hooks are handled by the following modules:
 
 * [`mod_caps`](../modules/mod_caps.md) - detects and caches capability information sent with certain messages for later use.
 
 * [`mod_carboncopy`](../modules/mod_carboncopy.md) - if the received packet is a message, it forwards it to all the user's resources which have carbon copying enabled.
 
-## `filter_packet`
+* [`mod_last`](../modules/mod_last.md) - filters queries for user's last activity according to presence subscriptions.
 
-```erlang
-mongoose_hooks:filter_packet({OrigFrom, OrigTo, OrigAcc, OrigPacket})
-```
+* [`mod_mam`](../modules/mod_mam.md) - stores incoming messages in an archive.
 
-This hook is run by `mongoose_router_global` when the packet is being routed by `ejaberd_router:route/3`.
-It is in fact the first call made within the routing procedure.
-If a function hooked in to `filter_packet` returns `drop`, the packet is not processed.
+* [`mod_presence`](../modules/mod_presence.md) - handles incoming presences from other users, updating their status, and responds to presence probes.
 
-The `ejaberd_router:route/3` is the most general function used to route stanzas across the entire cluster, and calls to it are scattered all over MongooseIM code.
-`ejabberd_c2s` calls it after it receives a packet from `ejabberd_receiver` (i.e. the socket) and multiple modules use it for
-sending replies and errors.
+* [`mod_privacy`](../modules/mod_privacy.md) - filters received stanzas according to privacy lists.
 
-The handlers expect no arguments other than the accumulator, which is a packet.
-It may or may not be filtered out (in case the handler chain returns `drop`) or modified.
-
-Note the hook code inside `mongoose_hooks`:
-```erlang
-filter_packet(Acc) ->
-    run_global_hook(filter_packet, Acc, []).
-```
-This hook is run not for a host type, but globally across the whole cluster.
-Keep that in mind when registering the handlers and appropriately use the atom `global` instead of a host type as the second argument.
+* [`mod_stream_management`](../modules/mod_stream_management.md) - fliters out stanzas with conflicting session ID's.
 
 ## `offline_message_hook`