Matrix html mode not working

[HarHarLinks]

📣 Notification Service(s) Impacted

Matrix.org

🐞 Describe the bug

Trying to send an html formatted message. Instead of applying the markup, the html code are escaped and displayed verbatim in a matrix client.

💡 Screenshots and Logs

$ docker run --rm --network matrix -v /apprise.yml:/apprise.yml caronc/apprise:latest apprise -vvv --config /apprise.yml --title 'creating an issue' --body "{{app_title}}\n<pre><code>about this on github</code></pre>"
2021-10-12 15:19:27,347 - DEBUG - Loaded URL: matrixs://matrixbot:[email protected]/%21someroomid?image=no&mode=off&format=html&overflow=upstream&rto=4.0&cto=4.0&verify=yes
2021-10-12 15:19:28,906 - DEBUG - Matrix Payload: {'msgtype': 'm.text', 'body': 'creating an issue\r\n{{app_title}}\\n&lt;pre&gt;&lt;code&gt;about&nbsp;this&nbsp;on&nbsp;github&lt;/code&gt;&lt;/pre&gt;', 'format': 'org.matrix.custom.html', 'formatted_body': 'creating an issue\\n&lt;pre&gt;&lt;code&gt;about&nbsp;this&nbsp;on&nbsp;github&lt;/code&gt;&lt;/pre&gt;'}

💻 Your System Details:

• OS: debian 11 / docker
• Python Version: whatever is in the docker container

🔮 Additional context
However it is supposed to work is not documented https://github.com/caronc/apprise/wiki/Notify_matrix

[caronc]

Do you have control over the apprise CLI command under the hood? By default it passes content as text. You're doing an edge case where you want to directly pass HTML as input to the tool.

You'll want to add --input-format=html (or -i html) and see if that helps at all?

[HarHarLinks]

Hm yes that does something. Is this how it is supposed to work and I just missed it in the doc somewhere? Why are both the GET param and the cli param necessary?

[caronc]

Basically the one used by the URL you specify tells you which output format you want to use on the service side. Most only have one format (so specifying it isn't really necessary) such as text messaging, etc. But some cater to multiple (like email for example, Slack).

The CLI tool needs to know what you're feeding it. By default it assumes text. So when it comes across a plugin that only has an HTML output for example, it needs to escape the <> characters (if present) otherwise what you see, won't be what you get.

To support edge cases, if you tell Apprise you're feeding it html and it detects that the output should be html, then no manipulation of you're payload is required.

Developers who use Apprise can optionally fix the default input type to text (like the CLI) or they can leave the default to not set to anything at all. In this case there is never any string manipulation at all. It presumes your aware of the upstream service you're using and want to just pass everything 'as-is'.

There is no question this comes down to a document issue. If things are working for you, I can still leave this ticket open until I add this info somewhere? Thoughts?

[HarHarLinks]

The wiki page about matrix says

Message Format: Text

Further down, format=markdown is mentioned with the slack mode. And as we have seen, format=html is also possible (necessary) at least in the native case I demonstrated, I haven't looked at slack/webhook. This certainly needs to be clarified.

What confuses me about the GET param is that matrix as a protocol has no "markdown" output type as you explained, there is only plain text which is

"msgtype": "m.text",
"body": "plain text here"

and there is markup text that builds on top of it, e.g.:

    "msgtype": "m.text",
    "body": "username",
    "format": "org.matrix.custom.html",
    "formatted_body": "<a href=\"https://matrix.to/#/@username:server.tld\">username</a>"

It might be that this markdown output type is only for the webhooks and might be warranted there and just needs updated docs. That said what would be nice is for apprise to ingest markdown and post that as matrix html (or just the missing documentation about it if that's the case).

Even with the correct params to produce a formatted message we figured out above, apprise does not handle it correctly:

docker run --rm -i --network matrix -v /apprise.yml:/apprise.yml caronc/apprise:latest apprise -vvv --config /apprise.yml --body "<a href=\"https://matrix.to/#/@username:server.tld\">username</a>" -i html

produces

    "msgtype": "m.text",
    "body": "<a href=\"https://matrix.to/#/@username:server.tld\">username</a>",
    "format": "org.matrix.custom.html",
    "formatted_body": "<a href=\"https://matrix.to/#/@username:server.tld\">username</a>"

, i.e. the html is just pasted into body, instead of the above "correct" plain html-stripped body.

[caronc]

Good point! This might be more tricky to solve. I may have to pull in another library into apprise to reverse the HTML into TEXT (or re-write the wheel for the light weight factor). the HTML update was brand spanking new as of PR #431. But i was so focused on markdown which leave the text untouched and updates the formatted part with the HTML.

[HarHarLinks]

I don't know if I mentioned, but can't find it so here goes:

sending a code block like

Lorem
ipsum

into matrix using element yields me:

"formatted_body": "<pre><code>Lorem\nipsum\n</code></pre>\n"

i.e. \n characters are inserted verbatim in these 2 places since the minified json can't have linebreaks.

running apprise with -i html and ?format=html and that body generated by element gets me

<pre><code>Lorem\\nipsum\\n</code></pre>\\n

There should be a way to do this properly (on a single line), e.g. I want to embed a command output to a cronjob like
--body "file contents:<br>\n<pre><code>$(cat /some/file)\n</code></pre>\n"

[caronc]

You're pretty close, i mean you can tell Apprise to interpret escapes with --interpret-escapes (-e) which would work smoothly with the last command you just provided.

You could also just prepare a template file:

# our template file:
cat << _EOF > my_apprise_template.html
<pre>
   <code>{{body}}</code>
</pre>
_EOF

# Then:
sed -e "s|{{body}}|Lorem\nipsum\n|g" my_apprise_template.hml | apprise -e

Alternatively, you're on the right track with your on-liner you've already prepared (just missing the -e switch).

Is this what you're asking?

[HarHarLinks]

Yes, the -e switch seems to be what i am looking for, that's great!
But it does not like me not supplying no title:

apprise -v -i html -e -b "<pre><code>Lorem\nipsum\n</code></pre>\n" 'matrixs://apprise:[email protected]/room/?msgtype=notice&format=html'
2021-10-25 20:44:49,758 - ERROR - Failed to escape message title

[caronc]

Doh, well that's embarassing (and a bug) 😳.

[caronc]

I believe I've solved the issue with the last merge request. It's part of the master branch now and will definitely be part of the next version of Apprise. The escaping should work with or without a title now (the way it should have been).

If you need to install the master branch to try it out, this should work:

# Install the branch
pip install git+https://github.com/caronc/apprise.git

[caronc]

Let me know how this goes; I'm pretty sure I nailed it, but I'll just wait your confirmation before I close the ticket off for good.

[HarHarLinks]

I think I did test and forgot to reply back, but anyway tested again and it seems good!