[GH-692] Fix formatting inside code block in issue created notification #697

raghavaggarwal2308 · 2023-09-13T13:48:39Z

Summary

Fixed the improper formatting of special characters inside the code block of issue-created channel notifications.

What to test?

Special characters are properly displayed inside the code block present in the description of the issue-created notification.

Screenshots

Earlier:

Now:

Ticket Link

Fixes #692

…reated notification (#35) * [MI-3405] Fix issue: improper formatting inside code block in issue notification * [MI-3461] Add a comment to clearify the usage of code

mickmister · 2023-09-27T13:56:02Z

server/plugin/webhook.go

+	// Replacing the unicodes with the respective special characters to have proper rendering in code block.
+	result := strings.ReplaceAll(policy.Sanitize(description), "&#39;", "'")
+	result = strings.ReplaceAll(result, "&#34;", "\"")
+	result = strings.ReplaceAll(result, "&amp;", "&")
+	result = strings.ReplaceAll(result, "&gt;", ">")
+	result = strings.ReplaceAll(result, "&lt;", "<")
+	return strings.TrimSpace(result)


LGTM, though I wonder if there is a more catch-all holistic approach

@mickmister I explored other packages to sanitize the HTML but the one we are using right now was the best I could find. But this converts the special characters to their unicode value which are not formatted inside a code block, so we are replacing them again here. Please feel free to share your opinions if you have a better approach in mind.

Hmm so it's the policy.Sanitize that's causing the issue. We really just want to use it to remove the <details> elements from the description, and not escape those characters.

As this comment microcosm-cc/bluemonday#39 (comment) mentions on a related issue for the bluemonday package, we can call html.UnescapeString from the result to make those characters go back to what we want them to be

@mickmister Updated

mickmister · 2023-10-13T15:48:25Z

server/plugin/webhook.go

-	result = strings.ReplaceAll(result, "&gt;", ">")
-	result = strings.ReplaceAll(result, "&lt;", "<")
+	result := policy.Sanitize(description)
+	html.UnescapeString(result)


I think we need to reassign the value right?

Suggested change

html.UnescapeString(result)

result = html.UnescapeString(result)

@mickmister Updated

codecov-commenter · 2023-10-16T14:07:22Z

Codecov Report

Attention: 2 lines in your changes are missing coverage. Please review.

Comparison is base (b8ac3c3) 15.78% compared to head (74db52f) 15.78%.
Report is 3 commits behind head on master.

Additional details and impacted files

@@            Coverage Diff             @@
##           master     #697      +/-   ##
==========================================
- Coverage   15.78%   15.78%   -0.01%     
==========================================
  Files          15       15              
  Lines        5574     5575       +1     
==========================================
  Hits          880      880              
- Misses       4652     4653       +1     
  Partials       42       42

Files	Coverage Δ
server/plugin/webhook.go	`0.80% <0.00%> (-0.01%)`	⬇️

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

mickmister · 2023-10-16T15:14:52Z

Does the bluemonday policy remove things other than the <details> tags? I'm thinking it may remove things like <script> because the tool is meant to sanitize the HTML. We would not want it to remove anything but the <details> tags in our case.

raghavaggarwal2308 · 2023-10-17T08:45:50Z

@mickmister Yes, it's removing the HTML tags in the string. For some tags, it's just removing the tags and for some, it's removing the tags along with the text inside.

mickmister · 2023-10-17T10:44:13Z

@raghavaggarwal2308 Since the main thing we want to sanitize is <details>, we could just only run the sanitize when there exists a <details> tag in the post body. It's not common to have those in the post body, but when it shows up it's usually spammy which is why we remove them

raghavaggarwal2308 · 2023-10-17T11:24:46Z

@raghavaggarwal2308 Since the main thing we want to sanitize is <details>, we could just only run the sanitize when there exists a <details> tag in the post body. It's not common to have those in the post body, but when it shows up it's usually spammy which is why we remove them

@mickmister If we do so and there are other tags along with the details tag, then they will be removed as well. Is that fine?

mickmister · 2023-10-17T11:32:21Z

@raghavaggarwal2308 Yes that's fine 👍

…>" tag is present in it.

raghavaggarwal2308 · 2023-10-17T15:23:14Z

@mickmister Added logic to sanitize description conditionally.

…notification (mattermost#697) * [MI-3405] Fix issue: improper formatting inside code block in issue created notification (#35) * [MI-3405] Fix issue: improper formatting inside code block in issue notification * [MI-3461] Add a comment to clearify the usage of code * Updated code to support less than and greater than symbols * Updated code to use html.UnescapeString * Review fix * [MI-3633] Updated logic to sanitize description only when a "<details>" tag is present in it.

* [GH-692] Fix formatting inside code block in issue created notification (#697) * [MI-3405] Fix issue: improper formatting inside code block in issue created notification (#35) * [MI-3405] Fix issue: improper formatting inside code block in issue notification * [MI-3461] Add a comment to clearify the usage of code * Updated code to support less than and greater than symbols * Updated code to use html.UnescapeString * Review fix * [MI-3633] Updated logic to sanitize description only when a "<details>" tag is present in it. * Bump plugin version to v2.1.7 --------- Co-authored-by: Raghav Aggarwal <[email protected]>

raghavaggarwal2308 added 2 commits September 12, 2023 23:20

[MI-3405] Fix issue: improper formatting inside code block in issue c…

e3c27ce

…reated notification (#35) * [MI-3405] Fix issue: improper formatting inside code block in issue notification * [MI-3461] Add a comment to clearify the usage of code

Updated code to support less than and greater than symbols

df6f8e2

raghavaggarwal2308 requested a review from hanzei as a code owner September 13, 2023 13:48

hanzei approved these changes Sep 26, 2023

View reviewed changes

hanzei requested a review from mickmister September 26, 2023 09:04

hanzei added 2: Dev Review Requires review by a core committer 3: QA Review Requires review by a QA tester labels Sep 26, 2023

hanzei added this to the v2.2.0 milestone Sep 26, 2023

mickmister approved these changes Sep 27, 2023

View reviewed changes

mickmister requested a review from AayushChaudhary0001 September 27, 2023 13:56

mickmister removed the 2: Dev Review Requires review by a core committer label Sep 27, 2023

Updated code to use html.UnescapeString

26cf5ed

mickmister reviewed Oct 13, 2023

View reviewed changes

Review fix

74db52f

mickmister approved these changes Oct 16, 2023

View reviewed changes

[MI-3633] Updated logic to sanitize description only when a "<details…

bbf95c6

…>" tag is present in it.

mickmister approved these changes Oct 17, 2023

View reviewed changes

mickmister merged commit b71f32e into mattermost:master Nov 2, 2023
7 checks passed

avas27JTG mentioned this pull request Nov 8, 2023

Bump version to 2.1.7 #708

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[GH-692] Fix formatting inside code block in issue created notification #697

[GH-692] Fix formatting inside code block in issue created notification #697

raghavaggarwal2308 commented Sep 13, 2023 •

edited

Loading

mickmister Sep 27, 2023 •

edited

Loading

raghavaggarwal2308 Oct 5, 2023

mickmister Oct 13, 2023

raghavaggarwal2308 Oct 13, 2023

mickmister Oct 13, 2023

raghavaggarwal2308 Oct 16, 2023

codecov-commenter commented Oct 16, 2023 •

edited

Loading

mickmister commented Oct 16, 2023

raghavaggarwal2308 commented Oct 17, 2023

mickmister commented Oct 17, 2023

raghavaggarwal2308 commented Oct 17, 2023

mickmister commented Oct 17, 2023

raghavaggarwal2308 commented Oct 17, 2023

	html.UnescapeString(result)
	result = html.UnescapeString(result)

[GH-692] Fix formatting inside code block in issue created notification #697

[GH-692] Fix formatting inside code block in issue created notification #697

Conversation

raghavaggarwal2308 commented Sep 13, 2023 • edited Loading

Summary

What to test?

Screenshots

Ticket Link

mickmister Sep 27, 2023 • edited Loading

Choose a reason for hiding this comment

raghavaggarwal2308 Oct 5, 2023

Choose a reason for hiding this comment

mickmister Oct 13, 2023

Choose a reason for hiding this comment

raghavaggarwal2308 Oct 13, 2023

Choose a reason for hiding this comment

mickmister Oct 13, 2023

Choose a reason for hiding this comment

raghavaggarwal2308 Oct 16, 2023

Choose a reason for hiding this comment

codecov-commenter commented Oct 16, 2023 • edited Loading

Codecov Report

mickmister commented Oct 16, 2023

raghavaggarwal2308 commented Oct 17, 2023

mickmister commented Oct 17, 2023

raghavaggarwal2308 commented Oct 17, 2023

mickmister commented Oct 17, 2023

raghavaggarwal2308 commented Oct 17, 2023

raghavaggarwal2308 commented Sep 13, 2023 •

edited

Loading

mickmister Sep 27, 2023 •

edited

Loading

codecov-commenter commented Oct 16, 2023 •

edited

Loading