Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Improve/Rewrite tests checking for CSP violations #576

Open
fred-wang opened this issue Jan 16, 2025 · 0 comments
Open

Improve/Rewrite tests checking for CSP violations #576

fred-wang opened this issue Jan 16, 2025 · 0 comments

Comments

@fred-wang
Copy link
Collaborator

fred-wang commented Jan 16, 2025 •
edited
Loading

See trusted-types/trusted-types*reporting as well as content-security-policy/securitypolicyviolation/source-file.html at least.

These tests seems to cause (or have caused) flakiness and timeouts, and maybe are not up-to-date with the spec.

Two recent PRs:
web-platform-tests/wpt#50098
web-platform-tests/wpt#50058

Two bad patterns that cause the tests to not work well:

  • assert_unreached() is called inside a try scope and the exception thrown by the testharness to indicate a failure is discarded.
  • The listener to "securitypolicyviolation" is added after actually running the statement that triggers the violation, so it could be possible that the violation is not caught.

I also noticed they are testing SVGScriptElement text sink (which is not defined in any spec currently) but not all the possible injection sinks in the spec (Window or Worker, see #494 (comment)). and the test for the sample is not super script:

// TODO(vogelheim): The current set of tests allows for more variance in the
// sample reports than the current spec draft does. Once the spec has
// been finalized, we should clamp this down to check byte-for-byte
// against the values mandated by the spec.

Some related issues:

https://bugzilla.mozilla.org/show_bug.cgi?id=1937764 for source-file.html
#575
#494

cc @lukewarlow
@fred-wang fred-wang mentioned this issue Jan 16, 2025
Closed
fred-wang added a commit to web-platform-tests/wpt that referenced this issue Jan 16, 2025
@fred-wang
Rewrite Trusted types tests for CSP violations
fea1317 
Currently the listener to "securitypolicyviolation" is added before
actually running the statement that triggers violations, so it could
be possible that some violations are not caught. This bad pattern is
duplicated in several `trusted-types*reporting*` tests.

This patch adds a new helper file to properly wrap the
listener registration and statement execution in a promise, and
reuses it in existing tests.

w3c/trusted-types#576
@fred-wang fred-wang mentioned this issue Jan 16, 2025
Merged
fred-wang added a commit to web-platform-tests/wpt that referenced this issue Jan 22, 2025
@fred-wang
Rewrite Trusted types tests for CSP violations (#50124)
bb5f835 
* Rewrite Trusted types tests for CSP violations

Currently the listener to "securitypolicyviolation" is added before
actually running the statement that triggers violations, so it could
be possible that some violations are not caught. This bad pattern is
duplicated in several `trusted-types*reporting*` tests.

This patch adds a new helper file to properly wrap the
listener registration and statement execution in a promise, and
reuses it in existing tests.

w3c/trusted-types#576
fred-wang added a commit to web-platform-tests/wpt that referenced this issue Jan 23, 2025
@fred-wang
Trusted Types: Improve testing of samples in violation reports.
6fd8a73 
w3c/trusted-types#576
w3c/trusted-types#494
@fred-wang fred-wang mentioned this issue Jan 23, 2025
Merged
fred-wang added a commit to web-platform-tests/wpt that referenced this issue Jan 23, 2025
@fred-wang
Trusted Types: Improve testing of samples in violation reports. (#50238)
ef244b7 
* Trusted Types: Improve testing of samples in violation reports.

w3c/trusted-types#576
w3c/trusted-types#494
moz-v2v-gh pushed a commit to mozilla/gecko-dev that referenced this issue Jan 27, 2025
@fred-wang @moz-wptsync-bot
Bug 1942119 [wpt PR 50124] - Rewrite Trusted types tests for CSP viol…
9c6d48e 
…ations, a=testonly

Automatic update from web-platform-tests
Rewrite Trusted types tests for CSP violations (#50124)

* Rewrite Trusted types tests for CSP violations

Currently the listener to "securitypolicyviolation" is added before
actually running the statement that triggers violations, so it could
be possible that some violations are not caught. This bad pattern is
duplicated in several `trusted-types*reporting*` tests.

This patch adds a new helper file to properly wrap the
listener registration and statement execution in a promise, and
reuses it in existing tests.

w3c/trusted-types#576

--

wpt-commits: bb5f8351e2b19b1f4cfd16ca891ca638461c1b4b
wpt-pr: 50124
i3roly pushed a commit to i3roly/firefox-dynasty that referenced this issue Jan 28, 2025
@fred-wang @moz-wptsync-bot
Bug 1942119 [wpt PR 50124] - Rewrite Trusted types tests for CSP viol…
a0d8a57 
…ations, a=testonly

Automatic update from web-platform-tests
Rewrite Trusted types tests for CSP violations (#50124)

* Rewrite Trusted types tests for CSP violations

Currently the listener to "securitypolicyviolation" is added before
actually running the statement that triggers violations, so it could
be possible that some violations are not caught. This bad pattern is
duplicated in several `trusted-types*reporting*` tests.

This patch adds a new helper file to properly wrap the
listener registration and statement execution in a promise, and
reuses it in existing tests.

w3c/trusted-types#576

--

wpt-commits: bb5f8351e2b19b1f4cfd16ca891ca638461c1b4b
wpt-pr: 50124
moz-v2v-gh pushed a commit to mozilla/gecko-dev that referenced this issue Jan 28, 2025
@fred-wang @moz-wptsync-bot
Bug 1943316 [wpt PR 50238] - Trusted Types: Improve testing of sample…
19bb0a1 
…s in violation reports., a=testonly

Automatic update from web-platform-tests
Trusted Types: Improve testing of samples in violation reports. (#50238)

* Trusted Types: Improve testing of samples in violation reports.

w3c/trusted-types#576
w3c/trusted-types#494

--

wpt-commits: ef244b70ed0105ed535584bc05d7bb1ec193b1d1
wpt-pr: 50238
i3roly pushed a commit to i3roly/firefox-dynasty that referenced this issue Jan 28, 2025
@fred-wang @moz-wptsync-bot
Bug 1943316 [wpt PR 50238] - Trusted Types: Improve testing of sample…
5c820c8 
…s in violation reports., a=testonly

Automatic update from web-platform-tests
Trusted Types: Improve testing of samples in violation reports. (#50238)

* Trusted Types: Improve testing of samples in violation reports.

w3c/trusted-types#576
w3c/trusted-types#494

--

wpt-commits: ef244b70ed0105ed535584bc05d7bb1ec193b1d1
wpt-pr: 50238
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
1 participant
@fred-wang