Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

chore(src/compiler/msvc): replace rust-encoding with encoding-rs #2010

Merged
merged 3 commits into from
Jan 10, 2024

Conversation

liushuyu
Copy link
Contributor

@liushuyu liushuyu commented Jan 6, 2024

This pull request replaces rust-encoding with encoding-rs.

The former is unmaintained and uses ambiguous encoding for ISO 8859-1 (note that CP1252 aka "Latin-1" on Windows(R) is a superset of ISO 8859-1). This pull request also corrects this issue by using the WINDOWS-1252 encoding to match the encoding more closely.

@Xuanwo
Copy link
Collaborator

Xuanwo commented Jan 6, 2024

Please rebase with main branch to remove unrelated commits.

@liushuyu
Copy link
Contributor Author

liushuyu commented Jan 6, 2024

Please rebase with main branch to remove unrelated commits.

Done. Rebased on top of the main branch

@liushuyu
Copy link
Contributor Author

liushuyu commented Jan 6, 2024

No, there are some differences I failed to account for when switching the implementations.

@liushuyu
Copy link
Contributor Author

liushuyu commented Jan 6, 2024

Fixed. I also added one more test case to test the decoding logic more thoroughly.

... for `test_responsefile_encoding_utf16le` since encoding_rs does not
have a real UTF-16LE encoder
... for Windows 1252 code page. This tests an undefined symbol in
ISO-8859-1 (but is defined in Windows-1252 and ISO-8859-15)
@codecov-commenter
Copy link

codecov-commenter commented Jan 6, 2024

Codecov Report

Attention: 15 lines in your changes are missing coverage. Please review.

Comparison is base (3d5d41f) 30.49% compared to head (65512a1) 30.95%.

Files Patch % Lines
src/compiler/msvc.rs 51.61% 3 Missing and 12 partials ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main    #2010      +/-   ##
==========================================
+ Coverage   30.49%   30.95%   +0.46%     
==========================================
  Files          51       51              
  Lines       19638    19617      -21     
  Branches     9437     9427      -10     
==========================================
+ Hits         5988     6072      +84     
+ Misses       8015     7874     -141     
- Partials     5635     5671      +36     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@liushuyu liushuyu marked this pull request as ready for review January 6, 2024 02:45
@liushuyu
Copy link
Contributor Author

liushuyu commented Jan 6, 2024

CI tests passed. The coverage misses are due to negative code-path not being reached (should I add tests to test invalid encodings?)

@xen0n
Copy link

xen0n commented Jan 6, 2024

CI tests passed. The coverage misses are due to negative code-path not being reached (should I add tests to test invalid encodings?)

In general, negative test cases are also, if not more, beneficial, as arguably the error-handling codepath is more valuable regarding UX and test coverage... and in this case I think we want to be robust against inputs that definitely can contain invalid encoding, and can be controlled by the user.

@sylvestre
Copy link
Collaborator

do you know if it fixes any issue ?

@liushuyu
Copy link
Contributor Author

liushuyu commented Jan 6, 2024

do you know if it fixes any issue ?

In theory, any valid Windows 1252 but invalid ISO 8859-1 code points (e.g. file names) should work now.

@sylvestre sylvestre merged commit 9863a99 into mozilla:main Jan 10, 2024
49 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants