feat(web): import the generator for the pred-text wordbreaker's Unicode-property data-table ⚡ #10690
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
User Test ResultsTest specification and instructions User tests are not required Test Artifacts
|
github-actions
bot
added
common/
common/models/
common/models/wordbreakers/
feat
web/
labels
…reaker-property-data-gen
…o feat/web/wordbreaker-property-data-gen
…breaker toolchain
github-actions
bot
added
common/resources/
github-actions
bot
added
common/web/
docs
web/
and removed
web/
common/web/
docs
labels
github-actions
bot
added
common/web/
docs
web/
and removed
web/
common/web/
docs
labels
jahorton
commented
Aug 23, 2024
mcdurdin
reviewed
Aug 23, 2024
github-actions
bot
added
common/web/
docs
web/
and removed
web/
common/web/
docs
labels
|
Changes in this pull request will be available for download in Keyman version 18.0.99-alpha |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Fixes #7224.
Note: approximately 2800 lines are from externally-defined Unicode property data, imported for use in-repo.
This PR imports https://github.com/eddieantonio/unicode-default-word-boundary/tree/master/libexec (MIT licensed!) for direct inclusion and use within our repository. After some reorganization and tweaks, I've got it spitting out a perfect data-table match for our current data.ts referenced by the wordbreaker;
git diffshows no differences between the table in our existingdata.tsand the table built by the newly-included generator as of commit 58c7575.To avoid accidental shifts in behavior at unexpected times, I've opted to require manual updates of the underlying data tables. This update may be run through
/resources/standards-data/unicode-character-database/build.sh configureafter updating the Unicode version specified inresources/build/minimum-versions.inc.sh. Refer to #12103 for related setup. This build-script is new with this PR, allowing a more streamlined update process whenever we feel the need to trigger it.Also note: the wordbreaker was operating with Unicode 13.0.0 data; this will update its data to 15.1.0, which is used throughout our other platforms.
@keymanapp-test-bot skip