forked from microsoft/icu
-
Notifications
You must be signed in to change notification settings - Fork 24
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[wasm] Enable Icu sharding #104
Closed
Closed
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
tqiu8
requested review from
lewing,
marek-safar,
steveisok,
EgorBo,
akoeplinger and
CoffeeFlux
April 22, 2021 03:04
Could you add readme.md file with some details about the reasoning and steps on how to regenerate the data on ICU update? |
lewing
reviewed
May 5, 2021
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Did you consider a dictionary structure more like (just an example not correct)
{
"packs": {
"base": {
"core": [
"icudt_base.dat",
"icudt_normalization.app.dat",
],
"currency": [ "icudt_currency.app.dat" ]
},
"efigs": {
"extends": "base",
"core": [
"icudt_efigs_locales.app.dat",
"icudt_efigs_coll.app.dat"
],
"zones": [ "icudt_efigs_zones.app.dat" ]
},
"cjk": {
"extends": "base",
"core": [
"icudt_cjk_locales.app.dat",
"icudt_cjk_coll.app.dat"
]
},
"nocjk": {
"extends": "base",
},
"full": {
"core": [ "icudt_all.dat" ]
},
},
"locales": {
"efigs": [ "en", "fr", "de", "se" ],
"cjk": [ "cn", "jp", "kr" ],
"nocjk": [ "*" ],
"full": "full",
"invariant": null
}
}
That would reduce the dictionary size a lot it would also potentially simplify the loading js (which could also be produced here). Ultimately the goal is to load the right pack from the browser not at build, we could (and probably should) select the optional features at build time. Also given a robust enough dictionary format we could probably use it to generate the filters instead of building them by hand.
Current dictionary looks like this:
|
directhex
pushed a commit
that referenced
this pull request
May 13, 2022
ilonatommy
added a commit
that referenced
this pull request
Jul 19, 2022
ilonatommy
added a commit
to ilonatommy/icu
that referenced
this pull request
Jul 19, 2022
ilonatommy
added a commit
to ilonatommy/icu
that referenced
this pull request
Jul 19, 2022
Closed
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Initial work for enabling ICU sharding.
Summary
PR Checklist
Detailed Description
This PR will enable sharding to work on WASM. Also added functionality to auto-generate
icu_dictionary.json
which maps relevant files to features and locales. Runtime functionalities included in dotnet/runtime#51665Additional documentation for sharding can be found in here.
Related to dotnet/runtime#49220 and dotnet/runtime#49221