From 368958e3220e38b3a149d46c5f241f387bb886e4 Mon Sep 17 00:00:00 2001 From: Martin Geisler Date: Sun, 8 Jan 2023 13:45:19 +0100 Subject: [PATCH] Add support for translations MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit This implements a translation pipeline using the industry-standard Gettext[1] system. I picked Gettext for the reasons described in [2] and [3]: * It’s widely used in open source software. This means that there are graphical editors which will help you in editing the `.po` files. An example is Poedit[4], which is available for all major platforms. There are also many online systems for doing translations. An example is Pontoon[5], which is used for the Rust website itself. We can consider setting up such an instance ourselves. * It is a light-weight yet structured format. This means that nothing changes with regards to how you update the original English text. We can still accept fixes and PRs like normal. The structure means that translators can see exactly which part of the course they need to update after a change. This is completely lost if you simply copy over the original text and translate it in-place in the Markdown files. The code here only adds support for translations. They are not yet tested, published or used for anything. Next steps will be: * Add support for switching languages via a bit of JavaScript on each page. * Update the speaker notes feature to support translations (right now “Speaker Notes” is hard-coded into the generated HTML). I think we should turn it into a mdbook preprocessor instead. * Add testing: We should test that the `.po` files are well-formed. We should also run `mdbook test` on each language since the translations can alter the embedded code. Fixes #115. [1]: https://www.gnu.org/software/gettext/manual/html_node/index.html [2]: https://github.com/rust-lang/mdBook/pull/1864 [3]: https://github.com/rust-lang/mdBook/issues/5#issuecomment-1144887806 [4]: https://poedit.net/ [5]: https://pontoon.rust-lang.org/ --- .gitignore | 3 +- TRANSLATIONS.md | 116 ++ book.toml | 7 + i18n-helpers/Cargo.lock | 1941 +++++++++++++++++++++++ i18n-helpers/Cargo.toml | 14 + i18n-helpers/src/bin/mdbook-gettext.rs | 230 +++ i18n-helpers/src/bin/mdbook-xgettext.rs | 128 ++ i18n-helpers/src/lib.rs | 106 ++ src/SUMMARY.md | 63 +- 9 files changed, 2575 insertions(+), 33 deletions(-) create mode 100644 TRANSLATIONS.md create mode 100644 i18n-helpers/Cargo.lock create mode 100644 i18n-helpers/Cargo.toml create mode 100644 i18n-helpers/src/bin/mdbook-gettext.rs create mode 100644 i18n-helpers/src/bin/mdbook-xgettext.rs create mode 100644 i18n-helpers/src/lib.rs diff --git a/.gitignore b/.gitignore index 564055b328f2..62bf91f0305a 100644 --- a/.gitignore +++ b/.gitignore @@ -1,2 +1,3 @@ /book/ -/target/ +target/ +po/messages.pot diff --git a/TRANSLATIONS.md b/TRANSLATIONS.md new file mode 100644 index 000000000000..207976d22ce6 --- /dev/null +++ b/TRANSLATIONS.md @@ -0,0 +1,116 @@ +# Translations of Comprehensive Rust 🦀 + +We would love to have your help with translating the course into other +languages! We use the [Gettext] system for translations. This means that you +don't modify the Markdown files directly: instead you modify `.po` files in a +`po/` directory. The `.po` files are small text-based translation databases. + +There is a `.po` file for each language. They are named after the [ISO 639] +language codes: Danish would go into `po/da.po`, Korean would go into +`po/ko.po`, etc. The `.po` files contain all the English text plus the +translations. They are initialized from a `messages.pot` file (a PO template) +which contains only the English text. + +We will show how to update and manipulate the `.po` and `.pot` files using the +GNU Gettext utilities below. + +[Gettext]: https://www.gnu.org/software/gettext/manual/html_node/index.html +[ISO 639]: https://en.wikipedia.org/wiki/List_of_ISO_639-1_codes + +## I18n Helpers + +We use two helpers for the translations: + +* `mdbook-xgettext`: This program extracts the English text. It is an mdbook + renderer. +* `mdbook-gettext`: This program translates the book into a target language. It + is an mdbook preprocessor. + +Install both helpers with the following command from the root of the course: + +```shell +$ cargo install --path i18n-helpers +``` + +## Creating and Updating Translations + +First, you need to know how to update the `.pot` and `.po` files. + +As a general rule, you should never touch the auto-generated `po/messages.pot` +file. You should also not edit the `msgid` entries in a `po/xx.po` file. If you +find mistakes, you need to update the original English text instead. The fixes +to the English text will flow into the `.po` files the next time the translators +update them. + +### Generating the PO Template + +To extract the original English text and generate a `messages.pot` file, you run +`mdbook` with a special renderer: + +```shell +$ MDBOOK_OUTPUT='{"xgettext": {"pot-file": "messages.pot"}}' \ + mdbook build -d po +``` + +You will find the generated POT file as `po/messages.pot`. + +### Initialize a New Translation + +To start a new translation, first generate the `po/messages.pot` file. Then use +`msginit` to create a `xx.po` file for the fictional `xx` language: + +```shell +$ msginit -i po/messages.pot -l xx -o po/xx.po +``` + +You can also simply copy `po/messages.pot` to `po/xx.po`. Then update the file +header (the first entry with `msgid ""`) to the correct language. + +### Updating an Existing Translation + +As the English text changes, translations gradually become outdated. To update +the `po/xx.po` file with new messages, first extract the English text into a +`po/messages.pot` template file. Then run + +```shell +$ msgmerge --update po/xx.po po/messages.pot +``` + +Unchanged messages will stay intact, deleted messages are marked as old, and +updated messages are marked "fuzzy". A fuzzy entry will reuse the previous +translation: you should then go over it and update it as necessary before you +remove the fuzzy marker. + +## Using Translations + +This will show you how to use the translations to generate localized HTML +output. + +## Building a Translation + +To use the `po/xx.po` file for your output, run the following command: + +```shell +$ MDBOOK_BOOK__LANGUAGE=xx \ + MDBOOK_PREPROCESSOR__GETTEXT__PO_FILE=po/xx.po \ + MDBOOK_PREPROCESSOR__GETTEXT__RENDERERS='["html"]' \ + MDBOOK_PREPROCESSOR__GETTEXT__BEFORE='["svgbob"]' \ + mdbook build -d book/xx +``` + +This will update the book's language to `xx`, it will make the `mdbook-gettext` +preprocessor become active and tell it to use the `po/xx.po` file, and finally +it will redirect the output to `book/xx`. + +## Serving a Translation + +Like normal, you can use `mdbook serve` to view your translation as you work on +it. You use the same command as with `mdbook build` above, but additionally +we'll tell `mdbook` to watch the `po/` directory for changes: + +```shell +$ MDBOOK_BOOK__LANGUAGE=xx \ + MDBOOK_PREPROCESSOR__GETTEXT__PO_FILE=po/xx.po + MDBOOK_BUILD__EXTRA_WATCH_DIRS='["po"]' + mdbook serve -d book/xx +``` diff --git a/book.toml b/book.toml index 32e858438f91..252e5085ea1e 100644 --- a/book.toml +++ b/book.toml @@ -8,7 +8,14 @@ title = "Comprehensive Rust 🦀" [rust] edition = "2021" +[preprocessor.links] +renderers = ["html"] + +[preprocessor.index] +renderers = ["html"] + [preprocessor.svgbob] +renderers = ["html"] class = "bob" [output.html] diff --git a/i18n-helpers/Cargo.lock b/i18n-helpers/Cargo.lock new file mode 100644 index 000000000000..90697099311d --- /dev/null +++ b/i18n-helpers/Cargo.lock @@ -0,0 +1,1941 @@ +# This file is automatically @generated by Cargo. +# It is not intended for manual editing. +version = 3 + +[[package]] +name = "aho-corasick" +version = "0.7.20" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "cc936419f96fa211c1b9166887b38e5e40b19958e5b895be7c1f93adec7071ac" +dependencies = [ + "memchr", +] + +[[package]] +name = "ammonia" +version = "3.3.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "64e6d1c7838db705c9b756557ee27c384ce695a1c51a6fe528784cb1c6840170" +dependencies = [ + "html5ever", + "maplit", + "once_cell", + "tendril", + "url", +] + +[[package]] +name = "android_system_properties" +version = "0.1.5" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "819e7219dbd41043ac279b19830f2efc897156490d7fd6ea916720117ee66311" +dependencies = [ + "libc", +] + +[[package]] +name = "anyhow" +version = "1.0.68" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "2cb2f989d18dd141ab8ae82f64d1a8cdd37e0840f73a406896cf5e99502fab61" + +[[package]] +name = "autocfg" +version = "1.1.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "d468802bab17cbc0cc575e9b053f41e72aa36bfa6b7f55e3529ffa43161b97fa" + +[[package]] +name = "base64" +version = "0.13.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "9e1b586273c5702936fe7b7d6896644d8be71e6314cfe09d3167c95f712589e8" + +[[package]] +name = "bitflags" +version = "1.3.2" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "bef38d45163c2f1dde094a7dfd33ccf595c92905c8f8f4fdc18d06fb1037718a" + +[[package]] +name = "block-buffer" +version = "0.10.3" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "69cce20737498f97b993470a6e536b8523f0af7892a4f928cceb1ac5e52ebe7e" +dependencies = [ + "generic-array", +] + +[[package]] +name = "bstr" +version = "0.2.17" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "ba3569f383e8f1598449f1a423e72e99569137b47740b1da11ef19af3d5c3223" +dependencies = [ + "lazy_static", + "memchr", + "regex-automata", +] + +[[package]] +name = "bumpalo" +version = "3.11.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "572f695136211188308f16ad2ca5c851a712c464060ae6974944458eb83880ba" + +[[package]] +name = "byteorder" +version = "1.4.3" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "14c189c53d098945499cdfa7ecc63567cf3886b3332b312a5b4585d8d3a6a610" + +[[package]] +name = "bytes" +version = "1.3.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "dfb24e866b15a1af2a1b663f10c6b6b8f397a84aadb828f12e5b289ec23a3a3c" + +[[package]] +name = "cc" +version = "1.0.78" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "a20104e2335ce8a659d6dd92a51a767a0c062599c73b343fd152cb401e828c3d" + +[[package]] +name = "cfg-if" +version = "1.0.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "baf1de4339761588bc0619e3cbc0120ee582ebb74b53b4efbf79117bd2da40fd" + +[[package]] +name = "chrono" +version = "0.4.23" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "16b0a3d9ed01224b22057780a37bb8c5dbfe1be8ba48678e7bf57ec4b385411f" +dependencies = [ + "iana-time-zone", + "js-sys", + "num-integer", + "num-traits", + "time", + "wasm-bindgen", + "winapi", +] + +[[package]] +name = "clap" +version = "4.0.32" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "a7db700bc935f9e43e88d00b0850dae18a63773cfbec6d8e070fccf7fef89a39" +dependencies = [ + "bitflags", + "clap_lex", + "is-terminal", + "once_cell", + "strsim", + "termcolor", + "terminal_size", +] + +[[package]] +name = "clap_complete" +version = "4.0.7" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "10861370d2ba66b0f5989f83ebf35db6421713fd92351790e7fdd6c36774c56b" +dependencies = [ + "clap", +] + +[[package]] +name = "clap_lex" +version = "0.3.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "0d4198f73e42b4936b35b5bb248d81d2b595ecb170da0bac7655c54eedfa8da8" +dependencies = [ + "os_str_bytes", +] + +[[package]] +name = "codespan-reporting" +version = "0.11.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "3538270d33cc669650c4b093848450d380def10c331d38c768e34cac80576e6e" +dependencies = [ + "termcolor", + "unicode-width", +] + +[[package]] +name = "core-foundation-sys" +version = "0.8.3" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "5827cebf4670468b8772dd191856768aedcb1b0278a04f989f7766351917b9dc" + +[[package]] +name = "cpufeatures" +version = "0.2.5" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "28d997bd5e24a5928dd43e46dc529867e207907fe0b239c3477d924f7f2ca320" +dependencies = [ + "libc", +] + +[[package]] +name = "crossbeam-channel" +version = "0.5.6" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "c2dd04ddaf88237dc3b8d8f9a3c1004b506b54b3313403944054d23c0870c521" +dependencies = [ + "cfg-if", + "crossbeam-utils", +] + +[[package]] +name = "crossbeam-utils" +version = "0.8.14" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "4fb766fa798726286dbbb842f174001dab8abc7b627a1dd86e0b7222a95d929f" +dependencies = [ + "cfg-if", +] + +[[package]] +name = "crypto-common" +version = "0.1.6" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "1bfb12502f3fc46cca1bb51ac28df9d618d813cdc3d2f25b9fe775a34af26bb3" +dependencies = [ + "generic-array", + "typenum", +] + +[[package]] +name = "cxx" +version = "1.0.86" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "51d1075c37807dcf850c379432f0df05ba52cc30f279c5cfc43cc221ce7f8579" +dependencies = [ + "cc", + "cxxbridge-flags", + "cxxbridge-macro", + "link-cplusplus", +] + +[[package]] +name = "cxx-build" +version = "1.0.86" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "5044281f61b27bc598f2f6647d480aed48d2bf52d6eb0b627d84c0361b17aa70" +dependencies = [ + "cc", + "codespan-reporting", + "once_cell", + "proc-macro2", + "quote", + "scratch", + "syn", +] + +[[package]] +name = "cxxbridge-flags" +version = "1.0.86" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "61b50bc93ba22c27b0d31128d2d130a0a6b3d267ae27ef7e4fae2167dfe8781c" + +[[package]] +name = "cxxbridge-macro" +version = "1.0.86" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "39e61fda7e62115119469c7b3591fd913ecca96fb766cfd3f2e2502ab7bc87a5" +dependencies = [ + "proc-macro2", + "quote", + "syn", +] + +[[package]] +name = "digest" +version = "0.10.6" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "8168378f4e5023e7218c89c891c0fd8ecdb5e5e4f18cb78f38cf245dd021e76f" +dependencies = [ + "block-buffer", + "crypto-common", +] + +[[package]] +name = "elasticlunr-rs" +version = "3.0.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "b94d9c8df0fe6879ca12e7633fdfe467c503722cc981fc463703472d2b876448" +dependencies = [ + "regex", + "serde", + "serde_derive", + "serde_json", +] + +[[package]] +name = "env_logger" +version = "0.10.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "85cdab6a89accf66733ad5a1693a4dcced6aeff64602b634530dd73c1f3ee9f0" +dependencies = [ + "humantime", + "is-terminal", + "log", + "regex", + "termcolor", +] + +[[package]] +name = "errno" +version = "0.2.8" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "f639046355ee4f37944e44f60642c6f3a7efa3cf6b78c78a0d989a8ce6c396a1" +dependencies = [ + "errno-dragonfly", + "libc", + "winapi", +] + +[[package]] +name = "errno-dragonfly" +version = "0.1.2" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "aa68f1b12764fab894d2755d2518754e71b4fd80ecfb822714a1206c2aab39bf" +dependencies = [ + "cc", + "libc", +] + +[[package]] +name = "fastrand" +version = "1.8.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "a7a407cfaa3385c4ae6b23e84623d48c2798d06e3e6a1878f7f59f17b3f86499" +dependencies = [ + "instant", +] + +[[package]] +name = "filetime" +version = "0.2.19" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "4e884668cd0c7480504233e951174ddc3b382f7c2666e3b7310b5c4e7b0c37f9" +dependencies = [ + "cfg-if", + "libc", + "redox_syscall", + "windows-sys", +] + +[[package]] +name = "fnv" +version = "1.0.7" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "3f9eec918d3f24069decb9af1554cad7c880e2da24a9afd88aca000531ab82c1" + +[[package]] +name = "form_urlencoded" +version = "1.1.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "a9c384f161156f5260c24a097c56119f9be8c798586aecc13afbcbe7b7e26bf8" +dependencies = [ + "percent-encoding", +] + +[[package]] +name = "fsevent-sys" +version = "4.1.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "76ee7a02da4d231650c7cea31349b889be2f45ddb3ef3032d2ec8185f6313fd2" +dependencies = [ + "libc", +] + +[[package]] +name = "futf" +version = "0.1.5" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "df420e2e84819663797d1ec6544b13c5be84629e7bb00dc960d6917db2987843" +dependencies = [ + "mac", + "new_debug_unreachable", +] + +[[package]] +name = "futures-channel" +version = "0.3.25" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "52ba265a92256105f45b719605a571ffe2d1f0fea3807304b522c1d778f79eed" +dependencies = [ + "futures-core", + "futures-sink", +] + +[[package]] +name = "futures-core" +version = "0.3.25" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "04909a7a7e4633ae6c4a9ab280aeb86da1236243a77b694a49eacd659a4bd3ac" + +[[package]] +name = "futures-macro" +version = "0.3.25" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "bdfb8ce053d86b91919aad980c220b1fb8401a9394410e1c289ed7e66b61835d" +dependencies = [ + "proc-macro2", + "quote", + "syn", +] + +[[package]] +name = "futures-sink" +version = "0.3.25" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "39c15cf1a4aa79df40f1bb462fb39676d0ad9e366c2a33b590d7c66f4f81fcf9" + +[[package]] +name = "futures-task" +version = "0.3.25" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "2ffb393ac5d9a6eaa9d3fdf37ae2776656b706e200c8e16b1bdb227f5198e6ea" + +[[package]] +name = "futures-util" +version = "0.3.25" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "197676987abd2f9cadff84926f410af1c183608d36641465df73ae8211dc65d6" +dependencies = [ + "futures-core", + "futures-macro", + "futures-sink", + "futures-task", + "pin-project-lite", + "pin-utils", + "slab", +] + +[[package]] +name = "generic-array" +version = "0.14.6" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "bff49e947297f3312447abdca79f45f4738097cc82b06e72054d2223f601f1b9" +dependencies = [ + "typenum", + "version_check", +] + +[[package]] +name = "getrandom" +version = "0.2.8" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "c05aeb6a22b8f62540c194aac980f2115af067bfe15a0734d7277a768d396b31" +dependencies = [ + "cfg-if", + "libc", + "wasi 0.11.0+wasi-snapshot-preview1", +] + +[[package]] +name = "gitignore" +version = "1.0.7" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "78aa90e4620c1498ac434c06ba6e521b525794bbdacf085d490cc794b4a2f9a4" +dependencies = [ + "glob", +] + +[[package]] +name = "glob" +version = "0.3.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "d2fabcfbdc87f4758337ca535fb41a6d701b65693ce38287d856d1674551ec9b" + +[[package]] +name = "h2" +version = "0.3.15" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "5f9f29bc9dda355256b2916cf526ab02ce0aeaaaf2bad60d65ef3f12f11dd0f4" +dependencies = [ + "bytes", + "fnv", + "futures-core", + "futures-sink", + "futures-util", + "http", + "indexmap", + "slab", + "tokio", + "tokio-util", + "tracing", +] + +[[package]] +name = "handlebars" +version = "4.3.6" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "035ef95d03713f2c347a72547b7cd38cbc9af7cd51e6099fb62d586d4a6dee3a" +dependencies = [ + "log", + "pest", + "pest_derive", + "serde", + "serde_json", + "thiserror", +] + +[[package]] +name = "hashbrown" +version = "0.12.3" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "8a9ee70c43aaf417c914396645a0fa852624801b24ebb7ae78fe8272889ac888" + +[[package]] +name = "headers" +version = "0.3.8" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "f3e372db8e5c0d213e0cd0b9be18be2aca3d44cf2fe30a9d46a65581cd454584" +dependencies = [ + "base64", + "bitflags", + "bytes", + "headers-core", + "http", + "httpdate", + "mime", + "sha1", +] + +[[package]] +name = "headers-core" +version = "0.2.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "e7f66481bfee273957b1f20485a4ff3362987f85b2c236580d81b4eb7a326429" +dependencies = [ + "http", +] + +[[package]] +name = "hermit-abi" +version = "0.2.6" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "ee512640fe35acbfb4bb779db6f0d80704c2cacfa2e39b601ef3e3f47d1ae4c7" +dependencies = [ + "libc", +] + +[[package]] +name = "html5ever" +version = "0.26.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "bea68cab48b8459f17cf1c944c67ddc572d272d9f2b274140f223ecb1da4a3b7" +dependencies = [ + "log", + "mac", + "markup5ever", + "proc-macro2", + "quote", + "syn", +] + +[[package]] +name = "http" +version = "0.2.8" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "75f43d41e26995c17e71ee126451dd3941010b0514a81a9d11f3b341debc2399" +dependencies = [ + "bytes", + "fnv", + "itoa", +] + +[[package]] +name = "http-body" +version = "0.4.5" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "d5f38f16d184e36f2408a55281cd658ecbd3ca05cce6d6510a176eca393e26d1" +dependencies = [ + "bytes", + "http", + "pin-project-lite", +] + +[[package]] +name = "httparse" +version = "1.8.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "d897f394bad6a705d5f4104762e116a75639e470d80901eed05a860a95cb1904" + +[[package]] +name = "httpdate" +version = "1.0.2" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "c4a1e36c821dbe04574f602848a19f742f4fb3c98d40449f11bcad18d6b17421" + +[[package]] +name = "humantime" +version = "2.1.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "9a3a5bfb195931eeb336b2a7b4d761daec841b97f947d34394601737a7bba5e4" + +[[package]] +name = "hyper" +version = "0.14.23" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "034711faac9d2166cb1baf1a2fb0b60b1f277f8492fd72176c17f3515e1abd3c" +dependencies = [ + "bytes", + "futures-channel", + "futures-core", + "futures-util", + "h2", + "http", + "http-body", + "httparse", + "httpdate", + "itoa", + "pin-project-lite", + "socket2", + "tokio", + "tower-service", + "tracing", + "want", +] + +[[package]] +name = "i18n-helpers" +version = "0.1.0" +dependencies = [ + "anyhow", + "mdbook", + "once_cell", + "polib", + "regex", + "semver", + "serde_json", +] + +[[package]] +name = "iana-time-zone" +version = "0.1.53" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "64c122667b287044802d6ce17ee2ddf13207ed924c712de9a66a5814d5b64765" +dependencies = [ + "android_system_properties", + "core-foundation-sys", + "iana-time-zone-haiku", + "js-sys", + "wasm-bindgen", + "winapi", +] + +[[package]] +name = "iana-time-zone-haiku" +version = "0.1.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "0703ae284fc167426161c2e3f1da3ea71d94b21bedbcc9494e92b28e334e3dca" +dependencies = [ + "cxx", + "cxx-build", +] + +[[package]] +name = "idna" +version = "0.3.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "e14ddfc70884202db2244c223200c204c2bda1bc6e0998d11b5e024d657209e6" +dependencies = [ + "unicode-bidi", + "unicode-normalization", +] + +[[package]] +name = "indexmap" +version = "1.9.2" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "1885e79c1fc4b10f0e172c475f458b7f7b93061064d98c3293e98c5ba0c8b399" +dependencies = [ + "autocfg", + "hashbrown", +] + +[[package]] +name = "inotify" +version = "0.9.6" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "f8069d3ec154eb856955c1c0fbffefbf5f3c40a104ec912d4797314c1801abff" +dependencies = [ + "bitflags", + "inotify-sys", + "libc", +] + +[[package]] +name = "inotify-sys" +version = "0.1.5" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "e05c02b5e89bff3b946cedeca278abc628fe811e604f027c45a8aa3cf793d0eb" +dependencies = [ + "libc", +] + +[[package]] +name = "instant" +version = "0.1.12" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "7a5bbe824c507c5da5956355e86a746d82e0e1464f65d862cc5e71da70e94b2c" +dependencies = [ + "cfg-if", +] + +[[package]] +name = "io-lifetimes" +version = "1.0.3" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "46112a93252b123d31a119a8d1a1ac19deac4fac6e0e8b0df58f0d4e5870e63c" +dependencies = [ + "libc", + "windows-sys", +] + +[[package]] +name = "is-terminal" +version = "0.4.2" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "28dfb6c8100ccc63462345b67d1bbc3679177c75ee4bf59bf29c8b1d110b8189" +dependencies = [ + "hermit-abi", + "io-lifetimes", + "rustix", + "windows-sys", +] + +[[package]] +name = "itoa" +version = "1.0.5" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "fad582f4b9e86b6caa621cabeb0963332d92eea04729ab12892c2533951e6440" + +[[package]] +name = "js-sys" +version = "0.3.60" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "49409df3e3bf0856b916e2ceaca09ee28e6871cf7d9ce97a692cacfdb2a25a47" +dependencies = [ + "wasm-bindgen", +] + +[[package]] +name = "kqueue" +version = "1.0.7" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "2c8fc60ba15bf51257aa9807a48a61013db043fcf3a78cb0d916e8e396dcad98" +dependencies = [ + "kqueue-sys", + "libc", +] + +[[package]] +name = "kqueue-sys" +version = "1.0.3" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "8367585489f01bc55dd27404dcf56b95e6da061a256a666ab23be9ba96a2e587" +dependencies = [ + "bitflags", + "libc", +] + +[[package]] +name = "lazy_static" +version = "1.4.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "e2abad23fbc42b3700f2f279844dc832adb2b2eb069b2df918f455c4e18cc646" + +[[package]] +name = "libc" +version = "0.2.139" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "201de327520df007757c1f0adce6e827fe8562fbc28bfd9c15571c66ca1f5f79" + +[[package]] +name = "link-cplusplus" +version = "1.0.8" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "ecd207c9c713c34f95a097a5b029ac2ce6010530c7b49d7fea24d977dede04f5" +dependencies = [ + "cc", +] + +[[package]] +name = "linux-raw-sys" +version = "0.1.4" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "f051f77a7c8e6957c0696eac88f26b0117e54f52d3fc682ab19397a8812846a4" + +[[package]] +name = "lock_api" +version = "0.4.9" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "435011366fe56583b16cf956f9df0095b405b82d76425bc8981c0e22e60ec4df" +dependencies = [ + "autocfg", + "scopeguard", +] + +[[package]] +name = "log" +version = "0.4.17" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "abb12e687cfb44aa40f41fc3978ef76448f9b6038cad6aef4259d3c095a2382e" +dependencies = [ + "cfg-if", +] + +[[package]] +name = "mac" +version = "0.1.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "c41e0c4fef86961ac6d6f8a82609f55f31b05e4fce149ac5710e439df7619ba4" + +[[package]] +name = "maplit" +version = "1.0.2" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "3e2e65a1a2e43cfcb47a895c4c8b10d1f4a61097f9f254f183aee60cad9c651d" + +[[package]] +name = "markup5ever" +version = "0.11.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "7a2629bb1404f3d34c2e921f21fd34ba00b206124c81f65c50b43b6aaefeb016" +dependencies = [ + "log", + "phf", + "phf_codegen", + "string_cache", + "string_cache_codegen", + "tendril", +] + +[[package]] +name = "mdbook" +version = "0.4.25" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "d1ed28d5903dde77bd5182645078a37ee57014cac6ccb2d54e1d6496386648e4" +dependencies = [ + "ammonia", + "anyhow", + "chrono", + "clap", + "clap_complete", + "elasticlunr-rs", + "env_logger", + "futures-util", + "gitignore", + "handlebars", + "log", + "memchr", + "notify", + "notify-debouncer-mini", + "once_cell", + "opener", + "pulldown-cmark", + "regex", + "serde", + "serde_json", + "shlex", + "tempfile", + "tokio", + "toml", + "topological-sort", + "warp", +] + +[[package]] +name = "memchr" +version = "2.5.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "2dffe52ecf27772e601905b7522cb4ef790d2cc203488bbd0e2fe85fcb74566d" + +[[package]] +name = "mime" +version = "0.3.16" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "2a60c7ce501c71e03a9c9c0d35b861413ae925bd979cc7a4e30d060069aaac8d" + +[[package]] +name = "mime_guess" +version = "2.0.4" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "4192263c238a5f0d0c6bfd21f336a313a4ce1c450542449ca191bb657b4642ef" +dependencies = [ + "mime", + "unicase", +] + +[[package]] +name = "mio" +version = "0.8.5" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "e5d732bc30207a6423068df043e3d02e0735b155ad7ce1a6f76fe2baa5b158de" +dependencies = [ + "libc", + "log", + "wasi 0.11.0+wasi-snapshot-preview1", + "windows-sys", +] + +[[package]] +name = "new_debug_unreachable" +version = "1.0.4" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "e4a24736216ec316047a1fc4252e27dabb04218aa4a3f37c6e7ddbf1f9782b54" + +[[package]] +name = "notify" +version = "5.0.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "ed2c66da08abae1c024c01d635253e402341b4060a12e99b31c7594063bf490a" +dependencies = [ + "bitflags", + "crossbeam-channel", + "filetime", + "fsevent-sys", + "inotify", + "kqueue", + "libc", + "mio", + "walkdir", + "winapi", +] + +[[package]] +name = "notify-debouncer-mini" +version = "0.2.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "e23e9fa24f094b143c1eb61f90ac6457de87be6987bc70746e0179f7dbc9007b" +dependencies = [ + "crossbeam-channel", + "notify", +] + +[[package]] +name = "num-integer" +version = "0.1.45" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "225d3389fb3509a24c93f5c29eb6bde2586b98d9f016636dff58d7c6f7569cd9" +dependencies = [ + "autocfg", + "num-traits", +] + +[[package]] +name = "num-traits" +version = "0.2.15" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "578ede34cf02f8924ab9447f50c28075b4d3e5b269972345e7e0372b38c6cdcd" +dependencies = [ + "autocfg", +] + +[[package]] +name = "num_cpus" +version = "1.15.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "0fac9e2da13b5eb447a6ce3d392f23a29d8694bff781bf03a16cd9ac8697593b" +dependencies = [ + "hermit-abi", + "libc", +] + +[[package]] +name = "once_cell" +version = "1.17.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "6f61fba1741ea2b3d6a1e3178721804bb716a68a6aeba1149b5d52e3d464ea66" + +[[package]] +name = "opener" +version = "0.5.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "4ea3ebcd72a54701f56345f16785a6d3ac2df7e986d273eb4395c0b01db17952" +dependencies = [ + "bstr", + "winapi", +] + +[[package]] +name = "os_str_bytes" +version = "6.4.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "9b7820b9daea5457c9f21c69448905d723fbd21136ccf521748f23fd49e723ee" + +[[package]] +name = "parking_lot" +version = "0.12.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "3742b2c103b9f06bc9fff0a37ff4912935851bee6d36f3c02bcc755bcfec228f" +dependencies = [ + "lock_api", + "parking_lot_core", +] + +[[package]] +name = "parking_lot_core" +version = "0.9.5" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "7ff9f3fef3968a3ec5945535ed654cb38ff72d7495a25619e2247fb15a2ed9ba" +dependencies = [ + "cfg-if", + "libc", + "redox_syscall", + "smallvec", + "windows-sys", +] + +[[package]] +name = "percent-encoding" +version = "2.2.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "478c572c3d73181ff3c2539045f6eb99e5491218eae919370993b890cdbdd98e" + +[[package]] +name = "pest" +version = "2.5.2" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "0f6e86fb9e7026527a0d46bc308b841d73170ef8f443e1807f6ef88526a816d4" +dependencies = [ + "thiserror", + "ucd-trie", +] + +[[package]] +name = "pest_derive" +version = "2.5.2" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "96504449aa860c8dcde14f9fba5c58dc6658688ca1fe363589d6327b8662c603" +dependencies = [ + "pest", + "pest_generator", +] + +[[package]] +name = "pest_generator" +version = "2.5.2" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "798e0220d1111ae63d66cb66a5dcb3fc2d986d520b98e49e1852bfdb11d7c5e7" +dependencies = [ + "pest", + "pest_meta", + "proc-macro2", + "quote", + "syn", +] + +[[package]] +name = "pest_meta" +version = "2.5.2" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "984298b75898e30a843e278a9f2452c31e349a073a0ce6fd950a12a74464e065" +dependencies = [ + "once_cell", + "pest", + "sha1", +] + +[[package]] +name = "phf" +version = "0.10.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "fabbf1ead8a5bcbc20f5f8b939ee3f5b0f6f281b6ad3468b84656b658b455259" +dependencies = [ + "phf_shared", +] + +[[package]] +name = "phf_codegen" +version = "0.10.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "4fb1c3a8bc4dd4e5cfce29b44ffc14bedd2ee294559a294e2a4d4c9e9a6a13cd" +dependencies = [ + "phf_generator", + "phf_shared", +] + +[[package]] +name = "phf_generator" +version = "0.10.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "5d5285893bb5eb82e6aaf5d59ee909a06a16737a8970984dd7746ba9283498d6" +dependencies = [ + "phf_shared", + "rand", +] + +[[package]] +name = "phf_shared" +version = "0.10.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "b6796ad771acdc0123d2a88dc428b5e38ef24456743ddb1744ed628f9815c096" +dependencies = [ + "siphasher", +] + +[[package]] +name = "pin-project" +version = "1.0.12" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "ad29a609b6bcd67fee905812e544992d216af9d755757c05ed2d0e15a74c6ecc" +dependencies = [ + "pin-project-internal", +] + +[[package]] +name = "pin-project-internal" +version = "1.0.12" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "069bdb1e05adc7a8990dce9cc75370895fbe4e3d58b9b73bf1aee56359344a55" +dependencies = [ + "proc-macro2", + "quote", + "syn", +] + +[[package]] +name = "pin-project-lite" +version = "0.2.9" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "e0a7ae3ac2f1173085d398531c705756c94a4c56843785df85a60c1a0afac116" + +[[package]] +name = "pin-utils" +version = "0.1.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "8b870d8c151b6f2fb93e84a13146138f05d02ed11c7e7c54f8826aaaf7c9f184" + +[[package]] +name = "polib" +version = "0.1.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "17009af1604eef4137497a743594fbe8f37e52f004cb5d8f7cf5130dc74a5644" + +[[package]] +name = "ppv-lite86" +version = "0.2.17" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "5b40af805b3121feab8a3c29f04d8ad262fa8e0561883e7653e024ae4479e6de" + +[[package]] +name = "precomputed-hash" +version = "0.1.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "925383efa346730478fb4838dbe9137d2a47675ad789c546d150a6e1dd4ab31c" + +[[package]] +name = "proc-macro2" +version = "1.0.49" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "57a8eca9f9c4ffde41714334dee777596264c7825420f521abc92b5b5deb63a5" +dependencies = [ + "unicode-ident", +] + +[[package]] +name = "pulldown-cmark" +version = "0.9.2" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "2d9cc634bc78768157b5cbfe988ffcd1dcba95cd2b2f03a88316c08c6d00ed63" +dependencies = [ + "bitflags", + "memchr", + "unicase", +] + +[[package]] +name = "quote" +version = "1.0.23" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "8856d8364d252a14d474036ea1358d63c9e6965c8e5c1885c18f73d70bff9c7b" +dependencies = [ + "proc-macro2", +] + +[[package]] +name = "rand" +version = "0.8.5" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "34af8d1a0e25924bc5b7c43c079c942339d8f0a8b57c39049bef581b46327404" +dependencies = [ + "libc", + "rand_chacha", + "rand_core", +] + +[[package]] +name = "rand_chacha" +version = "0.3.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "e6c10a63a0fa32252be49d21e7709d4d4baf8d231c2dbce1eaa8141b9b127d88" +dependencies = [ + "ppv-lite86", + "rand_core", +] + +[[package]] +name = "rand_core" +version = "0.6.4" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "ec0be4795e2f6a28069bec0b5ff3e2ac9bafc99e6a9a7dc3547996c5c816922c" +dependencies = [ + "getrandom", +] + +[[package]] +name = "redox_syscall" +version = "0.2.16" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "fb5a58c1855b4b6819d59012155603f0b22ad30cad752600aadfcb695265519a" +dependencies = [ + "bitflags", +] + +[[package]] +name = "regex" +version = "1.7.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "e076559ef8e241f2ae3479e36f97bd5741c0330689e217ad51ce2c76808b868a" +dependencies = [ + "aho-corasick", + "memchr", + "regex-syntax", +] + +[[package]] +name = "regex-automata" +version = "0.1.10" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "6c230d73fb8d8c1b9c0b3135c5142a8acee3a0558fb8db5cf1cb65f8d7862132" + +[[package]] +name = "regex-syntax" +version = "0.6.28" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "456c603be3e8d448b072f410900c09faf164fbce2d480456f50eea6e25f9c848" + +[[package]] +name = "remove_dir_all" +version = "0.5.3" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "3acd125665422973a33ac9d3dd2df85edad0f4ae9b00dafb1a05e43a9f5ef8e7" +dependencies = [ + "winapi", +] + +[[package]] +name = "rustix" +version = "0.36.6" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "4feacf7db682c6c329c4ede12649cd36ecab0f3be5b7d74e6a20304725db4549" +dependencies = [ + "bitflags", + "errno", + "io-lifetimes", + "libc", + "linux-raw-sys", + "windows-sys", +] + +[[package]] +name = "rustls-pemfile" +version = "0.2.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "5eebeaeb360c87bfb72e84abdb3447159c0eaececf1bef2aecd65a8be949d1c9" +dependencies = [ + "base64", +] + +[[package]] +name = "ryu" +version = "1.0.12" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "7b4b9743ed687d4b4bcedf9ff5eaa7398495ae14e61cba0a295704edbc7decde" + +[[package]] +name = "same-file" +version = "1.0.6" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "93fc1dc3aaa9bfed95e02e6eadabb4baf7e3078b0bd1b4d7b6b0b68378900502" +dependencies = [ + "winapi-util", +] + +[[package]] +name = "scoped-tls" +version = "1.0.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "e1cf6437eb19a8f4a6cc0f7dca544973b0b78843adbfeb3683d1a94a0024a294" + +[[package]] +name = "scopeguard" +version = "1.1.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "d29ab0c6d3fc0ee92fe66e2d99f700eab17a8d57d1c1d3b748380fb20baa78cd" + +[[package]] +name = "scratch" +version = "1.0.3" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "ddccb15bcce173023b3fedd9436f882a0739b8dfb45e4f6b6002bee5929f61b2" + +[[package]] +name = "semver" +version = "1.0.16" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "58bc9567378fc7690d6b2addae4e60ac2eeea07becb2c64b9f218b53865cba2a" + +[[package]] +name = "serde" +version = "1.0.152" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "bb7d1f0d3021d347a83e556fc4683dea2ea09d87bccdf88ff5c12545d89d5efb" +dependencies = [ + "serde_derive", +] + +[[package]] +name = "serde_derive" +version = "1.0.152" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "af487d118eecd09402d70a5d72551860e788df87b464af30e5ea6a38c75c541e" +dependencies = [ + "proc-macro2", + "quote", + "syn", +] + +[[package]] +name = "serde_json" +version = "1.0.91" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "877c235533714907a8c2464236f5c4b2a17262ef1bd71f38f35ea592c8da6883" +dependencies = [ + "itoa", + "ryu", + "serde", +] + +[[package]] +name = "serde_urlencoded" +version = "0.7.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "d3491c14715ca2294c4d6a88f15e84739788c1d030eed8c110436aafdaa2f3fd" +dependencies = [ + "form_urlencoded", + "itoa", + "ryu", + "serde", +] + +[[package]] +name = "sha-1" +version = "0.10.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "f5058ada175748e33390e40e872bd0fe59a19f265d0158daa551c5a88a76009c" +dependencies = [ + "cfg-if", + "cpufeatures", + "digest", +] + +[[package]] +name = "sha1" +version = "0.10.5" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "f04293dc80c3993519f2d7f6f511707ee7094fe0c6d3406feb330cdb3540eba3" +dependencies = [ + "cfg-if", + "cpufeatures", + "digest", +] + +[[package]] +name = "shlex" +version = "1.1.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "43b2853a4d09f215c24cc5489c992ce46052d359b5109343cbafbf26bc62f8a3" + +[[package]] +name = "siphasher" +version = "0.3.10" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "7bd3e3206899af3f8b12af284fafc038cc1dc2b41d1b89dd17297221c5d225de" + +[[package]] +name = "slab" +version = "0.4.7" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "4614a76b2a8be0058caa9dbbaf66d988527d86d003c11a94fbd335d7661edcef" +dependencies = [ + "autocfg", +] + +[[package]] +name = "smallvec" +version = "1.10.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "a507befe795404456341dfab10cef66ead4c041f62b8b11bbb92bffe5d0953e0" + +[[package]] +name = "socket2" +version = "0.4.7" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "02e2d2db9033d13a1567121ddd7a095ee144db4e1ca1b1bda3419bc0da294ebd" +dependencies = [ + "libc", + "winapi", +] + +[[package]] +name = "string_cache" +version = "0.8.4" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "213494b7a2b503146286049378ce02b482200519accc31872ee8be91fa820a08" +dependencies = [ + "new_debug_unreachable", + "once_cell", + "parking_lot", + "phf_shared", + "precomputed-hash", + "serde", +] + +[[package]] +name = "string_cache_codegen" +version = "0.5.2" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "6bb30289b722be4ff74a408c3cc27edeaad656e06cb1fe8fa9231fa59c728988" +dependencies = [ + "phf_generator", + "phf_shared", + "proc-macro2", + "quote", +] + +[[package]] +name = "strsim" +version = "0.10.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "73473c0e59e6d5812c5dfe2a064a6444949f089e20eec9a2e5506596494e4623" + +[[package]] +name = "syn" +version = "1.0.107" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "1f4064b5b16e03ae50984a5a8ed5d4f8803e6bc1fd170a3cda91a1be4b18e3f5" +dependencies = [ + "proc-macro2", + "quote", + "unicode-ident", +] + +[[package]] +name = "tempfile" +version = "3.3.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "5cdb1ef4eaeeaddc8fbd371e5017057064af0911902ef36b39801f67cc6d79e4" +dependencies = [ + "cfg-if", + "fastrand", + "libc", + "redox_syscall", + "remove_dir_all", + "winapi", +] + +[[package]] +name = "tendril" +version = "0.4.3" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "d24a120c5fc464a3458240ee02c299ebcb9d67b5249c8848b09d639dca8d7bb0" +dependencies = [ + "futf", + "mac", + "utf-8", +] + +[[package]] +name = "termcolor" +version = "1.1.3" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "bab24d30b911b2376f3a13cc2cd443142f0c81dda04c118693e35b3835757755" +dependencies = [ + "winapi-util", +] + +[[package]] +name = "terminal_size" +version = "0.2.3" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "cb20089a8ba2b69debd491f8d2d023761cbf196e999218c591fa1e7e15a21907" +dependencies = [ + "rustix", + "windows-sys", +] + +[[package]] +name = "thiserror" +version = "1.0.38" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "6a9cd18aa97d5c45c6603caea1da6628790b37f7a34b6ca89522331c5180fed0" +dependencies = [ + "thiserror-impl", +] + +[[package]] +name = "thiserror-impl" +version = "1.0.38" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "1fb327af4685e4d03fa8cbcf1716380da910eeb2bb8be417e7f9fd3fb164f36f" +dependencies = [ + "proc-macro2", + "quote", + "syn", +] + +[[package]] +name = "time" +version = "0.1.45" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "1b797afad3f312d1c66a56d11d0316f916356d11bd158fbc6ca6389ff6bf805a" +dependencies = [ + "libc", + "wasi 0.10.0+wasi-snapshot-preview1", + "winapi", +] + +[[package]] +name = "tinyvec" +version = "1.6.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "87cc5ceb3875bb20c2890005a4e226a4651264a5c75edb2421b52861a0a0cb50" +dependencies = [ + "tinyvec_macros", +] + +[[package]] +name = "tinyvec_macros" +version = "0.1.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "cda74da7e1a664f795bb1f8a87ec406fb89a02522cf6e50620d016add6dbbf5c" + +[[package]] +name = "tokio" +version = "1.24.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "1d9f76183f91ecfb55e1d7d5602bd1d979e38a3a522fe900241cf195624d67ae" +dependencies = [ + "autocfg", + "bytes", + "libc", + "memchr", + "mio", + "num_cpus", + "pin-project-lite", + "socket2", + "tokio-macros", + "windows-sys", +] + +[[package]] +name = "tokio-macros" +version = "1.8.2" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "d266c00fde287f55d3f1c3e96c500c362a2b8c695076ec180f27918820bc6df8" +dependencies = [ + "proc-macro2", + "quote", + "syn", +] + +[[package]] +name = "tokio-stream" +version = "0.1.11" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "d660770404473ccd7bc9f8b28494a811bc18542b915c0855c51e8f419d5223ce" +dependencies = [ + "futures-core", + "pin-project-lite", + "tokio", +] + +[[package]] +name = "tokio-tungstenite" +version = "0.17.2" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "f714dd15bead90401d77e04243611caec13726c2408afd5b31901dfcdcb3b181" +dependencies = [ + "futures-util", + "log", + "tokio", + "tungstenite", +] + +[[package]] +name = "tokio-util" +version = "0.7.4" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "0bb2e075f03b3d66d8d8785356224ba688d2906a371015e225beeb65ca92c740" +dependencies = [ + "bytes", + "futures-core", + "futures-sink", + "pin-project-lite", + "tokio", + "tracing", +] + +[[package]] +name = "toml" +version = "0.5.10" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "1333c76748e868a4d9d1017b5ab53171dfd095f70c712fdb4653a406547f598f" +dependencies = [ + "serde", +] + +[[package]] +name = "topological-sort" +version = "0.2.2" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "ea68304e134ecd095ac6c3574494fc62b909f416c4fca77e440530221e549d3d" + +[[package]] +name = "tower-service" +version = "0.3.2" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "b6bc1c9ce2b5135ac7f93c72918fc37feb872bdc6a5533a8b85eb4b86bfdae52" + +[[package]] +name = "tracing" +version = "0.1.37" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "8ce8c33a8d48bd45d624a6e523445fd21ec13d3653cd51f681abf67418f54eb8" +dependencies = [ + "cfg-if", + "log", + "pin-project-lite", + "tracing-core", +] + +[[package]] +name = "tracing-core" +version = "0.1.30" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "24eb03ba0eab1fd845050058ce5e616558e8f8d8fca633e6b163fe25c797213a" +dependencies = [ + "once_cell", +] + +[[package]] +name = "try-lock" +version = "0.2.4" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "3528ecfd12c466c6f163363caf2d02a71161dd5e1cc6ae7b34207ea2d42d81ed" + +[[package]] +name = "tungstenite" +version = "0.17.3" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "e27992fd6a8c29ee7eef28fc78349aa244134e10ad447ce3b9f0ac0ed0fa4ce0" +dependencies = [ + "base64", + "byteorder", + "bytes", + "http", + "httparse", + "log", + "rand", + "sha-1", + "thiserror", + "url", + "utf-8", +] + +[[package]] +name = "typenum" +version = "1.16.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "497961ef93d974e23eb6f433eb5fe1b7930b659f06d12dec6fc44a8f554c0bba" + +[[package]] +name = "ucd-trie" +version = "0.1.5" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "9e79c4d996edb816c91e4308506774452e55e95c3c9de07b6729e17e15a5ef81" + +[[package]] +name = "unicase" +version = "2.6.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "50f37be617794602aabbeee0be4f259dc1778fabe05e2d67ee8f79326d5cb4f6" +dependencies = [ + "version_check", +] + +[[package]] +name = "unicode-bidi" +version = "0.3.8" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "099b7128301d285f79ddd55b9a83d5e6b9e97c92e0ea0daebee7263e932de992" + +[[package]] +name = "unicode-ident" +version = "1.0.6" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "84a22b9f218b40614adcb3f4ff08b703773ad44fa9423e4e0d346d5db86e4ebc" + +[[package]] +name = "unicode-normalization" +version = "0.1.22" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "5c5713f0fc4b5db668a2ac63cdb7bb4469d8c9fed047b1d0292cc7b0ce2ba921" +dependencies = [ + "tinyvec", +] + +[[package]] +name = "unicode-width" +version = "0.1.10" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "c0edd1e5b14653f783770bce4a4dabb4a5108a5370a5f5d8cfe8710c361f6c8b" + +[[package]] +name = "url" +version = "2.3.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "0d68c799ae75762b8c3fe375feb6600ef5602c883c5d21eb51c09f22b83c4643" +dependencies = [ + "form_urlencoded", + "idna", + "percent-encoding", +] + +[[package]] +name = "utf-8" +version = "0.7.6" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "09cc8ee72d2a9becf2f2febe0205bbed8fc6615b7cb429ad062dc7b7ddd036a9" + +[[package]] +name = "version_check" +version = "0.9.4" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "49874b5167b65d7193b8aba1567f5c7d93d001cafc34600cee003eda787e483f" + +[[package]] +name = "walkdir" +version = "2.3.2" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "808cf2735cd4b6866113f648b791c6adc5714537bc222d9347bb203386ffda56" +dependencies = [ + "same-file", + "winapi", + "winapi-util", +] + +[[package]] +name = "want" +version = "0.3.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "1ce8a968cb1cd110d136ff8b819a556d6fb6d919363c61534f6860c7eb172ba0" +dependencies = [ + "log", + "try-lock", +] + +[[package]] +name = "warp" +version = "0.3.3" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "ed7b8be92646fc3d18b06147664ebc5f48d222686cb11a8755e561a735aacc6d" +dependencies = [ + "bytes", + "futures-channel", + "futures-util", + "headers", + "http", + "hyper", + "log", + "mime", + "mime_guess", + "percent-encoding", + "pin-project", + "rustls-pemfile", + "scoped-tls", + "serde", + "serde_json", + "serde_urlencoded", + "tokio", + "tokio-stream", + "tokio-tungstenite", + "tokio-util", + "tower-service", + "tracing", +] + +[[package]] +name = "wasi" +version = "0.10.0+wasi-snapshot-preview1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "1a143597ca7c7793eff794def352d41792a93c481eb1042423ff7ff72ba2c31f" + +[[package]] +name = "wasi" +version = "0.11.0+wasi-snapshot-preview1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "9c8d87e72b64a3b4db28d11ce29237c246188f4f51057d65a7eab63b7987e423" + +[[package]] +name = "wasm-bindgen" +version = "0.2.83" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "eaf9f5aceeec8be17c128b2e93e031fb8a4d469bb9c4ae2d7dc1888b26887268" +dependencies = [ + "cfg-if", + "wasm-bindgen-macro", +] + +[[package]] +name = "wasm-bindgen-backend" +version = "0.2.83" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "4c8ffb332579b0557b52d268b91feab8df3615f265d5270fec2a8c95b17c1142" +dependencies = [ + "bumpalo", + "log", + "once_cell", + "proc-macro2", + "quote", + "syn", + "wasm-bindgen-shared", +] + +[[package]] +name = "wasm-bindgen-macro" +version = "0.2.83" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "052be0f94026e6cbc75cdefc9bae13fd6052cdcaf532fa6c45e7ae33a1e6c810" +dependencies = [ + "quote", + "wasm-bindgen-macro-support", +] + +[[package]] +name = "wasm-bindgen-macro-support" +version = "0.2.83" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "07bc0c051dc5f23e307b13285f9d75df86bfdf816c5721e573dec1f9b8aa193c" +dependencies = [ + "proc-macro2", + "quote", + "syn", + "wasm-bindgen-backend", + "wasm-bindgen-shared", +] + +[[package]] +name = "wasm-bindgen-shared" +version = "0.2.83" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "1c38c045535d93ec4f0b4defec448e4291638ee608530863b1e2ba115d4fff7f" + +[[package]] +name = "winapi" +version = "0.3.9" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "5c839a674fcd7a98952e593242ea400abe93992746761e38641405d28b00f419" +dependencies = [ + "winapi-i686-pc-windows-gnu", + "winapi-x86_64-pc-windows-gnu", +] + +[[package]] +name = "winapi-i686-pc-windows-gnu" +version = "0.4.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "ac3b87c63620426dd9b991e5ce0329eff545bccbbb34f3be09ff6fb6ab51b7b6" + +[[package]] +name = "winapi-util" +version = "0.1.5" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "70ec6ce85bb158151cae5e5c87f95a8e97d2c0c4b001223f33a334e3ce5de178" +dependencies = [ + "winapi", +] + +[[package]] +name = "winapi-x86_64-pc-windows-gnu" +version = "0.4.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "712e227841d057c1ee1cd2fb22fa7e5a5461ae8e48fa2ca79ec42cfc1931183f" + +[[package]] +name = "windows-sys" +version = "0.42.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "5a3e1820f08b8513f676f7ab6c1f99ff312fb97b553d30ff4dd86f9f15728aa7" +dependencies = [ + "windows_aarch64_gnullvm", + "windows_aarch64_msvc", + "windows_i686_gnu", + "windows_i686_msvc", + "windows_x86_64_gnu", + "windows_x86_64_gnullvm", + "windows_x86_64_msvc", +] + +[[package]] +name = "windows_aarch64_gnullvm" +version = "0.42.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "41d2aa71f6f0cbe00ae5167d90ef3cfe66527d6f613ca78ac8024c3ccab9a19e" + +[[package]] +name = "windows_aarch64_msvc" +version = "0.42.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "dd0f252f5a35cac83d6311b2e795981f5ee6e67eb1f9a7f64eb4500fbc4dcdb4" + +[[package]] +name = "windows_i686_gnu" +version = "0.42.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "fbeae19f6716841636c28d695375df17562ca208b2b7d0dc47635a50ae6c5de7" + +[[package]] +name = "windows_i686_msvc" +version = "0.42.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "84c12f65daa39dd2babe6e442988fc329d6243fdce47d7d2d155b8d874862246" + +[[package]] +name = "windows_x86_64_gnu" +version = "0.42.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "bf7b1b21b5362cbc318f686150e5bcea75ecedc74dd157d874d754a2ca44b0ed" + +[[package]] +name = "windows_x86_64_gnullvm" +version = "0.42.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "09d525d2ba30eeb3297665bd434a54297e4170c7f1a44cad4ef58095b4cd2028" + +[[package]] +name = "windows_x86_64_msvc" +version = "0.42.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "f40009d85759725a34da6d89a94e63d7bdc50a862acf0dbc7c8e488f1edcb6f5" diff --git a/i18n-helpers/Cargo.toml b/i18n-helpers/Cargo.toml new file mode 100644 index 000000000000..adc750292d39 --- /dev/null +++ b/i18n-helpers/Cargo.toml @@ -0,0 +1,14 @@ +[package] +name = "i18n-helpers" +version = "0.1.0" +edition = "2021" +publish = false + +[dependencies] +anyhow = "1.0.68" +mdbook = "0.4.25" +once_cell = "1.17.0" +polib = "0.1.0" +regex = "1.7.0" +semver = "1.0.16" +serde_json = "1.0.91" diff --git a/i18n-helpers/src/bin/mdbook-gettext.rs b/i18n-helpers/src/bin/mdbook-gettext.rs new file mode 100644 index 000000000000..7c14a8319f30 --- /dev/null +++ b/i18n-helpers/src/bin/mdbook-gettext.rs @@ -0,0 +1,230 @@ +// Copyright 2023 Google LLC +// +// Licensed under the Apache License, Version 2.0 (the "License"); +// you may not use this file except in compliance with the License. +// You may obtain a copy of the License at +// +// http://www.apache.org/licenses/LICENSE-2.0 +// +// Unless required by applicable law or agreed to in writing, software +// distributed under the License is distributed on an "AS IS" BASIS, +// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +// See the License for the specific language governing permissions and +// limitations under the License. + +//! `gettext` for `mdbook` +//! +//! This program works like `gettext`, meaning it will translate +//! strings in your book. +//! +//! The translations come from GNU Gettext `xx.po` files. You must set +//! preprocessor.gettext.po-file to the PO file to use. If unset, a +//! warning is issued while building the book. +//! +//! See `TRANSLATIONS.md` in the repository root for more information. + +use anyhow::{anyhow, Context}; +use i18n_helpers::extract_paragraphs; +use mdbook::book::Book; +use mdbook::preprocess::{CmdPreprocessor, PreprocessorContext}; +use mdbook::BookItem; +use polib::catalog::Catalog; +use polib::po_file; +use semver::{Version, VersionReq}; +use std::io; +use std::path::Path; +use std::process; + +fn translate(text: &str, catalog: &Catalog) -> String { + let mut output = String::with_capacity(text.len()); + let mut target_lineno = 1; + + for (lineno, paragraph) in extract_paragraphs(text) { + // Fill in blank lines between paragraphs. This is important + // for code blocks where blank lines are significant. + while target_lineno < lineno { + output.push('\n'); + target_lineno += 1; + } + // Subtract 1 because the paragraph is missing a final '\n' + // due to the splitting in `extract_paragraphs`. + target_lineno += paragraph.lines().count() - 1; + + let translated = catalog + .find_message(paragraph) + .and_then(|msg| msg.get_msgstr().ok()) + .filter(|msgstr| !msgstr.is_empty()) + .map(|msgstr| msgstr.as_str()) + .unwrap_or(paragraph); + output.push_str(translated); + } + + let suffix = &text[text.trim_end_matches('\n').len()..]; + output.push_str(suffix); + output +} + +fn translate_book(ctx: &PreprocessorContext, mut book: Book) -> anyhow::Result { + let cfg = ctx + .config + .get_preprocessor("gettext") + .ok_or_else(|| anyhow!("Could not read preprocessor.gettext configuration"))?; + let path = cfg + .get("po-file") + .ok_or_else(|| anyhow!("Missing preprocessor.gettext.po-file config value"))? + .as_str() + .ok_or_else(|| anyhow!("Expected a string for preprocessor.gettext.po-file"))?; + let catalog = po_file::parse(Path::new(path)) + .map_err(|err| anyhow!("{err}")) + .with_context(|| format!("Could not parse {path} as PO file"))?; + + book.for_each_mut(|item| match item { + BookItem::Chapter(ch) => { + ch.content = translate(&ch.content, &catalog); + ch.name = translate(&ch.name, &catalog); + } + BookItem::Separator => {} + BookItem::PartTitle(title) => { + *title = translate(title, &catalog); + } + }); + + Ok(book) +} + +fn preprocess() -> anyhow::Result<()> { + let (ctx, book) = CmdPreprocessor::parse_input(io::stdin())?; + let book_version = Version::parse(&ctx.mdbook_version)?; + let version_req = VersionReq::parse(mdbook::MDBOOK_VERSION)?; + if !version_req.matches(&book_version) { + eprintln!( + "Warning: The gettext preprocessor was built against \ + mdbook version {}, but we're being called from version {}", + mdbook::MDBOOK_VERSION, + ctx.mdbook_version + ); + } + + let translated_book = translate_book(&ctx, book)?; + serde_json::to_writer(io::stdout(), &translated_book)?; + + Ok(()) +} + +fn main() -> anyhow::Result<()> { + if std::env::args().len() == 3 { + assert_eq!(std::env::args().nth(1).as_deref(), Some("supports")); + // Signal that we support all renderers. + process::exit(0); + } + + preprocess() +} + +#[cfg(test)] +mod tests { + use super::*; + use polib::message::Message; + + fn create_catalog(translations: &[(&str, &str)]) -> Catalog { + let mut catalog = Catalog::new(); + for (msgid, msgstr) in translations { + let message = Message::new_singular("", "", "", "", msgid, msgstr); + catalog.add_message(message); + } + catalog + } + + #[test] + fn test_translate_single_line() { + let catalog = create_catalog(&[("foo bar", "FOO BAR")]); + assert_eq!(translate("foo bar", &catalog), "FOO BAR"); + } + + #[test] + fn test_translate_single_paragraph() { + let catalog = create_catalog(&[("foo bar", "FOO BAR")]); + assert_eq!(translate("foo bar\n", &catalog), "FOO BAR\n"); + } + + #[test] + fn test_translate_paragraph_with_leading_newlines() { + let catalog = create_catalog(&[("foo bar", "FOO BAR")]); + assert_eq!(translate("\n\n\nfoo bar\n", &catalog), "\n\n\nFOO BAR\n"); + } + + #[test] + fn test_translate_paragraph_with_trailing_newlines() { + let catalog = create_catalog(&[("foo bar", "FOO BAR")]); + assert_eq!(translate("foo bar\n\n\n", &catalog), "FOO BAR\n\n\n"); + } + + #[test] + fn test_translate_multiple_paragraphs() { + let catalog = create_catalog(&[("foo bar", "FOO BAR")]); + assert_eq!( + translate( + "first paragraph\n\ + \n\ + foo bar\n\ + \n\ + last paragraph\n", + &catalog + ), + "first paragraph\n\ + \n\ + FOO BAR\n\ + \n\ + last paragraph\n" + ); + } + + #[test] + fn test_translate_multiple_paragraphs_extra_newlines() { + // Notice how the translated paragraphs have more lines. + let catalog = create_catalog(&[ + ( + "first\n\ + paragraph", + "FIRST\n\ + TRANSLATED\n\ + PARAGRAPH", + ), + ( + "last\n\ + paragraph", + "LAST\n\ + TRANSLATED\n\ + PARAGRAPH", + ), + ]); + // Paragraph separation is kept intact while translating. + assert_eq!( + translate( + "\n\ + first\n\ + paragraph\n\ + \n\ + \n\ + \n\ + last\n\ + paragraph\n\ + \n\ + \n", + &catalog + ), + "\n\ + FIRST\n\ + TRANSLATED\n\ + PARAGRAPH\n\ + \n\ + \n\ + \n\ + LAST\n\ + TRANSLATED\n\ + PARAGRAPH\n\ + \n\ + \n" + ); + } +} diff --git a/i18n-helpers/src/bin/mdbook-xgettext.rs b/i18n-helpers/src/bin/mdbook-xgettext.rs new file mode 100644 index 000000000000..630c8d95f082 --- /dev/null +++ b/i18n-helpers/src/bin/mdbook-xgettext.rs @@ -0,0 +1,128 @@ +// Copyright 2023 Google LLC +// +// Licensed under the Apache License, Version 2.0 (the "License"); +// you may not use this file except in compliance with the License. +// You may obtain a copy of the License at +// +// http://www.apache.org/licenses/LICENSE-2.0 +// +// Unless required by applicable law or agreed to in writing, software +// distributed under the License is distributed on an "AS IS" BASIS, +// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +// See the License for the specific language governing permissions and +// limitations under the License. + +//! `xgettext` for `mdbook` +//! +//! This program works like `xgettext`, meaning it will extract +//! translatable strings from your book. The strings are saved in a +//! GNU Gettext `messages.pot` file in your build directory (typically +//! `po/messages.pot`). +//! +//! See `TRANSLATIONS.md` in the repository root for more information. + +use anyhow::{anyhow, Context}; +use mdbook::renderer::RenderContext; +use mdbook::BookItem; +use polib::catalog::Catalog; +use polib::message::Message; +use std::fs; +use std::io; + +fn add_message(catalog: &mut Catalog, msgid: &str, source: &str) { + let sources = match catalog.find_message(msgid) { + Some(msg) => format!("{}\n{}", msg.source, source), + None => String::from(source), + }; + let message = Message::new_singular("", &sources, "", "", msgid, ""); + + // Carefully update the existing message or add a new one. It's an + // error to create a catalog with duplicate msgids. + match catalog.find_message_index(msgid) { + Some(&idx) => catalog.update_message_by_index(idx, message).unwrap(), + None => catalog.add_message(message), + } +} + +fn create_catalog(ctx: &RenderContext) -> anyhow::Result { + let mut catalog = Catalog::new(); + if let Some(title) = &ctx.config.book.title { + catalog.metadata.project_id_version = String::from(title); + } + if let Some(lang) = &ctx.config.book.language { + catalog.metadata.language = String::from(lang); + } + catalog.metadata.mime_version = String::from("1.0"); + catalog.metadata.content_type = String::from("text/plain; charset=UTF-8"); + catalog.metadata.content_transfer_encoding = String::from("8bit"); + + let summary_path = ctx.config.book.src.join("SUMMARY.md"); + let summary = std::fs::read_to_string(ctx.root.join(&summary_path))?; + + // First, add all chapter names and part titles from SUMMARY.md. + // The book items are in order of the summary, so we can assign + // correct line numbers for duplicate lines by tracking the index + // of our last search. + let mut last_idx = 0; + for item in ctx.book.iter() { + let line = match item { + BookItem::Chapter(chapter) => &chapter.name, + BookItem::PartTitle(title) => title, + BookItem::Separator => continue, + }; + + let idx = summary[last_idx..].find(line).ok_or_else(|| { + anyhow!( + "Could not find {line:?} in SUMMARY.md after line {} -- \ + please remove any formatting from SUMMARY.md", + summary[..last_idx].lines().count() + ) + })?; + last_idx += idx; + let lineno = summary[..last_idx].lines().count(); + let source = format!("{}:{}", summary_path.display(), lineno); + add_message(&mut catalog, line, &source); + } + + // Next, we add the chapter contents. + for item in ctx.book.iter() { + if let BookItem::Chapter(chapter) = item { + let path = match &chapter.path { + Some(path) => ctx.config.book.src.join(path), + None => continue, + }; + for (lineno, paragraph) in i18n_helpers::extract_paragraphs(&chapter.content) + { + let source = format!("{}:{}", path.display(), lineno); + add_message(&mut catalog, paragraph, &source); + } + } + } + + Ok(catalog) +} + +fn main() -> anyhow::Result<()> { + let ctx = RenderContext::from_json(&mut io::stdin()).context("Parsing stdin")?; + let cfg = ctx + .config + .get_renderer("xgettext") + .ok_or_else(|| anyhow!("Could not read output.xgettext configuration"))?; + let path = cfg + .get("pot-file") + .ok_or_else(|| anyhow!("Missing output.xgettext.pot-file config value"))? + .as_str() + .ok_or_else(|| anyhow!("Expected a string for output.xgettext.pot-file"))?; + fs::create_dir_all(&ctx.destination) + .with_context(|| format!("Could not create {}", ctx.destination.display()))?; + let output_path = ctx.destination.join(path); + if output_path.exists() { + fs::remove_file(&output_path) + .with_context(|| format!("Removing {}", output_path.display()))? + } + let catalog = create_catalog(&ctx).context("Extracting messages")?; + polib::po_file::write(&catalog, &output_path) + .with_context(|| format!("Writing messages to {}", output_path.display()))?; + + Ok(()) +} diff --git a/i18n-helpers/src/lib.rs b/i18n-helpers/src/lib.rs new file mode 100644 index 000000000000..d16ef1fc33ec --- /dev/null +++ b/i18n-helpers/src/lib.rs @@ -0,0 +1,106 @@ +// Copyright 2023 Google LLC +// +// Licensed under the Apache License, Version 2.0 (the "License"); +// you may not use this file except in compliance with the License. +// You may obtain a copy of the License at +// +// http://www.apache.org/licenses/LICENSE-2.0 +// +// Unless required by applicable law or agreed to in writing, software +// distributed under the License is distributed on an "AS IS" BASIS, +// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +// See the License for the specific language governing permissions and +// limitations under the License. + +use once_cell::sync::Lazy; +use regex::Regex; + +static PARAGRAPH_SEPARATOR: Lazy = Lazy::new(|| Regex::new(r"\n\n+").unwrap()); + +/// Extract paragraphs from text. +/// +/// Paragraphs are separated by at least two newlines. Returns an +/// iterator over line numbers (starting from 1) and paragraphs. +pub fn extract_paragraphs(text: &str) -> impl Iterator { + // TODO: This could be made more sophisticated by parsing the + // Markdown and stripping off the markup characters. + // + // As an example, a header like "## My heading" could become just + // "My heading" in the `.pot` file. Similarly, paragraphs could be + // unfolded and list items could be translated one-by-one. + + // Skip over leading empty lines. + let trimmed = text.trim_start_matches('\n'); + let mut matches = PARAGRAPH_SEPARATOR.find_iter(trimmed); + let mut lineno = 1 + text.len() - trimmed.len(); + let mut last = 0; + + std::iter::from_fn(move || match matches.next() { + Some(m) => { + let result = (lineno, &trimmed[last..m.start()]); + lineno += trimmed[last..m.end()].lines().count(); + last = m.end(); + Some(result) + } + None => { + if last < trimmed.len() { + let result = (lineno, trimmed[last..].trim_end_matches('\n')); + last = trimmed.len(); + Some(result) + } else { + None + } + } + }) +} + +#[cfg(test)] +mod tests { + use super::*; + + macro_rules! assert_iter_eq { + ($left_iter:expr, $right:expr) => { + assert_eq!($left_iter.collect::>(), $right) + }; + } + + #[test] + fn test_extract_paragraphs_empty() { + assert_iter_eq!(extract_paragraphs(""), vec![]); + } + + #[test] + fn test_extract_paragraphs_single_line() { + assert_iter_eq!( + extract_paragraphs("This is a paragraph."), + vec![(1, "This is a paragraph.")] + ); + } + + #[test] + fn test_extract_paragraphs_simple() { + assert_iter_eq!( + extract_paragraphs("This is\nthe first\nparagraph.\n\nSecond paragraph."), + vec![ + (1, "This is\nthe first\nparagraph."), + (5, "Second paragraph.") + ] + ); + } + + #[test] + fn test_extract_paragraphs_leading_newlines() { + assert_iter_eq!( + extract_paragraphs("\n\n\nThis is\na paragraph."), + vec![(4, "This is\na paragraph.")] + ); + } + + #[test] + fn test_extract_paragraphs_trailing_newlines() { + assert_iter_eq!( + extract_paragraphs("This is\na paragraph.\n\n\n"), + vec![(1, "This is\na paragraph.")] + ); + } +} diff --git a/src/SUMMARY.md b/src/SUMMARY.md index e1d9080111dd..6f186b9a2fa6 100644 --- a/src/SUMMARY.md +++ b/src/SUMMARY.md @@ -28,19 +28,19 @@ - [References](basic-syntax/references.md) - [Dangling References](basic-syntax/references-dangling.md) - [Slices](basic-syntax/slices.md) - - [`String` vs `str`](basic-syntax/string-slices.md) + - [String vs str](basic-syntax/string-slices.md) - [Functions](basic-syntax/functions.md) - [Methods](basic-syntax/methods.md) - [Overloading](basic-syntax/functions-interlude.md) - [Exercises](exercises/day-1/morning.md) - [Implicit Conversions](exercises/day-1/implicit-conversions.md) - - [Arrays and `for` Loops](exercises/day-1/for-loops.md) + - [Arrays and for Loops](exercises/day-1/for-loops.md) # Day 1: Afternoon - [Variables](basic-syntax/variables.md) - [Type Inference](basic-syntax/type-inference.md) - - [`static` & `const`](basic-syntax/static-and-const.md)) + - [static & const](basic-syntax/static-and-const.md)) - [Scopes and Shadowing](basic-syntax/scopes-shadowing.md) - [Memory Management](memory-management.md) - [Stack vs Heap](memory-management/stack-vs-heap.md) @@ -93,23 +93,23 @@ - [Control Flow](control-flow.md) - [Blocks](control-flow/blocks.md) - - [`if` expressions](control-flow/if-expressions.md) - - [`if let` expressions](control-flow/if-let-expressions.md) - - [`while` expressions](control-flow/while-expressions.md) - - [`while let` expressions](control-flow/while-let-expressions.md) - - [`for` expressions](control-flow/for-expressions.md) - - [`loop` expressions](control-flow/loop-expressions.md) - - [`match` expressions](control-flow/match-expressions.md) - - [`break` & `continue`](control-flow/break-continue.md) + - [if expressions](control-flow/if-expressions.md) + - [if let expressions](control-flow/if-let-expressions.md) + - [while expressions](control-flow/while-expressions.md) + - [while let expressions](control-flow/while-let-expressions.md) + - [for expressions](control-flow/for-expressions.md) + - [loop expressions](control-flow/loop-expressions.md) + - [match expressions](control-flow/match-expressions.md) + - [break & continue](control-flow/break-continue.md) - [Standard Library](std.md) - - [`String`](std/string.md) - - [`Option` and `Result`](std/option-result.md) - - [`Vec`](std/vec.md) - - [`HashMap`](std/hashmap.md) - - [`Box`](std/box.md) - - [`Recursive Data Types`](std/box-recursive.md) - - [`Niche Optimization`](std/box-niche.md) - - [`Rc`](std/rc.md) + - [String](std/string.md) + - [Option and Result](std/option-result.md) + - [Vec](std/vec.md) + - [HashMap](std/hashmap.md) + - [Box](std/box.md) + - [Recursive Data Types](std/box-recursive.md) + - [Niche Optimization](std/box-niche.md) + - [Rc](std/rc.md) - [Modules](modules.md) - [Visibility](modules/visibility.md) - [Paths](modules/paths.md) @@ -128,17 +128,16 @@ - [Deriving Traits](traits/deriving-traits.md) - [Default Methods](traits/default-methods.md) - [Important Traits](traits/important-traits.md) - - [`Iterator`](traits/iterator.md) - - [`FromIterator`](traits/from-iterator.md) - - [`From` and `Into`](traits/from-into.md) - - [`Read` and `Write`](traits/read-write.md) - - [`Add`, `Mul`, ...](traits/operators.md) - - [`Drop`](traits/drop.md) + - [Iterator](traits/iterator.md) + - [FromIterator](traits/from-iterator.md) + - [Read and Write](traits/read-write.md) + - [Add, Mul, ...](traits/operators.md) + - [Drop](traits/drop.md) - [Generics](generics.md) - [Generic Data Types](generics/data-types.md) - [Generic Methods](generics/methods.md) - [Trait Bounds](generics/trait-bounds.md) - - [`impl Trait`](generics/impl-trait.md) + - [impl Trait](generics/impl-trait.md) - [Closures](generics/closures.md) - [Monomorphization](generics/monomorphization.md) - [Trait Objects](generics/trait-objects.md) @@ -151,7 +150,7 @@ - [Panics](error-handling/panics.md) - [Catching Stack Unwinding](error-handling/panic-unwind.md) - [Structured Error Handling](error-handling/result.md) - - [Propagating Errors with `?`](error-handling/try-operator.md) + - [Propagating Errors with ?](error-handling/try-operator.md) - [Converting Error Types](error-handling/converting-error-types.md) - [Deriving Error Enums](error-handling/deriving-error-enums.md) - [Adding Context to Errors](error-handling/error-contexts.md) @@ -182,12 +181,12 @@ - [Unbounded Channels](concurrency/channels/unbounded.md) - [Bounded Channels](concurrency/channels/bounded.md) - [Shared State](concurrency/shared_state.md) - - [`Arc`](concurrency/shared_state/arc.md) - - [`Mutex`](concurrency/shared_state/mutex.md) + - [Arc](concurrency/shared_state/arc.md) + - [Mutex](concurrency/shared_state/mutex.md) - [Example](concurrency/shared_state/example.md) - - [`Send` and `Sync`](concurrency/send-sync.md) - - [`Send`](concurrency/send-sync/send.md) - - [`Sync`](concurrency/send-sync/sync.md) + - [Send and Sync](concurrency/send-sync.md) + - [Send](concurrency/send-sync/send.md) + - [Sync](concurrency/send-sync/sync.md) - [Examples](concurrency/send-sync/examples.md) - [Exercises](exercises/day-4/morning.md) - [Dining Philosophers](exercises/day-4/dining-philosophers.md)