Update docs about ui tests now using //@ headers
#1885
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
tshepang
reviewed
Feb 12, 2024
3 tasks
bors
added a commit
to rust-lang-ci/rust
that referenced
this pull request
Feb 16, 2024
… r=oli-obk Migrate ui tests from legacy compiletest-style directives `//` to `ui_test`-style directives `//@` ## Preface There's an on-going effort to rewrite parts of or the entirety of compiletest (<rust-lang/compiler-team#536>). A step towards this involve migrating ui tests to use the [`ui_test`](https://github.com/oli-obk/ui_test) framework, which involves changing compiletest directives in `// <directive-name>` style to `ui_test` `//@ <directive-name>` style (rust-lang/compiler-team#512). This PR aims to implement the directive-style change from `//` to `//`@`` for ui tests only and make compiletest only accept `//`@`` directives in ui test mode (only). ## Key Changes 1. All ui test `//` directives are replaced by `//`@`` directives. 2. Only accept `//`@`` directives for ui tests (specifically ui tests). 3. Errors if a comment could be interpreted as a legacy-style `//` directive. ## Diff Generation The diff is generated by: - Collecting directives from ui tests via hacking on compiletest. - Using a migration tool to replace `//` directives in ui tests with `//`@`.` ### Reproduction Steps 0. Delete the temporary directory `$RUSTC_REPO_PATH/build/x86_64-apple-darwin/test/ui/__directive_lines` and the temporary file `$RUSTC_REPO_PATH/build/x86_64-apple-darwin/test/ui/__directive_lines.txt` (if you ran the collection script before). 1. Use the <https://github.com/jieyouxu/rust/tree/collect-test-directives> collect-test-directives script, which outputs temporary files recording headers occuring in each ui test. - You need to checkout this branch: `git checkout collect-test-directives`. - You might need to rebase on lastest master and ensure there are no conflicts. - You likely need to run `./x test tests/ui --stage 1 --force-rerun` to generate the temporary files consistently. 2. Checkout the `migrate-ui-test-directives` branch. 4. Run the migration tool <https://github.com/jieyouxu/compiletest-ui_test-header-migration>. - You will need to first generate a `migration_config.toml` via `cargo run -- generate-config` under `$CWD`. - Then, update `manual_directives = ["// should-fail"]` in `migration_config.toml`. This is required because the collection script doesn't deal with some special meta ui tests and there are no other `// should-fail` occurrences. 5. Check that the migration at least does not cause UI test failures if you change compiletest to accept `//`@`` directives for ui tests only. - `RUSTC_TEST_FAIL_FAST=1 ./x test tests/ui --stage 1 --bless` 6. Confirm that there is no difference after running the migration tool when you are on the `migrate-ui-test-directives` branch. ## Next Steps - [x] ~~Need to implement some kind of warning or tidy script to help contributors catch old-style `// <directive-name>` directives, while only accepting `ui_test`-style `//@ <directive-name>` directives.~~ An error is emitted if a comment that could be interpreted as legacy-style test directive is encountered. - [ ] Need to properly document this change in e.g. rustc-dev-guide (rust-lang/rustc-dev-guide#1885). - [x] Add a `README.md` to `tests/ui` describing the directive style change.
bors
added a commit
to rust-lang-ci/rust
that referenced
this pull request
Feb 16, 2024
… r=oli-obk Migrate ui tests from legacy compiletest-style directives `//` to `ui_test`-style directives `//@` ## Preface There's an on-going effort to rewrite parts of or the entirety of compiletest (<rust-lang/compiler-team#536>). A step towards this involve migrating ui tests to use the [`ui_test`](https://github.com/oli-obk/ui_test) framework, which involves changing compiletest directives in `// <directive-name>` style to `ui_test` `//@ <directive-name>` style (rust-lang/compiler-team#512). This PR aims to implement the directive-style change from `//` to `//`@`` for ui tests only and make compiletest only accept `//`@`` directives in the "ui" test suite (only). ## Key Changes 1. All ui test `//` directives are replaced by `//`@`` directives. 2. Only accept `//`@`` directives for "ui" test suite. 3. Errors if a comment could be interpreted as a legacy-style `//` directive. ## Diff Generation The diff is generated by: - Collecting directives from ui tests via hacking on compiletest. - Using a migration tool to replace `//` directives in ui tests with `//`@`.` ### Reproduction Steps 0. Delete the temporary directory `$RUSTC_REPO_PATH/build/x86_64-apple-darwin/test/ui/__directive_lines` and the temporary file `$RUSTC_REPO_PATH/build/x86_64-apple-darwin/test/ui/__directive_lines.txt` (if you ran the collection script before). 1. Use the <https://github.com/jieyouxu/rust/tree/collect-test-directives> collect-test-directives script, which outputs temporary files recording headers occuring in each ui test. - You need to checkout this branch: `git checkout collect-test-directives`. - You might need to rebase on lastest master and ensure there are no conflicts. - You likely need to run `./x test tests/ui --stage 1 --force-rerun` to generate the temporary files consistently. 2. Checkout the `migrate-ui-test-directives` branch. 4. Run the migration tool <https://github.com/jieyouxu/compiletest-ui_test-header-migration>. - You will need to first generate a `migration_config.toml` via `cargo run -- generate-config` under `$CWD`. - Then, update `manual_directives = ["// should-fail"]` in `migration_config.toml`. This is required because the collection script doesn't deal with some special meta ui tests and there are no other `// should-fail` occurrences. 5. Check that the migration at least does not cause UI test failures if you change compiletest to accept `//`@`` directives for ui tests only. - `RUSTC_TEST_FAIL_FAST=1 ./x test tests/ui --stage 1 --bless` 6. Confirm that there is no difference after running the migration tool when you are on the `migrate-ui-test-directives` branch. ## Next Steps - [x] ~~Need to implement some kind of warning or tidy script to help contributors catch old-style `// <directive-name>` directives, while only accepting `ui_test`-style `//@ <directive-name>` directives.~~ An error is emitted if a comment that could be interpreted as legacy-style test directive is encountered. - [ ] Need to properly document this change in e.g. rustc-dev-guide (rust-lang/rustc-dev-guide#1885). - [x] Add a `README.md` to `tests/ui` describing the directive style change.
Co-authored-by: Tshepang Mbambo <[email protected]>
compiler-errors
approved these changes
Feb 16, 2024
jieyouxu
reviewed
Feb 16, 2024
Comment on lines
+11
to
+12
| Some test suites use `//@` to signal that a comment is a header, but most are still | ||
| just using plain comments. |
There was a problem hiding this comment.
There's an additional caveat:
// ignore-tidy and // ignore-tidy-* are specifically ignored by compiletest in "ui" test suite specifically.
matthiaskrgr
added a commit
to matthiaskrgr/rust
that referenced
this pull request
Feb 26, 2024
Update books ## rust-lang/book 1 commits in 71352deb20727b4dda9ebfe8182709d5bf17dfea..19c40bfd2d57641d962f3119a1c343355f1b3c5e 2024-02-19 20:39:35 UTC to 2024-02-19 20:39:35 UTC - Rust upgrades (rust-lang/book#3844) ## rust-lang/edition-guide 3 commits in 76bd48a273a0e0413a3bf22c699112d41497b99e..e1eead1181a691e56299294d5f1d62fe7a26d317 2024-02-24 21:15:28 UTC to 2024-02-17 21:44:36 UTC - Rename static_mut_refs (rust-lang/edition-guide#293) - Add 2024 prelude (rust-lang/edition-guide#292) - Add some more initial stub docs for 2024. (rust-lang/edition-guide#291) ## rust-lang/reference 5 commits in 8227666de13f6e7bb32dea9dc42e841adb5ce4b7..3417f866932cb1c09c6be0f31d2a02ee01b4b95d 2024-02-25 19:37:14 UTC to 2024-02-15 13:28:59 UTC - Document `target_abi` (rust-lang/reference#1446) - Add `rust-toolchain.toml` and revise README (rust-lang/reference#1474) - Clarify semantics of the various pointer to pointer casts (rust-lang/reference#1451) - Update aarch64 target feature docs to match LLVM (rust-lang/reference#1470) - Fix grammar for TypePathFn (rust-lang/reference#1281) ## rust-lang/rust-by-example 4 commits in e188d5d466f7f3ff9f1d518393235f4fe951be46..57f1e708f5d5850562bc385aaf610e6af14d6ec8 2024-02-24 18:31:19 UTC to 2024-02-14 13:06:53 UTC - bug: fix typo (rust-lang/rust-by-example#1822) - Fix typo in attribute.md (rust-lang/rust-by-example#1818) - Add Japanese translation (rust-lang/rust-by-example#1794) - Remove unused .travis.yml file (rust-lang/rust-by-example#1816) ## rust-lang/rustc-dev-guide 9 commits in 1f30cc7..7b0ef5b 2024-02-22 19:15:52 UTC to 2024-02-13 12:13:06 UTC - Update compiletest directives to be in `ui_test` style `//`@`` (rust-lang/rustc-dev-guide#1895) - Add notes on tweak rust-analyzer.check.overrideCommand (rust-lang/rustc-dev-guide#1890) - Fix more links (rust-lang/rustc-dev-guide#1884) - Remove references to -Z ast-json and -Z ast-json-noexpand (rust-lang/rustc-dev-guide#1893) - Specify ui test suite directives and add note on ignore-tidy directives (rust-lang/rustc-dev-guide#1892) - Update docs about ui tests now using `//`@`` headers (rust-lang/rustc-dev-guide#1885) - Make git rebase commands more bulletproof (rust-lang/rustc-dev-guide#1889) - updating-llvm.md: command does not work (rust-lang/rustc-dev-guide#1887) - fix update llvm build command (rust-lang/rustc-dev-guide#1886)
rust-timer
added a commit
to rust-lang-ci/rust
that referenced
this pull request
Feb 27, 2024
Rollup merge of rust-lang#121639 - rustbot:docs-update, r=ehuss Update books ## rust-lang/book 1 commits in 71352deb20727b4dda9ebfe8182709d5bf17dfea..19c40bfd2d57641d962f3119a1c343355f1b3c5e 2024-02-19 20:39:35 UTC to 2024-02-19 20:39:35 UTC - Rust upgrades (rust-lang/book#3844) ## rust-lang/edition-guide 3 commits in 76bd48a273a0e0413a3bf22c699112d41497b99e..e1eead1181a691e56299294d5f1d62fe7a26d317 2024-02-24 21:15:28 UTC to 2024-02-17 21:44:36 UTC - Rename static_mut_refs (rust-lang/edition-guide#293) - Add 2024 prelude (rust-lang/edition-guide#292) - Add some more initial stub docs for 2024. (rust-lang/edition-guide#291) ## rust-lang/reference 5 commits in 8227666de13f6e7bb32dea9dc42e841adb5ce4b7..3417f866932cb1c09c6be0f31d2a02ee01b4b95d 2024-02-25 19:37:14 UTC to 2024-02-15 13:28:59 UTC - Document `target_abi` (rust-lang/reference#1446) - Add `rust-toolchain.toml` and revise README (rust-lang/reference#1474) - Clarify semantics of the various pointer to pointer casts (rust-lang/reference#1451) - Update aarch64 target feature docs to match LLVM (rust-lang/reference#1470) - Fix grammar for TypePathFn (rust-lang/reference#1281) ## rust-lang/rust-by-example 4 commits in e188d5d466f7f3ff9f1d518393235f4fe951be46..57f1e708f5d5850562bc385aaf610e6af14d6ec8 2024-02-24 18:31:19 UTC to 2024-02-14 13:06:53 UTC - bug: fix typo (rust-lang/rust-by-example#1822) - Fix typo in attribute.md (rust-lang/rust-by-example#1818) - Add Japanese translation (rust-lang/rust-by-example#1794) - Remove unused .travis.yml file (rust-lang/rust-by-example#1816) ## rust-lang/rustc-dev-guide 9 commits in 1f30cc7..7b0ef5b 2024-02-22 19:15:52 UTC to 2024-02-13 12:13:06 UTC - Update compiletest directives to be in `ui_test` style `//`@`` (rust-lang/rustc-dev-guide#1895) - Add notes on tweak rust-analyzer.check.overrideCommand (rust-lang/rustc-dev-guide#1890) - Fix more links (rust-lang/rustc-dev-guide#1884) - Remove references to -Z ast-json and -Z ast-json-noexpand (rust-lang/rustc-dev-guide#1893) - Specify ui test suite directives and add note on ignore-tidy directives (rust-lang/rustc-dev-guide#1892) - Update docs about ui tests now using `//`@`` headers (rust-lang/rustc-dev-guide#1885) - Make git rebase commands more bulletproof (rust-lang/rustc-dev-guide#1889) - updating-llvm.md: command does not work (rust-lang/rustc-dev-guide#1887) - fix update llvm build command (rust-lang/rustc-dev-guide#1886)
github-actions bot
pushed a commit
to rust-lang/miri
that referenced
this pull request
Feb 27, 2024
Update books ## rust-lang/book 1 commits in 71352deb20727b4dda9ebfe8182709d5bf17dfea..19c40bfd2d57641d962f3119a1c343355f1b3c5e 2024-02-19 20:39:35 UTC to 2024-02-19 20:39:35 UTC - Rust upgrades (rust-lang/book#3844) ## rust-lang/edition-guide 3 commits in 76bd48a273a0e0413a3bf22c699112d41497b99e..e1eead1181a691e56299294d5f1d62fe7a26d317 2024-02-24 21:15:28 UTC to 2024-02-17 21:44:36 UTC - Rename static_mut_refs (rust-lang/edition-guide#293) - Add 2024 prelude (rust-lang/edition-guide#292) - Add some more initial stub docs for 2024. (rust-lang/edition-guide#291) ## rust-lang/reference 5 commits in 8227666de13f6e7bb32dea9dc42e841adb5ce4b7..3417f866932cb1c09c6be0f31d2a02ee01b4b95d 2024-02-25 19:37:14 UTC to 2024-02-15 13:28:59 UTC - Document `target_abi` (rust-lang/reference#1446) - Add `rust-toolchain.toml` and revise README (rust-lang/reference#1474) - Clarify semantics of the various pointer to pointer casts (rust-lang/reference#1451) - Update aarch64 target feature docs to match LLVM (rust-lang/reference#1470) - Fix grammar for TypePathFn (rust-lang/reference#1281) ## rust-lang/rust-by-example 4 commits in e188d5d466f7f3ff9f1d518393235f4fe951be46..57f1e708f5d5850562bc385aaf610e6af14d6ec8 2024-02-24 18:31:19 UTC to 2024-02-14 13:06:53 UTC - bug: fix typo (rust-lang/rust-by-example#1822) - Fix typo in attribute.md (rust-lang/rust-by-example#1818) - Add Japanese translation (rust-lang/rust-by-example#1794) - Remove unused .travis.yml file (rust-lang/rust-by-example#1816) ## rust-lang/rustc-dev-guide 9 commits in 1f30cc7cca9a3433bc1872abdc98960b36c21ca0..7b0ef5b0bea5e3ce3b9764aa5754a60e2cc05c52 2024-02-22 19:15:52 UTC to 2024-02-13 12:13:06 UTC - Update compiletest directives to be in `ui_test` style `//`@`` (rust-lang/rustc-dev-guide#1895) - Add notes on tweak rust-analyzer.check.overrideCommand (rust-lang/rustc-dev-guide#1890) - Fix more links (rust-lang/rustc-dev-guide#1884) - Remove references to -Z ast-json and -Z ast-json-noexpand (rust-lang/rustc-dev-guide#1893) - Specify ui test suite directives and add note on ignore-tidy directives (rust-lang/rustc-dev-guide#1892) - Update docs about ui tests now using `//`@`` headers (rust-lang/rustc-dev-guide#1885) - Make git rebase commands more bulletproof (rust-lang/rustc-dev-guide#1889) - updating-llvm.md: command does not work (rust-lang/rustc-dev-guide#1887) - fix update llvm build command (rust-lang/rustc-dev-guide#1886)
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
corresponding rustc PR: rust-lang/rust#120881