Make parallel frontend CI job compile tests in parallel#157705
Make parallel frontend CI job compile tests in parallel#157705zetanumbers wants to merge 2 commits into
Conversation
|
r? @jieyouxu rustbot has assigned @jieyouxu. Use Why was this reviewer chosen?The reviewer was selected based on:
|
|
@bors try jobs=optional-x86_64-gnu-parallel-frontend |
This comment has been minimized.
This comment has been minimized.
Make parallel frontend CI job compile tests in parallel try-job: optional-x86_64-gnu-parallel-frontend
|
💔 Test for 93be885 failed: CI. Failed job:
A workflow was considered to be a failure because it took only |
This comment has been minimized.
This comment has been minimized.
This comment was marked as off-topic.
This comment was marked as off-topic.
This comment has been minimized.
This comment has been minimized.
Make parallel frontend CI job compile tests in parallel
This comment was marked as off-topic.
This comment was marked as off-topic.
|
@bors try |
This comment has been minimized.
This comment has been minimized.
Make parallel frontend CI job compile tests in parallel try-job: optional-x86_64-gnu-parallel-frontend
|
💔 Test for 1810706 failed: CI. Failed job:
A workflow was considered to be a failure because it took only |
|
The job Click to see the possible cause of the failure (guessed by this bot) |
|
@zetanumbers is on vacation until July 15, but we want these changes sooner, so #158307 resubmits and tries to fix them. |
…eyouxu CI job for parallel frontend ui tests ## Summary Part of rust-lang/compiler-team#1005. Supersedes rust-lang#157705. ### Initial setup in this PR For the initial setup in this PR, we'll go ahead with the following combination: - `RUST_TEST_THREADS`: `max(1, $(nproc) // ${PARALLEL_FRONTEND_THREADS})` - `--parallel-frontend-threads`: **4** - `--iteration-count`: **2** Against `./x test tests/ui --stage=2` only. We can tune these knobs in follow-ups. In try jobs we ran, this should not exceed the current longest auto job duration (at around 3h 15m). ## Additional context ### Issues with the previous attempt The issue with the original change was arguments `--parallel-frontend-threads=4 --iteration-count=2` being compiletest-only. They were being passed to other parts of the test harness (e.g. libtest) as is. These parts, however, have no clue of the arguments, hence the errors. I've spent a lot of time on this and haven't found any reasonable way to fix this behavior. We could make bootstrap aware of these specific two arguments and have an additional internal logic for handling this. It's big of a hack, i reckon. The best decision i arrived at is to split testing into two parts: one for compiletest only and another for everything else. The issue is (AFAIK) we can't tell bootstrap (or x.py, at least) to "test the default stuff, but only for compiletest". When used like `x test tests/` it runs _all_ the tests in this directory, including non-default ones, and crashes as it can't find nodejs for doctests. `--skip compiler/ --skip library/ --skip src/tools/ --skip tests/incremental ...` is still not exhaustive list of exclusions. I went on with a whitelist instead of a blacklist. But we, again, can't tell what tests are "default". There's a mechanism in [bootstrap::core::builder::Builder](https://doc.rust-lang.org/nightly/nightly-rustc/bootstrap/core/builder/struct.Builder.html#structfield.log_cli_step_for_tests) for showing "dry-run" test suites, but it's not available from the cli. For now, my whitelist is quite small and i have no idea what should it be like. r? @petrochenkov --- try-job: optional-x86_64-gnu-parallel-frontend
…eyouxu CI job for parallel frontend ui tests ## Summary Part of rust-lang/compiler-team#1005. Supersedes rust-lang#157705. ### Initial setup in this PR For the initial setup in this PR, we'll go ahead with the following combination: - `RUST_TEST_THREADS`: `max(1, $(nproc) // ${PARALLEL_FRONTEND_THREADS})` - `--parallel-frontend-threads`: **4** - `--iteration-count`: **2** Against `./x test tests/ui --stage=2` only. We can tune these knobs in follow-ups. In try jobs we ran, this should not exceed the current longest auto job duration (at around 3h 15m). ## Additional context ### Issues with the previous attempt The issue with the original change was arguments `--parallel-frontend-threads=4 --iteration-count=2` being compiletest-only. They were being passed to other parts of the test harness (e.g. libtest) as is. These parts, however, have no clue of the arguments, hence the errors. I've spent a lot of time on this and haven't found any reasonable way to fix this behavior. We could make bootstrap aware of these specific two arguments and have an additional internal logic for handling this. It's big of a hack, i reckon. The best decision i arrived at is to split testing into two parts: one for compiletest only and another for everything else. The issue is (AFAIK) we can't tell bootstrap (or x.py, at least) to "test the default stuff, but only for compiletest". When used like `x test tests/` it runs _all_ the tests in this directory, including non-default ones, and crashes as it can't find nodejs for doctests. `--skip compiler/ --skip library/ --skip src/tools/ --skip tests/incremental ...` is still not exhaustive list of exclusions. I went on with a whitelist instead of a blacklist. But we, again, can't tell what tests are "default". There's a mechanism in [bootstrap::core::builder::Builder](https://doc.rust-lang.org/nightly/nightly-rustc/bootstrap/core/builder/struct.Builder.html#structfield.log_cli_step_for_tests) for showing "dry-run" test suites, but it's not available from the cli. For now, my whitelist is quite small and i have no idea what should it be like. r? @petrochenkov --- try-job: optional-x86_64-gnu-parallel-frontend
…eyouxu CI job for parallel frontend ui tests ## Summary Part of rust-lang/compiler-team#1005. Supersedes rust-lang#157705. ### Initial setup in this PR For the initial setup in this PR, we'll go ahead with the following combination: - `RUST_TEST_THREADS`: `max(1, $(nproc) // ${PARALLEL_FRONTEND_THREADS})` - `--parallel-frontend-threads`: **4** - `--iteration-count`: **2** Against `./x test tests/ui --stage=2` only. We can tune these knobs in follow-ups. In try jobs we ran, this should not exceed the current longest auto job duration (at around 3h 15m). ## Additional context ### Issues with the previous attempt The issue with the original change was arguments `--parallel-frontend-threads=4 --iteration-count=2` being compiletest-only. They were being passed to other parts of the test harness (e.g. libtest) as is. These parts, however, have no clue of the arguments, hence the errors. I've spent a lot of time on this and haven't found any reasonable way to fix this behavior. We could make bootstrap aware of these specific two arguments and have an additional internal logic for handling this. It's big of a hack, i reckon. The best decision i arrived at is to split testing into two parts: one for compiletest only and another for everything else. The issue is (AFAIK) we can't tell bootstrap (or x.py, at least) to "test the default stuff, but only for compiletest". When used like `x test tests/` it runs _all_ the tests in this directory, including non-default ones, and crashes as it can't find nodejs for doctests. `--skip compiler/ --skip library/ --skip src/tools/ --skip tests/incremental ...` is still not exhaustive list of exclusions. I went on with a whitelist instead of a blacklist. But we, again, can't tell what tests are "default". There's a mechanism in [bootstrap::core::builder::Builder](https://doc.rust-lang.org/nightly/nightly-rustc/bootstrap/core/builder/struct.Builder.html#structfield.log_cli_step_for_tests) for showing "dry-run" test suites, but it's not available from the cli. For now, my whitelist is quite small and i have no idea what should it be like. r? @petrochenkov --- try-job: optional-x86_64-gnu-parallel-frontend
…eyouxu CI job for parallel frontend ui tests ## Summary Part of rust-lang/compiler-team#1005. Supersedes rust-lang#157705. ### Initial setup in this PR For the initial setup in this PR, we'll go ahead with the following combination: - `RUST_TEST_THREADS`: `max(1, $(nproc) // ${PARALLEL_FRONTEND_THREADS})` - `--parallel-frontend-threads`: **4** - `--iteration-count`: **2** Against `./x test tests/ui --stage=2` only. We can tune these knobs in follow-ups. In try jobs we ran, this should not exceed the current longest auto job duration (at around 3h 15m). ## Additional context ### Issues with the previous attempt The issue with the original change was arguments `--parallel-frontend-threads=4 --iteration-count=2` being compiletest-only. They were being passed to other parts of the test harness (e.g. libtest) as is. These parts, however, have no clue of the arguments, hence the errors. I've spent a lot of time on this and haven't found any reasonable way to fix this behavior. We could make bootstrap aware of these specific two arguments and have an additional internal logic for handling this. It's big of a hack, i reckon. The best decision i arrived at is to split testing into two parts: one for compiletest only and another for everything else. The issue is (AFAIK) we can't tell bootstrap (or x.py, at least) to "test the default stuff, but only for compiletest". When used like `x test tests/` it runs _all_ the tests in this directory, including non-default ones, and crashes as it can't find nodejs for doctests. `--skip compiler/ --skip library/ --skip src/tools/ --skip tests/incremental ...` is still not exhaustive list of exclusions. I went on with a whitelist instead of a blacklist. But we, again, can't tell what tests are "default". There's a mechanism in [bootstrap::core::builder::Builder](https://doc.rust-lang.org/nightly/nightly-rustc/bootstrap/core/builder/struct.Builder.html#structfield.log_cli_step_for_tests) for showing "dry-run" test suites, but it's not available from the cli. For now, my whitelist is quite small and i have no idea what should it be like. r? @petrochenkov --- try-job: optional-x86_64-gnu-parallel-frontend
…eyouxu CI job for parallel frontend ui tests ## Summary Part of rust-lang/compiler-team#1005. Supersedes rust-lang#157705. ### Initial setup in this PR For the initial setup in this PR, we'll go ahead with the following combination: - `RUST_TEST_THREADS`: `max(1, $(nproc) // ${PARALLEL_FRONTEND_THREADS})` - `--parallel-frontend-threads`: **4** - `--iteration-count`: **2** Against `./x test tests/ui --stage=2` only. We can tune these knobs in follow-ups. In try jobs we ran, this should not exceed the current longest auto job duration (at around 3h 15m). ## Additional context ### Issues with the previous attempt The issue with the original change was arguments `--parallel-frontend-threads=4 --iteration-count=2` being compiletest-only. They were being passed to other parts of the test harness (e.g. libtest) as is. These parts, however, have no clue of the arguments, hence the errors. I've spent a lot of time on this and haven't found any reasonable way to fix this behavior. We could make bootstrap aware of these specific two arguments and have an additional internal logic for handling this. It's big of a hack, i reckon. The best decision i arrived at is to split testing into two parts: one for compiletest only and another for everything else. The issue is (AFAIK) we can't tell bootstrap (or x.py, at least) to "test the default stuff, but only for compiletest". When used like `x test tests/` it runs _all_ the tests in this directory, including non-default ones, and crashes as it can't find nodejs for doctests. `--skip compiler/ --skip library/ --skip src/tools/ --skip tests/incremental ...` is still not exhaustive list of exclusions. I went on with a whitelist instead of a blacklist. But we, again, can't tell what tests are "default". There's a mechanism in [bootstrap::core::builder::Builder](https://doc.rust-lang.org/nightly/nightly-rustc/bootstrap/core/builder/struct.Builder.html#structfield.log_cli_step_for_tests) for showing "dry-run" test suites, but it's not available from the cli. For now, my whitelist is quite small and i have no idea what should it be like. r? @petrochenkov --- try-job: optional-x86_64-gnu-parallel-frontend
…eyouxu CI job for parallel frontend ui tests ## Summary Part of rust-lang/compiler-team#1005. Supersedes rust-lang#157705. ### Initial setup in this PR For the initial setup in this PR, we'll go ahead with the following combination: - `RUST_TEST_THREADS`: `max(1, $(nproc) // ${PARALLEL_FRONTEND_THREADS})` - `--parallel-frontend-threads`: **4** - `--iteration-count`: **2** Against `./x test tests/ui --stage=2` only. We can tune these knobs in follow-ups. In try jobs we ran, this should not exceed the current longest auto job duration (at around 3h 15m). ## Additional context ### Issues with the previous attempt The issue with the original change was arguments `--parallel-frontend-threads=4 --iteration-count=2` being compiletest-only. They were being passed to other parts of the test harness (e.g. libtest) as is. These parts, however, have no clue of the arguments, hence the errors. I've spent a lot of time on this and haven't found any reasonable way to fix this behavior. We could make bootstrap aware of these specific two arguments and have an additional internal logic for handling this. It's big of a hack, i reckon. The best decision i arrived at is to split testing into two parts: one for compiletest only and another for everything else. The issue is (AFAIK) we can't tell bootstrap (or x.py, at least) to "test the default stuff, but only for compiletest". When used like `x test tests/` it runs _all_ the tests in this directory, including non-default ones, and crashes as it can't find nodejs for doctests. `--skip compiler/ --skip library/ --skip src/tools/ --skip tests/incremental ...` is still not exhaustive list of exclusions. I went on with a whitelist instead of a blacklist. But we, again, can't tell what tests are "default". There's a mechanism in [bootstrap::core::builder::Builder](https://doc.rust-lang.org/nightly/nightly-rustc/bootstrap/core/builder/struct.Builder.html#structfield.log_cli_step_for_tests) for showing "dry-run" test suites, but it's not available from the cli. For now, my whitelist is quite small and i have no idea what should it be like. r? @petrochenkov --- try-job: optional-x86_64-gnu-parallel-frontend
…eyouxu CI job for parallel frontend ui tests ## Summary Part of rust-lang/compiler-team#1005. Supersedes rust-lang#157705. ### Initial setup in this PR For the initial setup in this PR, we'll go ahead with the following combination: - `RUST_TEST_THREADS`: `max(1, $(nproc) // ${PARALLEL_FRONTEND_THREADS})` - `--parallel-frontend-threads`: **4** - `--iteration-count`: **2** Against `./x test tests/ui --stage=2` only. We can tune these knobs in follow-ups. In try jobs we ran, this should not exceed the current longest auto job duration (at around 3h 15m). ## Additional context ### Issues with the previous attempt The issue with the original change was arguments `--parallel-frontend-threads=4 --iteration-count=2` being compiletest-only. They were being passed to other parts of the test harness (e.g. libtest) as is. These parts, however, have no clue of the arguments, hence the errors. I've spent a lot of time on this and haven't found any reasonable way to fix this behavior. We could make bootstrap aware of these specific two arguments and have an additional internal logic for handling this. It's big of a hack, i reckon. The best decision i arrived at is to split testing into two parts: one for compiletest only and another for everything else. The issue is (AFAIK) we can't tell bootstrap (or x.py, at least) to "test the default stuff, but only for compiletest". When used like `x test tests/` it runs _all_ the tests in this directory, including non-default ones, and crashes as it can't find nodejs for doctests. `--skip compiler/ --skip library/ --skip src/tools/ --skip tests/incremental ...` is still not exhaustive list of exclusions. I went on with a whitelist instead of a blacklist. But we, again, can't tell what tests are "default". There's a mechanism in [bootstrap::core::builder::Builder](https://doc.rust-lang.org/nightly/nightly-rustc/bootstrap/core/builder/struct.Builder.html#structfield.log_cli_step_for_tests) for showing "dry-run" test suites, but it's not available from the cli. For now, my whitelist is quite small and i have no idea what should it be like. r? @petrochenkov --- try-job: optional-x86_64-gnu-parallel-frontend
Rollup merge of #158307 - heinwol:parallel-frontend-CI, r=jieyouxu CI job for parallel frontend ui tests ## Summary Part of rust-lang/compiler-team#1005. Supersedes #157705. ### Initial setup in this PR For the initial setup in this PR, we'll go ahead with the following combination: - `RUST_TEST_THREADS`: `max(1, $(nproc) // ${PARALLEL_FRONTEND_THREADS})` - `--parallel-frontend-threads`: **4** - `--iteration-count`: **2** Against `./x test tests/ui --stage=2` only. We can tune these knobs in follow-ups. In try jobs we ran, this should not exceed the current longest auto job duration (at around 3h 15m). ## Additional context ### Issues with the previous attempt The issue with the original change was arguments `--parallel-frontend-threads=4 --iteration-count=2` being compiletest-only. They were being passed to other parts of the test harness (e.g. libtest) as is. These parts, however, have no clue of the arguments, hence the errors. I've spent a lot of time on this and haven't found any reasonable way to fix this behavior. We could make bootstrap aware of these specific two arguments and have an additional internal logic for handling this. It's big of a hack, i reckon. The best decision i arrived at is to split testing into two parts: one for compiletest only and another for everything else. The issue is (AFAIK) we can't tell bootstrap (or x.py, at least) to "test the default stuff, but only for compiletest". When used like `x test tests/` it runs _all_ the tests in this directory, including non-default ones, and crashes as it can't find nodejs for doctests. `--skip compiler/ --skip library/ --skip src/tools/ --skip tests/incremental ...` is still not exhaustive list of exclusions. I went on with a whitelist instead of a blacklist. But we, again, can't tell what tests are "default". There's a mechanism in [bootstrap::core::builder::Builder](https://doc.rust-lang.org/nightly/nightly-rustc/bootstrap/core/builder/struct.Builder.html#structfield.log_cli_step_for_tests) for showing "dry-run" test suites, but it's not available from the cli. For now, my whitelist is quite small and i have no idea what should it be like. r? @petrochenkov --- try-job: optional-x86_64-gnu-parallel-frontend
CI job for parallel frontend ui tests ## Summary Part of rust-lang/compiler-team#1005. Supersedes rust-lang/rust#157705. ### Initial setup in this PR For the initial setup in this PR, we'll go ahead with the following combination: - `RUST_TEST_THREADS`: `max(1, $(nproc) // ${PARALLEL_FRONTEND_THREADS})` - `--parallel-frontend-threads`: **4** - `--iteration-count`: **2** Against `./x test tests/ui --stage=2` only. We can tune these knobs in follow-ups. In try jobs we ran, this should not exceed the current longest auto job duration (at around 3h 15m). ## Additional context ### Issues with the previous attempt The issue with the original change was arguments `--parallel-frontend-threads=4 --iteration-count=2` being compiletest-only. They were being passed to other parts of the test harness (e.g. libtest) as is. These parts, however, have no clue of the arguments, hence the errors. I've spent a lot of time on this and haven't found any reasonable way to fix this behavior. We could make bootstrap aware of these specific two arguments and have an additional internal logic for handling this. It's big of a hack, i reckon. The best decision i arrived at is to split testing into two parts: one for compiletest only and another for everything else. The issue is (AFAIK) we can't tell bootstrap (or x.py, at least) to "test the default stuff, but only for compiletest". When used like `x test tests/` it runs _all_ the tests in this directory, including non-default ones, and crashes as it can't find nodejs for doctests. `--skip compiler/ --skip library/ --skip src/tools/ --skip tests/incremental ...` is still not exhaustive list of exclusions. I went on with a whitelist instead of a blacklist. But we, again, can't tell what tests are "default". There's a mechanism in [bootstrap::core::builder::Builder](https://doc.rust-lang.org/nightly/nightly-rustc/bootstrap/core/builder/struct.Builder.html#structfield.log_cli_step_for_tests) for showing "dry-run" test suites, but it's not available from the cli. For now, my whitelist is quite small and i have no idea what should it be like. r? @petrochenkov --- try-job: optional-x86_64-gnu-parallel-frontend
It is time to have a CI job to catch parallel frontend issues for tests. This PR sets
RUST_TEST_THREADSto 1 to sequentially execute each test to achieve max thread utilization by a single rustc process. Theniteration-countoption is set to 2 to repeat each test since parallel frontend issues usually don't reproduce reliably.try-job: optional-x86_64-gnu-parallel-frontend