Tracking Issue for minicore (core prelude stubs test auxiliary) #131485

jieyouxu · 2024-10-10T06:24:13Z

Context

We have a bunch of #![no_core] ui/codegen/assembly tests that roll their own core prelude stubs because they need to build (but not run) on a cross-compiled target. But this means:

A bunch of duplicated implementations of core prelude stubs scattered in different tests.
A heavy burden on a contributor of possibly having to update many tests if a new required lang item wants to be added.
Prelude stubs can obfuscate the intent of the test (more things that the reader has to read and mentally filter out).

Instead of continuing to add more copies of core prelude stubs, let's centralize such core prelude stubs into a shared test auxiliary called minicore. minicore is for core prelude stubs specifically. std prelude which can include liballoc items and such is beyond the scope of minicore, because core is usable by more tests. This does not preclude adding another ministd or minialloc for stubs of std/liballoc respectively.

See #130375 for more discussions.

Steps

1. Open an MCP to add minicore test auxiliary.
- MCP is accepted: Support tests to use minicore std/core prelude stubs compiler-team#786
- Note that the MCP title specified std/core prelude stubs, but we revised the scope to core stubs only.
2. Implement initial test infra support for minicore.
- Add minicore test auxiliary and support //@ add-core-stubs directive in ui/assembly/codegen tests #130693
3. Adjust rustc-dev-guide docs.
- Describe minicore test auxiliary and directive rustc-dev-guide#2097
4. ~~Figure out the logistics on how to absorb core stubs into minicore, then use minicore to simplify existing #![no_core] tests.~~
- Currently, I think it's better to just have the infra in place to allow newer and changed tests to use minicore.rs, it seems like unnecessary churn to also force existing tests to use minicore.rs atm.

Unresolved questions

What do we do with existing #![no_core] tests that could benefit from minicore?

The text was updated successfully, but these errors were encountered:

scottmcm · 2024-10-10T15:58:56Z

Great news! I've been doing some MIR work that uses lang items, and ended up just making it work a bit worse instead of dealing with all the no-core tests. If there was a centralized place to deal with problems like this, that'd be great 🎉

Add `minicore` test auxiliary and support `//@ add-core-stubs` directive in ui/assembly/codegen tests Context: [Real cross-compiling tests instead of `#![no_core]` silliness rust-lang#130375](rust-lang#130375) MCP: rust-lang/compiler-team#786 Tracking issue: rust-lang#131485 This prototype PR is subject to further changes based on feedback. ### New `minicore` test auxiliary and `//@ add-core-stubs` compiletest directive This PR introduces a prototype implementation of a `minicore` auxiliary test helper that provides `core` stubs for `#![no_core]` ui/assembly/codegen tests that need to build but not run on both the host platform and the cross-compiled target platform. Key summary: - `tests/auxiliary/minicore.rs` contains stub definitions of `core` items intended for consumption by `check-pass`/`build-pass` tests that want the typical prelude items like `Copy` to be stubbed out under `#![no_core]` scenarios, so that the test can be built (not run) for cross-compiled target platforms. Such tests don't want nor need full `-Z build-std` (e.g. `tests/ui/abi/compatibility.rs`). - `minicore` is intended for `core` items **only**, not `std`- or `alloc`-exclusive items. If stubs for `alloc` or `std` are wanted, they should be provided by an additional directive and test auxiliary, and not be conflated with `minicore` or `core` stubs. This is because a wider range of tests can benefit from `core`-only stubs. ### Implementation - The `minicore` auxiliary is a single source file `tests/auxiliary/minicore.rs`. - The path to `minicore` is made avaiable from bootstrap to compiletest via the `--minicore-path` compiletest flag. - `minicore` is then built on-demand via the `//@ add-core-stubs` compiletest directive, for each test revision for the given target platform (this distinction is important for when host platform != target platform in cross-compilation scenario). - `minicore` is then made available to the test as an [extern prelude]. [extern prelude]: https://doc.rust-lang.org/reference/names/preludes.html#extern-prelude ### Example usage ```rs // tests/ui/abi/my-abi-test.rs //@ check-pass //@ add-core-stubs //@ compile-flags: --target i686-unknown-linux-gnu //@ needs-llvm-components: x86 #![feature(no_core, lang_items)] #![no_std] #![no_core] #![allow(unused, internal_features)] extern crate minicore; use minicore::*; #[lang = "clone"] pub trait Clone: Sized { // `Sized` is provided by `minicore` fn clone(&self) -> Self; } ``` ### Implementation steps - [x] 1. Add an initial `minicore` test auxiliary. - [x] 2. Build `minicore` in bootstrap. - [x] 3. Setup a `--minicore-path` compiletest cli flag and pass `minicore` build artifact path from bootstrap to compiletest. - [x] 4. Assert `add-core-stubs` is mutually incompatible with tests that require to be `run`, as the stubs are only good for tests that only need to be built (i.e. no `run-{pass,fail}`). - [x] 5. Add some self-tests to sanity check the behavior. - [x] 6. Ensure that `tests/auxiliary/minicore.rs` is input stamped, i.e. modifying `tests/auxiliary/minicore.rs` should invalidate test cache and force the test to be rerun. ### Known limitations - The current `minicore` is very minimal, because this PR is intended to focus on supporting the test infrastructure first. Further stubs could be added in follow-up PRs and/or on a as-needed basis. try-job: aarch64-apple try-job: armhf-gnu try-job: x86_64-msvc try-job: test-various try-job: dist-various-1

Rollup merge of rust-lang#130693 - jieyouxu:minicore, r=bjorn3 Add `minicore` test auxiliary and support `//@ add-core-stubs` directive in ui/assembly/codegen tests Context: [Real cross-compiling tests instead of `#![no_core]` silliness rust-lang#130375](rust-lang#130375) MCP: rust-lang/compiler-team#786 Tracking issue: rust-lang#131485 This prototype PR is subject to further changes based on feedback. ### New `minicore` test auxiliary and `//@ add-core-stubs` compiletest directive This PR introduces a prototype implementation of a `minicore` auxiliary test helper that provides `core` stubs for `#![no_core]` ui/assembly/codegen tests that need to build but not run on both the host platform and the cross-compiled target platform. Key summary: - `tests/auxiliary/minicore.rs` contains stub definitions of `core` items intended for consumption by `check-pass`/`build-pass` tests that want the typical prelude items like `Copy` to be stubbed out under `#![no_core]` scenarios, so that the test can be built (not run) for cross-compiled target platforms. Such tests don't want nor need full `-Z build-std` (e.g. `tests/ui/abi/compatibility.rs`). - `minicore` is intended for `core` items **only**, not `std`- or `alloc`-exclusive items. If stubs for `alloc` or `std` are wanted, they should be provided by an additional directive and test auxiliary, and not be conflated with `minicore` or `core` stubs. This is because a wider range of tests can benefit from `core`-only stubs. ### Implementation - The `minicore` auxiliary is a single source file `tests/auxiliary/minicore.rs`. - The path to `minicore` is made avaiable from bootstrap to compiletest via the `--minicore-path` compiletest flag. - `minicore` is then built on-demand via the `//@ add-core-stubs` compiletest directive, for each test revision for the given target platform (this distinction is important for when host platform != target platform in cross-compilation scenario). - `minicore` is then made available to the test as an [extern prelude]. [extern prelude]: https://doc.rust-lang.org/reference/names/preludes.html#extern-prelude ### Example usage ```rs // tests/ui/abi/my-abi-test.rs //@ check-pass //@ add-core-stubs //@ compile-flags: --target i686-unknown-linux-gnu //@ needs-llvm-components: x86 #![feature(no_core, lang_items)] #![no_std] #![no_core] #![allow(unused, internal_features)] extern crate minicore; use minicore::*; #[lang = "clone"] pub trait Clone: Sized { // `Sized` is provided by `minicore` fn clone(&self) -> Self; } ``` ### Implementation steps - [x] 1. Add an initial `minicore` test auxiliary. - [x] 2. Build `minicore` in bootstrap. - [x] 3. Setup a `--minicore-path` compiletest cli flag and pass `minicore` build artifact path from bootstrap to compiletest. - [x] 4. Assert `add-core-stubs` is mutually incompatible with tests that require to be `run`, as the stubs are only good for tests that only need to be built (i.e. no `run-{pass,fail}`). - [x] 5. Add some self-tests to sanity check the behavior. - [x] 6. Ensure that `tests/auxiliary/minicore.rs` is input stamped, i.e. modifying `tests/auxiliary/minicore.rs` should invalidate test cache and force the test to be rerun. ### Known limitations - The current `minicore` is very minimal, because this PR is intended to focus on supporting the test infrastructure first. Further stubs could be added in follow-up PRs and/or on a as-needed basis. try-job: aarch64-apple try-job: armhf-gnu try-job: x86_64-msvc try-job: test-various try-job: dist-various-1

jieyouxu · 2024-10-31T15:56:08Z

Initial test infra PR #130693 has now merged.

RalfJung · 2025-02-16T16:57:52Z

Thanks a lot for adding this! I only now finally got around to play with it, and it seems to work great. :)

I admit I am a bit confused by the naming -- it's called "core-stubs" but then the import is called "minicore". Are these two
different things, or are we using two different names to refer to the same thing?

jieyouxu · 2025-02-16T17:07:54Z

Thanks a lot for adding this! I only now finally got around to play with it, and it seems to work great. :)

I admit I am a bit confused by the naming -- it's called "core-stubs" but then the import is called "minicore". Are these two different things, or are we using two different names to refer to the same thing?

So... I am not very good with naming things, what happened was:

Initially, the test auxiliary file was called minicore.rs, and the directive was called //@ use-minicore.
This was discussed during a compiler triage meeting, was suggested to rename it //@ add-core-stubs for ppl who don't know what a "minicore" is. I don't really have a strong opinion on the exact directive name, so I went with that lol.

There really isn't any strong reasoning here 😆

The same thing happened with the new run-make infra... It's still called run-make and rmake.rs because nobody suggested a better name / had a concern with the rmake.rs / test suite name when I implemented the new run-make infra lol.

jieyouxu · 2025-02-16T17:09:03Z

Also I should close this specific tracking issue as it has served its purpose, specific bugs/enchancements should probably be in their separate issues.

jieyouxu · 2025-02-16T17:16:32Z

For easier search purposes, there is a drawback to the current implementation in that it's possible for actual library/core definitions to diverge from minicore. This is tracked in #134652, where ideally we'd able to use host "real" library/core when available, and only the "minicore" stubs on cross. Even better, is if somehow

maybe minicore could be a file or module within library/core that can be built as a standalone crate. Then compiletest would use this directly and the real core just reexports things in the right place, so we'll always be using the same API regardless of whether or not all of core is available.

as mentioned in #134652 (comment).

RalfJung · 2025-02-16T17:29:30Z

Is there an issue tracking "port all the existing tests to minicore" or is that not planned?

This was discussed during a compiler triage meeting, was suggested to rename it //@ add-core-stubs for ppl who don't know what a "minicore" is. I don't really have a strong opinion on the exact directive name, so I went with that lol.

Hm, people have to read the docs anyway I think. "core-stubs" doesn't seem more clear and the minicore terminology still exists. But 🤷

jieyouxu · 2025-02-16T17:31:03Z

Is there an issue tracking "port all the existing tests to minicore" or is that not planned?

I considered this. However, I felt like

Currently, I think it's better to just have the infra in place to allow newer and changed tests to use minicore.rs, it seems like unnecessary churn to also force existing tests to use minicore.rs atm.

RalfJung · 2025-02-16T17:32:30Z

Yeah fair.

use add-core-stubs / minicore for a few more tests See rust-lang#131485 for context. These are some tests I worked on in the past so I figured I'd see if `minicore` works for them. :)

Rollup merge of rust-lang#137145 - RalfJung:minicore, r=jieyouxu use add-core-stubs / minicore for a few more tests See rust-lang#131485 for context. These are some tests I worked on in the past so I figured I'd see if `minicore` works for them. :)

use add-core-stubs / minicore for a few more tests See rust-lang/rust#131485 for context. These are some tests I worked on in the past so I figured I'd see if `minicore` works for them. :)

This was referenced Oct 11, 2024

Add minicore test auxiliary and support //@ add-core-stubs directive in ui/assembly/codegen tests #130693

Merged

Describe minicore test auxiliary and directive rust-lang/rustc-dev-guide#2097

Merged

RalfJung mentioned this issue Feb 16, 2025

use add-core-stubs / minicore for a few more tests #137145

Merged

jieyouxu closed this as completed Feb 16, 2025

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Tracking Issue for minicore (core prelude stubs test auxiliary) #131485

Tracking Issue for minicore (core prelude stubs test auxiliary) #131485

jieyouxu commented Oct 10, 2024 •

edited

Loading

scottmcm commented Oct 10, 2024

jieyouxu commented Oct 31, 2024 •

edited

Loading

RalfJung commented Feb 16, 2025

jieyouxu commented Feb 16, 2025 •

edited

Loading

jieyouxu commented Feb 16, 2025

jieyouxu commented Feb 16, 2025 •

edited

Loading

RalfJung commented Feb 16, 2025

jieyouxu commented Feb 16, 2025

RalfJung commented Feb 16, 2025

Tracking Issue for minicore (core prelude stubs test auxiliary) #131485

Tracking Issue for minicore (core prelude stubs test auxiliary) #131485

Comments

jieyouxu commented Oct 10, 2024 • edited Loading

Context

Steps

Unresolved questions

scottmcm commented Oct 10, 2024

jieyouxu commented Oct 31, 2024 • edited Loading

RalfJung commented Feb 16, 2025

jieyouxu commented Feb 16, 2025 • edited Loading

jieyouxu commented Feb 16, 2025

jieyouxu commented Feb 16, 2025 • edited Loading

RalfJung commented Feb 16, 2025

jieyouxu commented Feb 16, 2025

RalfJung commented Feb 16, 2025

jieyouxu commented Oct 10, 2024 •

edited

Loading

jieyouxu commented Oct 31, 2024 •

edited

Loading

jieyouxu commented Feb 16, 2025 •

edited

Loading

jieyouxu commented Feb 16, 2025 •

edited

Loading