Auto merge of #49219 - eddyb:proc-macro-decouple, r=alexcrichton

Decouple proc_macro from the rest of the compiler.

This PR removes all dependencies of `proc_macro` on compiler crates and allows multiple copies of `proc_macro`, built even by different compilers (but from the same source), to interoperate.

Practically, it allows:
* running proc macro tests at stage1 (I moved most from `-fulldeps` to the regular suites)
* using proc macros in the compiler itself (may require some rustbuild trickery)

On the server (i.e. compiler front-end) side:
* `server::*` traits are implemented to provide the concrete types and methods
  * the concrete types are completely separated from the `proc_macro` public API
  * the only use of the type implementing `Server` is to be passed to `Client::run`

On the client (i.e. proc macro) side (potentially using a different `proc_macro` instance!):
* `client::Client` wraps around client-side (expansion) function pointers
  * it encapsulates the `proc_macro` instance used by the client
  * its `run` method can be called by a server, to execute the client-side function
    * the client instance is bridged to the provided server, while it runs
    * ~~currently a thread is spawned, could use process isolation in the future~~
(not the case anymore, see #56058)
* proc macro crates get a generated `static` holding a `&[ProcMacro]`
  * this describes all derives/attr/bang proc macros, replacing the "registrar" function
  * each variant of `ProcMacro` contains an appropriately typed `Client<fn(...) -> ...>`

`proc_macro` public APIs call into the server via an internal "bridge":
* only a currently running proc macro `Client` can interact with those APIs
  * server code might not be able to (if it uses a different `proc_macro` instance)
    * however, it can always create and `run` its own `Client`, but that may be inefficient
* the `bridge` uses serialization, C ABI and integer handles to avoid Rust ABI instability
* each invocation of a proc macro results in disjoint integers in its `proc_macro` handles
  * this prevents using values of those types across invocations (if they even can be kept)

r? @alexcrichton cc @jseyfried @nikomatsakis @Zoxc @thepowersgang

Author: bors

Cargo.lock

@@ -1599,12 +1599,6 @@ dependencies = [
 [[package]]
 name = "proc_macro"
 version = "0.0.0"
-dependencies = [
- "rustc_data_structures 0.0.0",
- "rustc_errors 0.0.0",
- "syntax 0.0.0",
- "syntax_pos 0.0.0",
-]
 
 [[package]]
 name = "profiler_builtins"
@@ -1933,7 +1927,6 @@ dependencies = [
  "log 0.4.5 (registry+https://github.com/rust-lang/crates.io-index)",
  "parking_lot 0.6.4 (registry+https://github.com/rust-lang/crates.io-index)",
  "polonius-engine 0.5.0 (registry+https://github.com/rust-lang/crates.io-index)",
- "proc_macro 0.0.0",
  "rustc-rayon 0.1.1 (registry+https://github.com/rust-lang/crates.io-index)",
  "rustc-rayon-core 0.1.1 (registry+https://github.com/rust-lang/crates.io-index)",
  "rustc_apfloat 0.0.0",
@@ -2351,7 +2344,6 @@ dependencies = [
  "flate2 1.0.3 (registry+https://github.com/rust-lang/crates.io-index)",
  "log 0.4.5 (registry+https://github.com/rust-lang/crates.io-index)",
  "memmap 0.6.2 (registry+https://github.com/rust-lang/crates.io-index)",
- "proc_macro 0.0.0",
  "rustc 0.0.0",
  "rustc_data_structures 0.0.0",
  "rustc_errors 0.0.0",
@@ -2890,7 +2882,6 @@ version = "0.0.0"
 dependencies = [
  "fmt_macros 0.0.0",
  "log 0.4.5 (registry+https://github.com/rust-lang/crates.io-index)",
- "proc_macro 0.0.0",
  "rustc_data_structures 0.0.0",
  "rustc_errors 0.0.0",
  "rustc_target 0.0.0",
@@ -2980,6 +2971,7 @@ name = "test"
 version = "0.0.0"
 dependencies = [
  "getopts 0.2.17 (registry+https://github.com/rust-lang/crates.io-index)",
+ "proc_macro 0.0.0",
  "term 0.0.0",
 ]
 

src/bootstrap/bin/rustc.rs

@@ -129,10 +129,12 @@ fn main() {
         // Help the libc crate compile by assisting it in finding the MUSL
         // native libraries.
         if let Some(s) = env::var_os("MUSL_ROOT") {
-            let mut root = OsString::from("native=");
-            root.push(&s);
-            root.push("/lib");
-            cmd.arg("-L").arg(&root);
+            if target.contains("musl") {
+                let mut root = OsString::from("native=");
+                root.push(&s);
+                root.push("/lib");
+                cmd.arg("-L").arg(&root);
+            }
         }
 
         // Override linker if necessary.