enable `PathBuf` usage in proc_macro bridge, `tracked::path` #88909

drahnr · 2021-09-13T15:21:11Z

Extracted from #87173

Adds limited PathBuf support for convenience to the proc_macro-bridge code. Currently does not handle invalid utf-t cases gracefully.

@rustbot assign @petrochenkov

rust-highfive · 2021-09-13T15:21:52Z

r? @petrochenkov

(rust-highfive has picked a reviewer for you, use r? to override)

rust-log-analyzer · 2021-09-13T15:57:24Z

The job x86_64-gnu-llvm-10 failed! Check out the build log: (web) (plain)

Click to see the possible cause of the failure (guessed by this bot)

Suite("src/test/run-make") not skipped for "bootstrap::test::RunMake" -- not in ["src/tools/tidy"]
Check compiletest suite=run-make mode=run-make (x86_64-unknown-linux-gnu -> x86_64-unknown-linux-gnu)

running 36 tests
Some tests failed in compiletest suite=run-make mode=run-make host=x86_64-unknown-linux-gnu target=x86_64-unknown-linux-gnu
iii.iii..iiiiiii.iiii..F............

---- [run-make] run-make/track-path-dep-info stdout ----

error: make failed
error: make failed
status: exit status: 2
command: "make"
stdout:
------------------------------------------
# Proc macro
LD_LIBRARY_PATH="/checkout/obj/build/x86_64-unknown-linux-gnu/test/run-make/track-path-dep-info/track-path-dep-info:/checkout/obj/build/x86_64-unknown-linux-gnu/stage2/lib:/checkout/obj/build/x86_64-unknown-linux-gnu/stage0-bootstrap-tools/x86_64-unknown-linux-gnu/release/deps:/checkout/obj/build/x86_64-unknown-linux-gnu/stage0/lib" '/checkout/obj/build/x86_64-unknown-linux-gnu/stage2/bin/rustc'  --out-dir /checkout/obj/build/x86_64-unknown-linux-gnu/test/run-make/track-path-dep-info/track-path-dep-info macro_def.rs
Makefile:10: recipe for target 'all' failed
------------------------------------------
stderr:
------------------------------------------
error[E0658]: use of unstable library feature 'track_path'
error[E0658]: use of unstable library feature 'track_path'
  --> macro_def.rs:11:5
   |
11 |     tracked_path::path("emojis.txt");
   |
   = note: see issue #73921 <https://github.com/rust-lang/rust/issues/73921> for more information
   = help: add `#![feature(track_path)]` to the crate attributes to enable


error[E0658]: use of unstable library feature 'track_path'
  --> macro_def.rs:18:5
   |
18 |     tracked_path::path(invalid);
   |
   = note: see issue #73921 <https://github.com/rust-lang/rust/issues/73921> for more information
   = help: add `#![feature(track_path)]` to the crate attributes to enable


error: aborting due to 2 previous errors

For more information about this error, try `rustc --explain E0658`.
make: *** [all] Error 1
------------------------------------------




failures:
    [run-make] run-make/track-path-dep-info

test result: FAILED. 18 passed; 1 failed; 17 ignored; 0 measured; 0 filtered out; finished in 3.01s



command did not execute successfully: "/checkout/obj/build/x86_64-unknown-linux-gnu/stage0-tools-bin/compiletest" "--compile-lib-path" "/checkout/obj/build/x86_64-unknown-linux-gnu/stage2/lib" "--run-lib-path" "/checkout/obj/build/x86_64-unknown-linux-gnu/stage2/lib/rustlib/x86_64-unknown-linux-gnu/lib" "--rustc-path" "/checkout/obj/build/x86_64-unknown-linux-gnu/stage2/bin/rustc" "--rustdoc-path" "/checkout/obj/build/x86_64-unknown-linux-gnu/stage2/bin/rustdoc" "--src-base" "/checkout/src/test/run-make" "--build-base" "/checkout/obj/build/x86_64-unknown-linux-gnu/test/run-make" "--stage-id" "stage2-x86_64-unknown-linux-gnu" "--suite" "run-make" "--mode" "run-make" "--target" "x86_64-unknown-linux-gnu" "--host" "x86_64-unknown-linux-gnu" "--llvm-filecheck" "/usr/lib/llvm-10/bin/FileCheck" "--nodejs" "/usr/bin/node" "--host-rustcflags" "-Crpath -O -Cdebuginfo=0  -Lnative=/checkout/obj/build/x86_64-unknown-linux-gnu/native/rust-test-helpers" "--target-rustcflags" "-Crpath -O -Cdebuginfo=0  -Lnative=/checkout/obj/build/x86_64-unknown-linux-gnu/native/rust-test-helpers" "--docck-python" "/usr/bin/python3" "--lldb-python" "/usr/bin/python3" "--gdb" "/usr/bin/gdb" "--quiet" "--llvm-version" "10.0.0" "--llvm-components" "aarch64 aarch64asmparser aarch64codegen aarch64desc aarch64disassembler aarch64info aarch64utils aggressiveinstcombine all all-targets amdgpu amdgpuasmparser amdgpucodegen amdgpudesc amdgpudisassembler amdgpuinfo amdgpuutils analysis arm armasmparser armcodegen armdesc armdisassembler arminfo armutils asmparser asmprinter avr avrasmparser avrcodegen avrdesc avrdisassembler avrinfo binaryformat bitreader bitstreamreader bitwriter bpf bpfasmparser bpfcodegen bpfdesc bpfdisassembler bpfinfo cfguard codegen core coroutines coverage debuginfocodeview debuginfodwarf debuginfogsym debuginfomsf debuginfopdb demangle dlltooldriver dwarflinker engine executionengine frontendopenmp fuzzmutate globalisel hexagon hexagonasmparser hexagoncodegen hexagondesc hexagondisassembler hexagoninfo instcombine instrumentation interpreter ipo irreader jitlink lanai lanaiasmparser lanaicodegen lanaidesc lanaidisassembler lanaiinfo libdriver lineeditor linker lto mc mca mcdisassembler mcjit mcparser mips mipsasmparser mipscodegen mipsdesc mipsdisassembler mipsinfo mirparser msp430 msp430asmparser msp430codegen msp430desc msp430disassembler msp430info native nativecodegen nvptx nvptxcodegen nvptxdesc nvptxinfo objcarcopts object objectyaml option orcerror orcjit passes perfjitevents powerpc powerpcasmparser powerpccodegen powerpcdesc powerpcdisassembler powerpcinfo profiledata remarks riscv riscvasmparser riscvcodegen riscvdesc riscvdisassembler riscvinfo riscvutils runtimedyld scalaropts selectiondag sparc sparcasmparser sparccodegen sparcdesc sparcdisassembler sparcinfo support symbolize systemz systemzasmparser systemzcodegen systemzdesc systemzdisassembler systemzinfo tablegen target textapi transformutils vectorize webassembly webassemblyasmparser webassemblycodegen webassemblydesc webassemblydisassembler webassemblyinfo windowsmanifest x86 x86asmparser x86codegen x86desc x86disassembler x86info x86utils xcore xcorecodegen xcoredesc xcoredisassembler xcoreinfo xray" "--system-llvm" "--llvm-bin-dir" "/usr/lib/llvm-10/bin" "--cc" "cc" "--cxx" "c++" "--cflags" "-ffunction-sections -fdata-sections -fPIC -m64" "--ar" "ar" "--adb-path" "adb" "--adb-test-dir" "/data/tmp/work" "--android-cross-path" "" "--channel" "nightly" "--color" "always"


Build completed unsuccessfully in 0:32:33

petrochenkov · 2021-09-14T11:38:51Z

compiler/rustc_expand/src/proc_macro_server.rs

@@ -406,7 +407,7 @@ impl server::FreeFunctions for Rustc<'_> {
    }

    fn track_path(&mut self, path: &str) {
-        self.sess.file_depinfo.borrow_mut().insert(Symbol::intern(path));
+        self.sess.file_depinfo.borrow_mut().insert(path::PathBuf::from(path));


Suggested change

self.sess.file_depinfo.borrow_mut().insert(path::PathBuf::from(path));

self.sess.file_depinfo.borrow_mut().insert(PathBuf::from(path));

Nit (here and in the other files as well): types are usually imported directly, without parent module.

petrochenkov · 2021-09-14T11:43:24Z

src/test/run-make/track-path-dep-info/macro_def.rs

+    // currently only valid utf-8 paths are supported
+    let invalid = [1_u8, 2,123, 254, 0, 0, 1, 1];
+    let invalid: &str = unsafe {
+        str::from_utf8_unchecked(&invalid[..])


This is undefined behavior though.
I don't think UB is something that we should be testing.

In #87173 it would be possible to safely construct a PathBuf that is not a valid UTF-8 path and then pass it to libproc_macro.

I suggest including the AsRef<Path> part of #87173 into this PR and using it to avoid the UB here.

petrochenkov · 2021-09-14T11:45:30Z

What is the purpose of changing this internally, if the user-facing interface in proc_macro/src/lib.rs still only accepts strings?

drahnr · 2021-09-14T12:11:36Z

Technically the limitation does not need to be there, since the internal representation of a Pathbuf is a Vec<u8>, this could hence be resolved getting access to that internal representation.

drahnr · 2021-09-16T14:21:34Z

@rustbot label -S-waiting-on-author +S-waiting-on-review

petrochenkov · 2021-09-16T15:55:10Z

@drahnr
I'm not sure what the previous commit was answering.
In #88909 (comment) I meant that this PR doesn't make sense without including the AsRef<Path> part from #87173.

(Also, #88909 (comment) wasn't addressed.)

JohnCSimon · 2021-10-03T06:41:44Z

Ping from triage:
Can you please address the build failures and the comments?
@drahnr

drahnr · 2021-10-14T07:05:02Z

Sorry for lagging behind on this. Will resume working on this soon.

JohnCSimon · 2021-10-31T08:13:18Z

Thanks, please adjust the label with @rustbot ready when you're ready for another review

bors · 2021-11-13T11:22:10Z

☔ The latest upstream changes (presumably #87264) made this pull request unmergeable. Please resolve the merge conflicts.

camelid · 2021-12-10T02:17:17Z

triage: @drahnr any updates?

JohnCSimon · 2022-01-30T04:16:55Z

Ping from triage:
@drahnr
I'm closing this due to inactivity, Please to reopen when you are ready to continue with this. Thank you.

@rustbot label: +S-inactive

enable PathBuf usage in proc_macro bridge

0a2ae6d

rust-highfive assigned petrochenkov Sep 13, 2021

rust-highfive added the S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. label Sep 13, 2021

petrochenkov reviewed Sep 14, 2021

View reviewed changes

petrochenkov added S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. and removed S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. labels Sep 14, 2021

rustbot added S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. and removed S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. labels Sep 16, 2021

petrochenkov added S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. and removed S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. labels Sep 16, 2021

apiraino added the T-compiler Relevant to the compiler team, which will review and decide on the PR/issue. label Oct 14, 2021

JohnCSimon added S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. and removed S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. labels Oct 31, 2021

JohnCSimon closed this Jan 30, 2022

rustbot added the S-inactive Status: Inactive and waiting on the author. This is often applied to closed PRs. label Jan 30, 2022

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

enable `PathBuf` usage in proc_macro bridge, `tracked::path` #88909

enable `PathBuf` usage in proc_macro bridge, `tracked::path` #88909

drahnr commented Sep 13, 2021

rust-highfive commented Sep 13, 2021

rust-log-analyzer commented Sep 13, 2021

petrochenkov Sep 14, 2021

petrochenkov Sep 14, 2021

bjorn3 Sep 16, 2021

petrochenkov Sep 16, 2021

petrochenkov commented Sep 14, 2021

drahnr commented Sep 14, 2021

drahnr commented Sep 16, 2021

petrochenkov commented Sep 16, 2021

JohnCSimon commented Oct 3, 2021

drahnr commented Oct 14, 2021

JohnCSimon commented Oct 31, 2021

bors commented Nov 13, 2021

camelid commented Dec 10, 2021

JohnCSimon commented Jan 30, 2022

	self.sess.file_depinfo.borrow_mut().insert(path::PathBuf::from(path));
	self.sess.file_depinfo.borrow_mut().insert(PathBuf::from(path));

enable PathBuf usage in proc_macro bridge, tracked::path #88909

enable PathBuf usage in proc_macro bridge, tracked::path #88909

Conversation

drahnr commented Sep 13, 2021

rust-highfive commented Sep 13, 2021

rust-log-analyzer commented Sep 13, 2021

petrochenkov Sep 14, 2021

Choose a reason for hiding this comment

petrochenkov Sep 14, 2021

Choose a reason for hiding this comment

bjorn3 Sep 16, 2021

Choose a reason for hiding this comment

petrochenkov Sep 16, 2021

Choose a reason for hiding this comment

petrochenkov commented Sep 14, 2021

drahnr commented Sep 14, 2021

drahnr commented Sep 16, 2021

petrochenkov commented Sep 16, 2021

JohnCSimon commented Oct 3, 2021

drahnr commented Oct 14, 2021

JohnCSimon commented Oct 31, 2021

bors commented Nov 13, 2021

camelid commented Dec 10, 2021

JohnCSimon commented Jan 30, 2022

enable `PathBuf` usage in proc_macro bridge, `tracked::path` #88909

enable `PathBuf` usage in proc_macro bridge, `tracked::path` #88909