Incorrect code behavior on aarch64

[berkus]

The following code compiles and works correctly on x86_64. However, the same code when compiled for aarch64 with opt-level = "s" does not work.

https://github.com/berkus/mre-write_to
The culprit is the function show() in write_to.rs:44

(one-page version here -- this however is not enough for reproduction)

When used from the same module it correctly generates a &str from buffer slice. When used from another module the resulting str is "" (empty).

The original code location, where bug is easily reproducible by running makers test is here

(Disclaimer: custom target file, should match LLVM layout for aarch64 however; custom linker script for OS kernel)

There, marking the function with #[inline] makes it work correctly at call sites other than the same module (for example, from here). Removing inline modifier makes it produce "" (empty) strs. It still works correctly from within the same module (e.g. local unit tests pass).

Changing opt-level from "s" to 2 or 3 also fixes the issue.

Replacing from_utf8_unchecked with from_utf8().unwrap() causes different behavior. Now opt-levels 2 and "s" produce empty strs while opt-level 3 works as intended.

It looks like code generation bug to me.

$ rustc --version
rustc 1.42.0-nightly (cd1ef39 2020-01-31)

Ready to provide more information or test possible fixes.

[LeSeulArtichaut]

Let’s try to have an MCVE for this?
@rustbot ping arm

[rustbot]

Hey ARM Group! This bug has been identified as a good "ARM candidate".
In case it's useful, here are some [instructions] for tackling these sorts of
bugs. Maybe take a look?
Thanks! <3
[instructions]: https://rustc-dev-guide.rust-lang.org/notification-groups/arm.html

cc @JamieCunliffe @joaopaulocarreiro @raw-bin @Stammark @vigoux

[raw-bin]

Thanks! :) We'll have a look and respond. Cheers

…

On Fri, Jun 12, 2020 at 7:31 PM rustbot ***@***.***> wrote: Hey ARM Group! This bug has been identified as a good "ARM candidate". In case it's useful, here are some [instructions] for tackling these sorts of bugs. Maybe take a look? Thanks! <3 [instructions]: https://rustc-dev-guide.rust-lang.org/notification-groups/arm.html cc @JamieCunliffe <https://github.com/JamieCunliffe> @joaopaulocarreiro <https://github.com/joaopaulocarreiro> @raw-bin <https://github.com/raw-bin> @Stammark <https://github.com/Stammark> @vigoux <https://github.com/vigoux> — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#68812 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAFMKYRJN6ZVUFWCMR4YXHTRWJYAZANCNFSM4KPNWSYA> .

[berkus]

Hmm, if I understand what MCVE is, does this actually reproduce this for you?

[JamieCunliffe]

I ran that repo on my aarch64 box and it looked ok. It printed hello world when ran (both debug and release) and the test passes when run.

Is there something else I have to do to get that to show the issue?

[berkus]

@JamieCunliffe which rustc version did you use?

[raw-bin]

Hi @berkus.

A data point: I just had a play with your reproducer on a native AArch64 machine:

jagran01@scm-arm1 ~/work/projects/rustc-tickets/68812/mre-write_to $ uname -a
Linux scm-arm1 5.0.0-38-generic #41-Ubuntu SMP Tue Dec 3 00:29:46 UTC 2019 aarch64 GNU/Linux

jagran01@scm-arm1 ~/work/projects/rustc-tickets/68812 $ git clone https://github.com/berkus/mre-write_to.git
Cloning into 'mre-write_to'...
remote: Enumerating objects: 9, done.
remote: Counting objects: 100% (9/9), done.
remote: Compressing objects: 100% (8/8), done.
remote: Total 9 (delta 0), reused 9 (delta 0), pack-reused 0
Unpacking objects: 100% (9/9), done.

jagran01@scm-arm1 ~/work/projects/rustc-tickets/68812 $ cd mre-write_to/

jagran01@scm-arm1 ~/work/projects/rustc-tickets/68812/mre-write_to $ cargo build
   Compiling test-write-to v0.1.0 (/home/jagran01/work/projects/rustc-tickets/68812/mre-write_to)
error[E0554]: `#![feature]` may not be used on the stable release channel
 --> src/main.rs:2:1
  |
2 | #![feature(format_args_nl)]
  | ^^^^^^^^^^^^^^^^^^^^^^^^^^^

error: aborting due to previous error

For more information about this error, try `rustc --explain E0554`.
error: could not compile `test-write-to`.

To learn more, run the command again with --verbose.

jagran01@scm-arm1 ~/work/projects/rustc-tickets/68812/mre-write_to $ echo nightly > rust-toolchain

jagran01@scm-arm1 ~/work/projects/rustc-tickets/68812/mre-write_to $ ~/.rustup/toolchains/nightly-aarch64-unknown-linux-gnu/bin/rustc --version
rustc 1.47.0-nightly (81e754c35 2020-08-02)

jagran01@scm-arm1 ~/work/projects/rustc-tickets/68812/mre-write_to $ cargo build
   Compiling test-write-to v0.1.0 (/home/jagran01/work/projects/rustc-tickets/68812/mre-write_to)
    Finished dev [unoptimized + debuginfo] target(s) in 1.05s

jagran01@scm-arm1 ~/work/projects/rustc-tickets/68812/mre-write_to $ cargo run
    Finished dev [unoptimized + debuginfo] target(s) in 0.01s
     Running `target/debug/test-write-to`
Converting to str
Converted to str
Hello, world!

jagran01@scm-arm1 ~/work/projects/rustc-tickets/68812/mre-write_to $ cargo test
    Finished test [unoptimized + debuginfo] target(s) in 0.02s
     Running target/debug/deps/test_write_to-d86fea7b5f3da2c4

running 1 test
Converting to str
test tests::write_to_works ... ok

test result: ok. 1 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out

jagran01@scm-arm1 ~/work/projects/rustc-tickets/68812/mre-write_to $ cargo clean

jagran01@scm-arm1 ~/work/projects/rustc-tickets/68812/mre-write_to $ cargo build --verbose
   Compiling test-write-to v0.1.0 (/home/jagran01/work/projects/rustc-tickets/68812/mre-write_to)
     Running `rustc --crate-name test_write_to --edition=2018 src/main.rs --error-format=json --json=diagnostic-rendered-ansi --crate-type bin --emit=dep-info,link -C panic=abort -Cembed-bitcode=no -C debuginfo=2 -C metadata=50ae77b73baa9539 -C extra-filename=-50ae77b73baa9539 --out-dir /home/jagran01/work/projects/rustc-tickets/68812/mre-write_to/target/debug/deps -C incremental=/home/jagran01/work/projects/rustc-tickets/68812/mre-write_to/target/debug/incremental -L dependency=/home/jagran01/work/projects/rustc-tickets/68812/mre-write_to/target/debug/deps`
    Finished dev [unoptimized + debuginfo] target(s) in 1.00s

On the face of it, it appears that the test succeeds with nightly. There are some assumptions in the test that couple it to nightly so I couldn't test other versions.

Can you confirm that this is expected ?
Is there anything else you would like us to try ?

Thanks

[berkus]

@raw-bin yeah, I want to get to root of it, so from your output looks like this is fixed in the recent nightlies - I will give it a try.

Still wanted know what version @JamieCunliffe used, but I guess it's too late for that - been a while.

Give me a few days and I'll run the check on my Raspberry Pi with latest nightly and tell you how it went.

[berkus]

@raw-bin looking at my cargo.toml I realised you have to run it in --release mode, have you tried that?

[profile.release]
opt-level = 's'   <-- this is the thing
panic = "abort"

[raw-bin]

@berkus. Just tried this with the same setup. Looks OK to me but please confirm:

jagran01@scm-arm1 ~/work/projects/rustc-tickets/68812/mre-write_to $ cargo clean
jagran01@scm-arm1 ~/work/projects/rustc-tickets/68812/mre-write_to $ cargo test --release
   Compiling test-write-to v0.1.0 (/home/jagran01/work/projects/rustc-tickets/68812/mre-write_to)
    Finished release [optimized] target(s) in 1.12s
     Running target/release/deps/test_write_to-8d8e43f8696dcaf3

running 1 test
Converting to str
test tests::write_to_works ... ok

test result: ok. 1 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out

[berkus]

Ok, I will run regression test from the moment I reported it to the latest compiler, gimme a few days pls.

[berkus]

I've tested in the real project and it looks like the problem is not fixed:

git clone [email protected]:metta-systems/vesper.git
git checkout  test/aarch64-miscompile
just test

this will run tests in qemu, tests print text stuff.

if you run it now it will print garbage instead of text:

[cargo-make][2] INFO - Running Task: test-runner
Exception return from AArch64 EL2 to AArch64 EL1 PC 0x80040
Taking exception 16 [Semihosting call]
...from EL1 to EL0
...handling as semihosting call 0x4
�������Taking exception 16 [Semihosting call]
...from EL1 to EL0
...handling as semihosting call 0x4
�������Taking exception 16 [Semihosting call]
...from EL1 to EL0
...handling as semihosting call 0x18
[cargo-make][2] INFO - Build Done in 0 seconds.

this code is miscompiled.

Now go back 1 commit to 8202ba3e8b1728c45c1587f84c577da27e596713 (With #[inline] attribute it works) and lo and behold, it works:

just test

...


[cargo-make][2] INFO - Running Task: test-runner
Exception return from AArch64 EL2 to AArch64 EL1 PC 0x80040
Taking exception 16 [Semihosting call]
...from EL1 to EL0
...handling as semihosting call 0x4
Running 2 tests
Taking exception 16 [Semihosting call]
...from EL1 to EL0
...handling as semihosting call 0x4
[success]
Taking exception 16 [Semihosting call]
...from EL1 to EL0
...handling as semihosting call 0x18
[cargo-make][2] INFO - Build Done in 0 seconds.

[Aaron1011]

@berkus: It looks like you're not declaring some registers as being clobbered:

https://github.com/metta-systems/vesper/blob/bd267e44abd72712bf88cc6d489cf2a2939ad4cf/nucleus/src/qemu.rs#L5-L13

You're writing to the w0 and x1 registers, but you're only declaring a memory clobber. LLVM doesn't try to parse your ASM snippet to determine clobbers, so LLVM may decide to store other things in w0 or x1 without saving them.

Changing the clobbers line to "memory", "w0", "x1" fixed the miscompilation for me.

[berkus]

Oh, thanks, @Aaron1011 , I missed this. Asm IS hard :)

Switching from llvm_asm!() to asm!() should help here, I'll validate and then close. Thank you again!

[berkus]

I'm an idiot!

Replaced the code with

        let cmd = 0x04;
        unsafe {
            asm!(
                "hlt #0xF000"
                , in("w0") cmd
                , in("x1") text.as_ptr() as u64
            );
        }

and it now works! Thank you!