mutable_transmutes lint should catch transmutes from a type without interior mutability to one with

[glandium]

I'm filing this as a regression, although the summary is worded in a feature request-ish way.

So, the regression itself is this: code that transmutes from a type without interior mutability to one with interior mutability leads to really bad outcomes after the bump to LLVM 16, because of the changes in llvm/llvm-project@01859da. This is a pattern that happens in the wild, probably mostly around FFI. At least, that's how it happens in the Firefox codebase.

Code

Here is a reduced testcase using a similar pattern to what Firefox is using:

pub struct Foo(std::cell::UnsafeCell<usize>);

pub struct Bar([u8; 0]);

pub fn foo(f: &Bar) {
    unsafe {
        let f = std::mem::transmute::<&Bar, &Foo>(f);
        *(f.0.get()) += 1;
    }
}

With rustc up to 1.69.0 in --release mode, this produces the following IR:

define void @_ZN10playground3foo17hc0e352349f95f9ecE(ptr noalias nocapture noundef nonnull readonly align 1 %f) unnamed_addr #0 {
start:
  %0 = load i64, ptr %f, align 8, !noundef !2
  %1 = add i64 %0, 1
  store i64 %1, ptr %f, align 8
  ret void
}

With rustc 1.70.0-beta.2 in --release mode, this produces the following IR:

define void @_ZN10playground3foo17h2141d3a0b5fe8d73E(ptr noalias nocapture noundef nonnull readonly align 1 %f) unnamed_addr #0 {
start:
  ret void
}

Note how everything is gone because the input pointer is marked as noalias readonly, and thus the code is not expected to change what it points to, so it's all removed. This is the typical example of undefined behavior leading to the optimizer doing unexpected things. I'm not arguing that there isn't undefined behavior. The undefined behavior existed before. But with LLVM 16, now the undefined behavior is actively dangerous.

Now, to come back to the feature-request-y summary I wrote, the following code does not compile:

pub struct Bar([u8; 0]);

pub fn foo(f: &Bar) {
    unsafe {
        let _ = std::mem::transmute::<&Bar, &mut Bar>(f);
    }
}

The produced error is:

error: transmuting &T to &mut T is undefined behavior, even if the reference is unused, consider instead using an UnsafeCell

I would argue that the code that is now compiled to nothing should also produce a similar error, and that error should be shipped in 1.70.0.

Cc: @pcwalton, @nikic

[apiraino]

WG-prioritization assigning priority (Zulip discussion).

@rustbot label -I-prioritize +P-high +T-compiler +A-LLVM +regression-from-stable-to-beta

[ChayimFriedman2]

@rustbot claim