refcell_ref.md: RefCell::RefMut is fine

[RalfJung]

refcell_ref.md says:

What is interesting about these types are the value fields. They use a normal Rust type, but in fact the guarantees for this are slightly weaker than normal: the reference is valid until either (a) the end of 'b OR (b) the field borrow is dropped, whichever happens first.

However, I think that applies only to RefCell::Ref, not to RefCell::RefMut. (The mistake is probably mine in the original mail to Niko.) The reason for this is that &mut T is non-Copy, so this actually is a reference that's valid for as long as the lifetime says. If the destructor of RefCell::RefMut is executed, ownership is moved from RefMut back to the RefCell. In some sense, having a variable of type &'a mut T does not actually guarantee that the borrow lives for lifetime 'a, all it really says is that if we keep hold of this variable, then the borrow lasts. But if we give up the variable, pass it to someone else, that someone may well kill the borrow earlier.

In contrast to that, &T is Copy, so there's no "giving up" of ownership here.

(I was unsure whether an issue or the internals is the right place for this discussion. Feel free to move.)

[arielb1]

Not sure of it. It's the "disposing of &mut pointers" problem all over again, except you force the pointer to live long enough.

example:

fn foo(var: &mut u32) -> u32 {
    let ptr: *mut u32 = var;
    let ref_ = &mut *var;
    mk_eqty!(ref_, var);
    *ref_ = 2;
    unsafe { *ptr }
}

The mk_eqty would seem to coerce the borrow to last for the entire function, and prohibit the read from ptr.

[RalfJung]

It's the "disposing of &mut pointers" problem all over again

What are you referring to here?

I also don't see the relation between your example and RefMut. What you are doing here (if I understand correctly) is moving the (non-duplicable) ownership associated with var over to ref_, but not in a way that the type system could understand. In contrast, the reference in RefMut is only "invalidated prematurely" if RefMut is dropped, in which case ownership moves back to the RefCell (and whoever is to call borrow/borrow_mut next). In a single function, I would write this as something like

fn foo(var: &mut u32) -> u32 {
    let ref = { let ptr : *mut u32 = var;  let ref_ = &mut *var;  mk_eqty!(ref_, var); drop(var); ref_ };
    *ref = 2;
    *ref
}

[arielb1]

But drop of an &mut reference is a no-op, so it is equivalent. If you inline the drop, you get:

fn foo(var: &mut u32) -> u32 {
    let ptr: *mut u32 = var;
    let ref_ = &mut *var;
    mk_eqty!(ref_, var);
    *ref_ = 2;
    { let _x = ref_; }
    unsafe { *ptr }
}

[arielb1]

You can also have a write-sided version of this, which is more likely to be misoptimized (because abstract-interpretation optimizations move writes backward and reads forward).

fn foo_will_break(var: &mut u32) -> u32 {
    let ptr: *mut u32 = var;

    let ref_ = &mut *var;
    mk_eqty!(ref_, var);
    let result = *ref_;
    { let _x = ref_; }

    unsafe { *ptr = 2; }
    result
}

If the compiler DSEs the write to _x, and then NRVOs (both optimizations that we want to do), this will break.

[RalfJung]

drop is a no-op operationally, but in terms of the ownership available to the function, it makes a huge difference. In my code above, the block after let ref = is a weird way of writing an identity function and making sure we use the full lifetime of var for ref. This makes sure we actually do an ownership transfer from var to ref, like RefMut does when it is dropped. All of this is just my attempt of communicating the ownership transfer going on in RefMut via code, which obviously failed. ;-)

What I am saying in my first post is: The semantic type RefMut<'a, T> has strictly more ownership than its syntactic description indicates. There actually is a &'a T there (actually, it's some superlifetime of 'a). Adding safe code that performs read-only operations on (the internals of) RefMut cannot break the module.

This is in contrast to Ref<'a, T>, where the semantic type is incomparable to the syntactic appearance (it's neither more nor less ownership), and where one can easily write safe code performing read-only operations on RefMut which can lead to UB. For example, this is not safe:

impl<'b, T> Ref<'b, T> {
  fn get_ptr(self) -> &'b T {
    self.value
  }
}

The latter is the observation I brought up in a mail to Niko, which is the source of refcell_ref.md. I can't see any similar issue with RefMut, and that's what this issue is about.

Unfortunately, I don't understand what this observation has to do with the code you are bringing up. There's no compiler transformations involved here, I am just looking at the flow of ownership. Here, "ownership" is a fairly wide term, it refers to anything you can "own" in Rust -- in particular, you can "own a mutable borrow of a T", which means you have sth. of type &mut T, just like "owning a T" is having sth. of type T. A type can own more than is syntactically apparent, for example, Vec owns the heap allocation it refers to even though there's just a raw pointer there (well, there's a Unique in a RawVec, but you get the point). And Ref goes to show that a type can also own less than is syntactically apparent.

[arielb1]

The problem is that, after inlining, foo_will_break is equivalent to

fn refcell_example(r: &mut RefCell<u32>) -> u32 {
    let ref_ = r.borrow_mut();
    let result = *ref_;
    drop(ref_);

    *r.borrow_mut() = 2;
    return result;
}

An &'a mut T controls all access to T until 'a ends. If you forget it, that means that nobody can access T until 'a ends.

My chain-based model allows you to "expose" a reference and remove its control over accesses, but drop::<&mut T> is a no-op, not an expose.

[RalfJung]

I would argue there is a very fundamental difference between refcell_example and foo_will_break: The latter removes a layer of abstraction. Also, the raw pointer ptr fails to capture the way r owns its content.

If we have a semantics that reasons by looking at types (e.g. to exclude aliasing), I am not surprised that doing such a type-erasing kind of inlining breaks programs.

In other words, the unsafe code in RefCell is safe because of careful tuning of all the types exported from that module. Hence optimizations that "smear" this border must be fairly conservative, because they can quickly fail to respect the abstraction that has been so carefully established.

[arielb1]

But the example does not mess with types. It only messes with privacy. I am a little worried about introducing privacy to the aliasing rules.

[RalfJung]

Privacy is the one mechanism in Rust which makes abstraction work. Without privacy, the entire story of "ascribing additional meaning to types" (and therefore the entire story of hiding unsafety behind an abstraction barrier) breaks down.

[arielb1]

@RalfJung

Sure. But UB does not care about which additional meaning you ascribe to your types - if it did, then Ref would work too.

In your way, the aliasing rules would have to say something like "if you drop a struct containing an private &mut field, the &mut field is then counted as destroyed". And I don't want that.

[nikomatsakis]

@RalfJung I do not see the difference between Ref and RefMut. Either way, the reference stored in the value field is not necessarily valid for the entirety of its lifetime. Once the borrow is dropped, it is no longer conceptually valid.

The example I gave in the original Tootsie Pop model post seems to apply. Imagine that we had this function:

impl<'b> RefMut<'b, u32> {
    pub fn broken(self) {
        let x = *self.value;
        mem::drop(self.borrow);
        use(x);
    }
}

It seems to me that, at least under an aggressive set of rules, the compiler would be within its rights to reorder those statements:

impl<'b> RefMut<'b, u32> {
    pub fn broken(self) {
        mem::drop(self.borrow);
        let x = *self.value;
        use(x);
    }
}

After all, self.value should be a unique reference not aliased anywhere else. Dropping self.borrow is a normal, safe function call, so we'd like to be able to assume that it cannot have effects exceeding those of a normal, safe function -- which wouldn't be able to access self.value (and certainly not invalidate it early).

(And of course to make this all more deadly we can think of RefCell as a true mutex instead of one specialized to a single thread.)