Clean up and reorganize traits chapter

src/SUMMARY.md

@@ -18,6 +18,9 @@
 - [The `ty` module: representing types](./ty.md)
 - [Type inference](./type-inference.md)
 - [Trait resolution](./trait-resolution.md)
+    - [Higher-ranked trait bounds](./trait-hrtb.md)
+    - [Caching subtleties](./trait-caching.md)
+    - [Speciailization](./trait-specialization.md)
 - [Type checking](./type-checking.md)
 - [The MIR (Mid-level IR)](./mir.md)
     - [MIR construction](./mir-construction.md)

src/trait-caching.md

@@ -0,0 +1,65 @@
+# Caching and subtle considerations therewith
+
+In general, we attempt to cache the results of trait selection.  This
+is a somewhat complex process. Part of the reason for this is that we
+want to be able to cache results even when all the types in the trait
+reference are not fully known. In that case, it may happen that the
+trait selection process is also influencing type variables, so we have
+to be able to not only cache the *result* of the selection process,
+but *replay* its effects on the type variables.
+
+## An example
+
+The high-level idea of how the cache works is that we first replace
+all unbound inference variables with skolemized versions. Therefore,
+if we had a trait reference `usize : Foo<$t>`, where `$t` is an unbound
+inference variable, we might replace it with `usize : Foo<$0>`, where
+`$0` is a skolemized type. We would then look this up in the cache.
+
+If we found a hit, the hit would tell us the immediate next step to
+take in the selection process (e.g. apply impl #22, or apply where
+clause `X : Foo<Y>`).
+
+On the other hand, if there is no hit, we need to go through the [selection
+process] from scratch. Suppose, we come to the conclusion that the only
+possible impl is this one, with def-id 22:
+
+[selection process]: ./trait-resolution.html#selection
+
+```rust
+impl Foo<isize> for usize { ... } // Impl #22
+```
+
+We would then record in the cache `usize : Foo<$0> => ImplCandidate(22)`. Next
+we would [confirm] `ImplCandidate(22)`, which would (as a side-effect) unify
+`$t` with `isize`.
+
+[confirm]: ./trait-resolution.html#confirmation
+
+Now, at some later time, we might come along and see a `usize :
+Foo<$u>`. When skolemized, this would yield `usize : Foo<$0>`, just as
+before, and hence the cache lookup would succeed, yielding
+`ImplCandidate(22)`. We would confirm `ImplCandidate(22)` which would
+(as a side-effect) unify `$u` with `isize`.
+
+## Where clauses and the local vs global cache
+
+One subtle interaction is that the results of trait lookup will vary
+depending on what where clauses are in scope. Therefore, we actually
+have *two* caches, a local and a global cache. The local cache is
+attached to the [`ParamEnv`], and the global cache attached to the
+[`tcx`]. We use the local cache whenever the result might depend on the
+where clauses that are in scope. The determination of which cache to
+use is done by the method `pick_candidate_cache` in `select.rs`. At
+the moment, we use a very simple, conservative rule: if there are any
+where-clauses in scope, then we use the local cache.  We used to try
+and draw finer-grained distinctions, but that led to a serious of
+annoying and weird bugs like #22019 and #18290. This simple rule seems
+to be pretty clearly safe and also still retains a very high hit rate
+(~95% when compiling rustc).
+
+**TODO**: it looks like `pick_candidate_cache` no longer exists. In
+general, is this section still accurate at all?
+
+[`ParamEnv`]: ./param_env.html
+[`tcx`]: ./ty.html

src/trait-hrtb.md

@@ -0,0 +1,125 @@
+# Higher-ranked trait bounds
+
+One of the more subtle concepts in trait resolution is *higher-ranked trait
+bounds*. An example of such a bound is `for<'a> MyTrait<&'a isize>`.
+Let's walk through how selection on higher-ranked trait references
+works.
+
+## Basic matching and skolemization leaks
+
+Suppose we have a trait `Foo`:
+
+```rust
+trait Foo<X> {
+    fn foo(&self, x: X) { }
+}
+```
+
+Let's say we have a function `want_hrtb` that wants a type which
+implements `Foo<&'a isize>` for any `'a`:
+
+```rust
+fn want_hrtb<T>() where T : for<'a> Foo<&'a isize> { ... }
+```
+
+Now we have a struct `AnyInt` that implements `Foo<&'a isize>` for any
+`'a`:
+
+```rust
+struct AnyInt;
+impl<'a> Foo<&'a isize> for AnyInt { }
+```
+
+And the question is, does `AnyInt : for<'a> Foo<&'a isize>`? We want the
+answer to be yes. The algorithm for figuring it out is closely related
+to the subtyping for higher-ranked types (which is described in [here][hrsubtype]
+and also in a [paper by SPJ]. If you wish to understand higher-ranked
+subtyping, we recommend you read the paper). There are a few parts:
+
+**TODO**: We should define _skolemize_.
+
+1. _Skolemize_ the obligation.
+2. Match the impl against the skolemized obligation.
+3. Check for _skolemization leaks_.
+
+[hrsubtype]: https://github.com/rust-lang/rust/tree/master/src/librustc/infer/higher_ranked/README.md
+[paper by SPJ]: http://research.microsoft.com/en-us/um/people/simonpj/papers/higher-rank/
+
+So let's work through our example.
+
+1. The first thing we would do is to
+skolemize the obligation, yielding `AnyInt : Foo<&'0 isize>` (here `'0`
+represents skolemized region #0). Note that we now have no quantifiers;
+in terms of the compiler type, this changes from a `ty::PolyTraitRef`
+to a `TraitRef`. We would then create the `TraitRef` from the impl,
+using fresh variables for it's bound regions (and thus getting
+`Foo<&'$a isize>`, where `'$a` is the inference variable for `'a`).
+
+2. Next
+we relate the two trait refs, yielding a graph with the constraint
+that `'0 == '$a`.
+
+3. Finally, we check for skolemization "leaks" – a
+leak is basically any attempt to relate a skolemized region to another
+skolemized region, or to any region that pre-existed the impl match.
+The leak check is done by searching from the skolemized region to find
+the set of regions that it is related to in any way. This is called
+the "taint" set. To pass the check, that set must consist *solely* of
+itself and region variables from the impl. If the taint set includes
+any other region, then the match is a failure. In this case, the taint
+set for `'0` is `{'0, '$a}`, and hence the check will succeed.
+
+Let's consider a failure case. Imagine we also have a struct
+
+```rust
+struct StaticInt;
+impl Foo<&'static isize> for StaticInt;
+```
+
+We want the obligation `StaticInt : for<'a> Foo<&'a isize>` to be
+considered unsatisfied. The check begins just as before. `'a` is
+skolemized to `'0` and the impl trait reference is instantiated to
+`Foo<&'static isize>`. When we relate those two, we get a constraint
+like `'static == '0`. This means that the taint set for `'0` is `{'0,
+'static}`, which fails the leak check.
+
+**TODO**: This is because `'static` is not a region variable but is in the taint set, right?
+
+## Higher-ranked trait obligations
+
+Once the basic matching is done, we get to another interesting topic:
+how to deal with impl obligations. I'll work through a simple example
+here. Imagine we have the traits `Foo` and `Bar` and an associated impl:
+
+```rust
+trait Foo<X> {
+    fn foo(&self, x: X) { }
+}
+
+trait Bar<X> {
+    fn bar(&self, x: X) { }
+}
+
+impl<X,F> Foo<X> for F
+    where F : Bar<X>
+{
+}
+```
+
+Now let's say we have a obligation `Baz: for<'a> Foo<&'a isize>` and we match
+this impl. What obligation is generated as a result? We want to get
+`Baz: for<'a> Bar<&'a isize>`, but how does that happen?
+
+After the matching, we are in a position where we have a skolemized
+substitution like `X => &'0 isize`. If we apply this substitution to the
+impl obligations, we get `F : Bar<&'0 isize>`. Obviously this is not
+directly usable because the skolemized region `'0` cannot leak out of
+our computation.
+
+What we do is to create an inverse mapping from the taint set of `'0`
+back to the original bound region (`'a`, here) that `'0` resulted
+from. (This is done in `higher_ranked::plug_leaks`). We know that the
+leak check passed, so this taint set consists solely of the skolemized
+region itself plus various intermediate region variables. We then walk
+the trait-reference and convert every region in that taint set back to
+a late-bound region, so in this case we'd wind up with `Baz: for<'a> Bar<&'a isize>`.