Auto merge of #143545 - compiler-errors:coroutine-obl, r=oli-obk

`-Zhigher-ranked-assumptions`: Consider WF of coroutine witness when proving outlives assumptions

### TL;DR

This PR introduces an unstable flag `-Zhigher-ranked-assumptions` which tests out a new algorithm for dealing with some of the higher-ranked outlives problems that come from auto trait bounds on coroutines. See:

* #110338

While it doesn't fix all of the issues, it certainly fixed many of them, so I'd like to get this landed so people can test the flag on their own code.

### Background

Consider, for example:

```rust
use std::future::Future;

trait Client {
    type Connecting<'a>: Future + Send
    where
        Self: 'a;

    fn connect(&self) -> Self::Connecting<'_>;
}

fn call_connect<C>(c: C) -> impl Future + Send
where
    C: Client + Send + Sync,
{
    async move { c.connect().await }
}
```

Due to the fact that we erase the lifetimes in a coroutine, we can think of the interior type of the async block as something like: `exists<'r, 's> { C, &'r C, C::Connecting<'s> }`. The first field is the `c` we capture, the second is the auto-ref that we perform on the call to `.connect()`, and the third is the resulting future we're awaiting at the first and only await point. Note that every region is uniquified differently in the interior types.

For the async block to be `Send`, we must prove that both of the interior types are `Send`. First, we have an `exists<'r, 's>` binder, which needs to be instantiated universally since we treat the regions in this binder as *unknown*[^exist]. This gives us two types: `{ &'!r C, C::Connecting<'!s> }`. Proving `&'!r C: Send` is easy due to a [`Send`](https://doc.rust-lang.org/nightly/std/marker/trait.Send.html#impl-Send-for-%26T) impl for references.

Proving `C::Connecting<'!s>: Send` can only be done via the item bound, which then requires `C: '!s` to hold (due to the `where Self: 'a` on the associated type definition). Unfortunately, we don't know that `C: '!s` since we stripped away any relationship between the interior type and the param `C`. This leads to a bogus borrow checker error today!

### Approach

Coroutine interiors are well-formed by virtue of them being borrow-checked, as long as their callers are invoking their parent functions in a well-formed way, then substitutions should also be well-formed. Therefore, in our example above, we should be able to deduce the assumption that `C: '!s` holds from the well-formedness of the interior type `C::Connecting<'!s>`.

This PR introduces the notion of *coroutine assumptions*, which are the outlives assumptions that we can assume hold due to the well-formedness of a coroutine's interior types. These are computed alongside the coroutine types in the `CoroutineWitnessTypes` struct. When we instantiate the binder when proving an auto trait for a coroutine, we instantiate the `CoroutineWitnessTypes` and stash these newly instantiated assumptions in the region storage in the `InferCtxt`. Later on in lexical region resolution or MIR borrowck, we use these registered assumptions to discharge any placeholder outlives obligations that we would otherwise not be able to prove.

### How well does it work?

I've added a ton of tests of different reported situations that users have shared on issues like #110338, and an (anecdotally) large number of those examples end up working straight out of the box! Some limitations are described below.

### How badly does it not work?

The behavior today is quite rudimentary, since we currently discharge the placeholder assumptions pretty early in region resolution. This manifests itself as some limitations on the code that we accept.

For example, `tests/ui/async-await/higher-ranked-auto-trait-11.rs` continues to fail. In that test, we must prove that a placeholder is equal to a universal for a param-env candidate to hold when proving an auto trait, e.g. `'!1 = 'a` is required to prove `T: Trait<'!1>` in a param-env that has `T: Trait<'a>`. Unfortunately, at that point in the MIR body, we only know that the placeholder is equal to some body-local existential NLL var `'?2`, which only gets equated to the universal `'a` when being stored into the return local later on in MIR borrowck.

This could be fixed by integrating these assumptions into the type outlives machinery in a more first-class way, and delaying things to the end of MIR typeck when we know the full relationship between existential and universal NLL vars. Doing this integration today is quite difficult today.

`tests/ui/async-await/higher-ranked-auto-trait-11.rs` fails because we don't compute the full transitive outlives relations between placeholders. In that test, we have in our region assumptions that some `'!1 = '!2` and `'!2 = '!3`, but we must prove `'!1 = '!3`.

This can be fixed by computing the set of coroutine outlives assumptions in a more transitive way, or as I mentioned above, integrating these assumptions into the type outlives machinery in a more first-class way, since it's already responsible for the transitive outlives assumptions of universals.

### Moving forward

I'm still quite happy with this implementation, and I'd like to land it for testing. I may work on overhauling both the way we compute these coroutine assumptions and also how we deal with the assumptions during (lexical/nll) region checking. But for now, I'd like to give users a chance to try out this new `-Zhigher-ranked-assumptions` flag to uncover more shortcomings.

[^exist]: Instantiating this binder with infer regions would be incomplete, since we'd be asking for *some* instantiation of the interior types, not proving something for *all* instantiations of the interior types.

Author: bors

compiler/rustc_borrowck/src/type_check/constraint_conversion.rs

@@ -1,3 +1,4 @@
+use rustc_data_structures::fx::FxHashSet;
 use rustc_hir::def_id::LocalDefId;
 use rustc_infer::infer::canonical::QueryRegionConstraints;
 use rustc_infer::infer::outlives::env::RegionBoundPairs;
@@ -7,7 +8,7 @@ use rustc_infer::infer::{InferCtxt, SubregionOrigin};
 use rustc_infer::traits::query::type_op::DeeplyNormalize;
 use rustc_middle::bug;
 use rustc_middle::ty::{
-    self, GenericArgKind, Ty, TyCtxt, TypeFoldable, TypeVisitableExt, fold_regions,
+    self, GenericArgKind, Ty, TyCtxt, TypeFoldable, TypeVisitableExt, elaborate, fold_regions,
 };
 use rustc_span::Span;
 use rustc_trait_selection::traits::query::type_op::{TypeOp, TypeOpOutput};
@@ -70,10 +71,12 @@ impl<'a, 'tcx> ConstraintConversion<'a, 'tcx> {
 
     #[instrument(skip(self), level = "debug")]
     pub(super) fn convert_all(&mut self, query_constraints: &QueryRegionConstraints<'tcx>) {
-        let QueryRegionConstraints { outlives } = query_constraints;
+        let QueryRegionConstraints { outlives, assumptions } = query_constraints;
+        let assumptions =
+            elaborate::elaborate_outlives_assumptions(self.infcx.tcx, assumptions.iter().copied());
 
         for &(predicate, constraint_category) in outlives {
-            self.convert(predicate, constraint_category);
+            self.convert(predicate, constraint_category, &assumptions);
         }
     }
 
@@ -112,15 +115,20 @@ impl<'a, 'tcx> ConstraintConversion<'a, 'tcx> {
 
             self.category = outlives_requirement.category;
             self.span = outlives_requirement.blame_span;
-            self.convert(ty::OutlivesPredicate(subject, outlived_region), self.category);
+            self.convert(
+                ty::OutlivesPredicate(subject, outlived_region),
+                self.category,
+                &Default::default(),
+            );
         }
         (self.category, self.span, self.from_closure) = backup;
     }
 
     fn convert(
         &mut self,
-        predicate: ty::OutlivesPredicate<'tcx, ty::GenericArg<'tcx>>,
+        predicate: ty::ArgOutlivesPredicate<'tcx>,
         constraint_category: ConstraintCategory<'tcx>,
+        higher_ranked_assumptions: &FxHashSet<ty::ArgOutlivesPredicate<'tcx>>,
     ) {
         let tcx = self.infcx.tcx;
         debug!("generate: constraints at: {:#?}", self.locations);
@@ -150,7 +158,15 @@ impl<'a, 'tcx> ConstraintConversion<'a, 'tcx> {
             }
 
             let mut next_outlives_predicates = vec![];
-            for (ty::OutlivesPredicate(k1, r2), constraint_category) in outlives_predicates {
+            for (pred, constraint_category) in outlives_predicates {
+                // Constraint is implied by a coroutine's well-formedness.
+                if self.infcx.tcx.sess.opts.unstable_opts.higher_ranked_assumptions
+                    && higher_ranked_assumptions.contains(&pred)
+                {
+                    continue;
+                }
+
+                let ty::OutlivesPredicate(k1, r2) = pred;
                 match k1.kind() {
                     GenericArgKind::Lifetime(r1) => {
                         let r1_vid = self.to_region_vid(r1);
@@ -266,14 +282,15 @@ impl<'a, 'tcx> ConstraintConversion<'a, 'tcx> {
         &self,
         ty: Ty<'tcx>,
         next_outlives_predicates: &mut Vec<(
-            ty::OutlivesPredicate<'tcx, ty::GenericArg<'tcx>>,
+            ty::ArgOutlivesPredicate<'tcx>,
             ConstraintCategory<'tcx>,
         )>,
     ) -> Ty<'tcx> {
         match self.param_env.and(DeeplyNormalize { value: ty }).fully_perform(self.infcx, self.span)
         {
             Ok(TypeOpOutput { output: ty, constraints, .. }) => {
-                if let Some(QueryRegionConstraints { outlives }) = constraints {
+                // FIXME(higher_ranked_auto): What should we do with the assumptions here?
+                if let Some(QueryRegionConstraints { outlives, assumptions: _ }) = constraints {
                     next_outlives_predicates.extend(outlives.iter().copied());
                 }
                 ty

compiler/rustc_borrowck/src/type_check/mod.rs

@@ -131,6 +131,11 @@ pub(crate) fn type_check<'tcx>(
         pre_obligations.is_empty(),
         "there should be no incoming region obligations = {pre_obligations:#?}",
     );
+    let pre_assumptions = infcx.take_registered_region_assumptions();
+    assert!(
+        pre_assumptions.is_empty(),
+        "there should be no incoming region assumptions = {pre_assumptions:#?}",
+    );
 
     debug!(?normalized_inputs_and_output);
 

compiler/rustc_hir_analysis/src/outlives/utils.rs

@@ -7,8 +7,7 @@ use smallvec::smallvec;
 
 /// Tracks the `T: 'a` or `'a: 'a` predicates that we have inferred
 /// must be added to the struct header.
-pub(crate) type RequiredPredicates<'tcx> =
-    FxIndexMap<ty::OutlivesPredicate<'tcx, ty::GenericArg<'tcx>>, Span>;
+pub(crate) type RequiredPredicates<'tcx> = FxIndexMap<ty::ArgOutlivesPredicate<'tcx>, Span>;
 
 /// Given a requirement `T: 'a` or `'b: 'a`, deduce the
 /// outlives_component and add it to `required_predicates`

compiler/rustc_infer/src/infer/canonical/query_response.rs

@@ -116,9 +116,15 @@ impl<'tcx> InferCtxt<'tcx> {
         }
 
         let region_obligations = self.take_registered_region_obligations();
+        let region_assumptions = self.take_registered_region_assumptions();
         debug!(?region_obligations);
         let region_constraints = self.with_region_constraints(|region_constraints| {
-            make_query_region_constraints(tcx, region_obligations, region_constraints)
+            make_query_region_constraints(
+                tcx,
+                region_obligations,
+                region_constraints,
+                region_assumptions,
+            )
         });
         debug!(?region_constraints);
 
@@ -169,6 +175,11 @@ impl<'tcx> InferCtxt<'tcx> {
             self.register_outlives_constraint(predicate, cause);
         }
 
+        for assumption in &query_response.value.region_constraints.assumptions {
+            let assumption = instantiate_value(self.tcx, &result_args, *assumption);
+            self.register_region_assumption(assumption);
+        }
+
         let user_result: R =
             query_response.instantiate_projected(self.tcx, &result_args, |q_r| q_r.value.clone());
 
@@ -292,6 +303,18 @@ impl<'tcx> InferCtxt<'tcx> {
             }),
         );
 
+        // FIXME(higher_ranked_auto): Optimize this to instantiate all assumptions
+        // at once, rather than calling `instantiate_value` repeatedly which may
+        // create more universes.
+        output_query_region_constraints.assumptions.extend(
+            query_response
+                .value
+                .region_constraints
+                .assumptions
+                .iter()
+                .map(|&r_c| instantiate_value(self.tcx, &result_args, r_c)),
+        );
+
         let user_result: R =
             query_response.instantiate_projected(self.tcx, &result_args, |q_r| q_r.value.clone());
 
@@ -567,6 +590,7 @@ pub fn make_query_region_constraints<'tcx>(
     tcx: TyCtxt<'tcx>,
     outlives_obligations: Vec<TypeOutlivesConstraint<'tcx>>,
     region_constraints: &RegionConstraintData<'tcx>,
+    assumptions: Vec<ty::ArgOutlivesPredicate<'tcx>>,
 ) -> QueryRegionConstraints<'tcx> {
     let RegionConstraintData { constraints, verifys } = region_constraints;
 
@@ -602,5 +626,5 @@ pub fn make_query_region_constraints<'tcx>(
         }))
         .collect();
 
-    QueryRegionConstraints { outlives }
+    QueryRegionConstraints { outlives, assumptions }
 }

compiler/rustc_infer/src/infer/mod.rs

@@ -149,6 +149,13 @@ pub struct InferCtxtInner<'tcx> {
     /// that all type inference variables have been bound and so forth.
     region_obligations: Vec<TypeOutlivesConstraint<'tcx>>,
 
+    /// The outlives bounds that we assume must hold about placeholders that
+    /// come from instantiating the binder of coroutine-witnesses. These bounds
+    /// are deduced from the well-formedness of the witness's types, and are
+    /// necessary because of the way we anonymize the regions in a coroutine,
+    /// which may cause types to no longer be considered well-formed.
+    region_assumptions: Vec<ty::ArgOutlivesPredicate<'tcx>>,
+
     /// Caches for opaque type inference.
     opaque_type_storage: OpaqueTypeStorage<'tcx>,
 }
@@ -164,7 +171,8 @@ impl<'tcx> InferCtxtInner<'tcx> {
             int_unification_storage: Default::default(),
             float_unification_storage: Default::default(),
             region_constraint_storage: Some(Default::default()),
-            region_obligations: vec![],
+            region_obligations: Default::default(),
+            region_assumptions: Default::default(),
             opaque_type_storage: Default::default(),
         }
     }
@@ -174,6 +182,11 @@ impl<'tcx> InferCtxtInner<'tcx> {
         &self.region_obligations
     }
 
+    #[inline]
+    pub fn region_assumptions(&self) -> &[ty::ArgOutlivesPredicate<'tcx>] {
+        &self.region_assumptions
+    }
+
     #[inline]
     pub fn projection_cache(&mut self) -> traits::ProjectionCache<'_, 'tcx> {
         self.projection_cache.with_log(&mut self.undo_log)

compiler/rustc_infer/src/infer/outlives/env.rs

@@ -1,4 +1,4 @@
-use rustc_data_structures::fx::FxIndexSet;
+use rustc_data_structures::fx::{FxHashSet, FxIndexSet};
 use rustc_data_structures::transitive_relation::TransitiveRelationBuilder;
 use rustc_middle::{bug, ty};
 use tracing::debug;
@@ -39,6 +39,9 @@ pub struct OutlivesEnvironment<'tcx> {
     /// optimized in the future, though.
     region_bound_pairs: RegionBoundPairs<'tcx>,
     known_type_outlives: Vec<ty::PolyTypeOutlivesPredicate<'tcx>>,
+    /// Assumptions that come from the well-formedness of coroutines that we prove
+    /// auto trait bounds for during the type checking of this body.
+    higher_ranked_assumptions: FxHashSet<ty::ArgOutlivesPredicate<'tcx>>,
 }
 
 /// "Region-bound pairs" tracks outlives relations that are known to
@@ -52,6 +55,7 @@ impl<'tcx> OutlivesEnvironment<'tcx> {
         param_env: ty::ParamEnv<'tcx>,
         known_type_outlives: Vec<ty::PolyTypeOutlivesPredicate<'tcx>>,
         extra_bounds: impl IntoIterator<Item = OutlivesBound<'tcx>>,
+        higher_ranked_assumptions: FxHashSet<ty::ArgOutlivesPredicate<'tcx>>,
     ) -> Self {
         let mut region_relation = TransitiveRelationBuilder::default();
         let mut region_bound_pairs = RegionBoundPairs::default();
@@ -88,6 +92,7 @@ impl<'tcx> OutlivesEnvironment<'tcx> {
             known_type_outlives,
             free_region_map: FreeRegionMap { relation: region_relation.freeze() },
             region_bound_pairs,
+            higher_ranked_assumptions,
         }
     }
 
@@ -102,4 +107,8 @@ impl<'tcx> OutlivesEnvironment<'tcx> {
     pub fn known_type_outlives(&self) -> &[ty::PolyTypeOutlivesPredicate<'tcx>] {
         &self.known_type_outlives
     }
+
+    pub fn higher_ranked_assumptions(&self) -> &FxHashSet<ty::ArgOutlivesPredicate<'tcx>> {
+        &self.higher_ranked_assumptions
+    }
 }

compiler/rustc_infer/src/infer/outlives/mod.rs

@@ -10,6 +10,7 @@ use super::region_constraints::{RegionConstraintData, UndoLog};
 use super::{InferCtxt, RegionResolutionError, SubregionOrigin};
 use crate::infer::free_regions::RegionRelations;
 use crate::infer::lexical_region_resolve;
+use crate::infer::region_constraints::Constraint;
 
 pub mod env;
 pub mod for_liveness;
@@ -54,18 +55,29 @@ impl<'tcx> InferCtxt<'tcx> {
             }
         };
 
-        let storage = {
+        let mut storage = {
             let mut inner = self.inner.borrow_mut();
             let inner = &mut *inner;
             assert!(
                 self.tainted_by_errors().is_some() || inner.region_obligations.is_empty(),
                 "region_obligations not empty: {:#?}",
-                inner.region_obligations
+                inner.region_obligations,
             );
             assert!(!UndoLogs::<UndoLog<'_>>::in_snapshot(&inner.undo_log));
             inner.region_constraint_storage.take().expect("regions already resolved")
         };
 
+        // Filter out any region-region outlives assumptions that are implied by
+        // coroutine well-formedness.
+        if self.tcx.sess.opts.unstable_opts.higher_ranked_assumptions {
+            storage.data.constraints.retain(|(constraint, _)| match *constraint {
+                Constraint::RegSubReg(r1, r2) => !outlives_env
+                    .higher_ranked_assumptions()
+                    .contains(&ty::OutlivesPredicate(r2.into(), r1)),
+                _ => true,
+            });
+        }
+
         let region_rels = &RegionRelations::new(self.tcx, outlives_env.free_region_map());
 
         let (lexical_region_resolutions, errors) =
@@ -93,6 +105,11 @@ impl<'tcx> InferCtxt<'tcx> {
             "region_obligations not empty: {:#?}",
             self.inner.borrow().region_obligations
         );
+        assert!(
+            self.inner.borrow().region_assumptions.is_empty(),
+            "region_assumptions not empty: {:#?}",
+            self.inner.borrow().region_assumptions
+        );
 
         self.inner.borrow_mut().unwrap_region_constraints().take_and_reset_data()
     }

compiler/rustc_infer/src/infer/outlives/obligations.rs

@@ -84,7 +84,7 @@ use crate::traits::{ObligationCause, ObligationCauseCode};
 impl<'tcx> InferCtxt<'tcx> {
     pub fn register_outlives_constraint(
         &self,
-        ty::OutlivesPredicate(arg, r2): ty::OutlivesPredicate<'tcx, ty::GenericArg<'tcx>>,
+        ty::OutlivesPredicate(arg, r2): ty::ArgOutlivesPredicate<'tcx>,
         cause: &ObligationCause<'tcx>,
     ) {
         match arg.kind() {
@@ -170,6 +170,16 @@ impl<'tcx> InferCtxt<'tcx> {
         std::mem::take(&mut self.inner.borrow_mut().region_obligations)
     }
 
+    pub fn register_region_assumption(&self, assumption: ty::ArgOutlivesPredicate<'tcx>) {
+        let mut inner = self.inner.borrow_mut();
+        inner.undo_log.push(UndoLog::PushRegionAssumption);
+        inner.region_assumptions.push(assumption);
+    }
+
+    pub fn take_registered_region_assumptions(&self) -> Vec<ty::ArgOutlivesPredicate<'tcx>> {
+        std::mem::take(&mut self.inner.borrow_mut().region_assumptions)
+    }
+
     /// Process the region obligations that must be proven (during
     /// `regionck`) for the given `body_id`, given information about
     /// the region bounds in scope and so forth.
@@ -220,6 +230,14 @@ impl<'tcx> InferCtxt<'tcx> {
                 let (sup_type, sub_region) =
                     (sup_type, sub_region).fold_with(&mut OpportunisticRegionResolver::new(self));
 
+                if self.tcx.sess.opts.unstable_opts.higher_ranked_assumptions
+                    && outlives_env
+                        .higher_ranked_assumptions()
+                        .contains(&ty::OutlivesPredicate(sup_type.into(), sub_region))
+                {
+                    continue;
+                }
+
                 debug!(?sup_type, ?sub_region, ?origin);
 
                 let outlives = &mut TypeOutlives::new(

compiler/rustc_infer/src/infer/snapshot/undo_log.rs

@@ -28,6 +28,7 @@ pub(crate) enum UndoLog<'tcx> {
     RegionUnificationTable(sv::UndoLog<ut::Delegate<RegionVidKey<'tcx>>>),
     ProjectionCache(traits::UndoLog<'tcx>),
     PushTypeOutlivesConstraint,
+    PushRegionAssumption,
 }
 
 macro_rules! impl_from {
@@ -77,6 +78,9 @@ impl<'tcx> Rollback<UndoLog<'tcx>> for InferCtxtInner<'tcx> {
                 let popped = self.region_obligations.pop();
                 assert_matches!(popped, Some(_), "pushed region constraint but could not pop it");
             }
+            UndoLog::PushRegionAssumption => {
+                self.region_assumptions.pop();
+            }
         }
     }
 }

compiler/rustc_middle/src/infer/canonical.rs

@@ -81,13 +81,18 @@ pub struct QueryResponse<'tcx, R> {
 #[derive(HashStable, TypeFoldable, TypeVisitable)]
 pub struct QueryRegionConstraints<'tcx> {
     pub outlives: Vec<QueryOutlivesConstraint<'tcx>>,
+    pub assumptions: Vec<ty::ArgOutlivesPredicate<'tcx>>,
 }
 
 impl QueryRegionConstraints<'_> {
-    /// Represents an empty (trivially true) set of region
-    /// constraints.
+    /// Represents an empty (trivially true) set of region constraints.
+    ///
+    /// FIXME(higher_ranked_auto): This could still just be true if there are only assumptions?
+    /// Because I don't expect for us to get cases where an assumption from one query would
+    /// discharge a requirement from another query, which is a potential problem if we did throw
+    /// away these assumptions because there were no constraints.
     pub fn is_empty(&self) -> bool {
-        self.outlives.is_empty()
+        self.outlives.is_empty() && self.assumptions.is_empty()
     }
 }
 
@@ -130,8 +135,7 @@ impl<'tcx, R> QueryResponse<'tcx, R> {
     }
 }
 
-pub type QueryOutlivesConstraint<'tcx> =
-    (ty::OutlivesPredicate<'tcx, GenericArg<'tcx>>, ConstraintCategory<'tcx>);
+pub type QueryOutlivesConstraint<'tcx> = (ty::ArgOutlivesPredicate<'tcx>, ConstraintCategory<'tcx>);
 
 #[derive(Default)]
 pub struct CanonicalParamEnvCache<'tcx> {