From 67607372a125d14a61914c06f1c58fce8c85a25f Mon Sep 17 00:00:00 2001 From: Mason Boeman Date: Wed, 24 Nov 2021 17:52:47 -0600 Subject: [PATCH 01/15] beginning progress --- text/0000-unchecked-lifetime.md | 152 ++++++++++++++++++++++++++++++++ 1 file changed, 152 insertions(+) create mode 100644 text/0000-unchecked-lifetime.md diff --git a/text/0000-unchecked-lifetime.md b/text/0000-unchecked-lifetime.md new file mode 100644 index 00000000000..5edad98408b --- /dev/null +++ b/text/0000-unchecked-lifetime.md @@ -0,0 +1,152 @@ +- Feature Name: (fill me in with a unique ident, `my_awesome_feature`) +- Start Date: (fill me in with today's date, YYYY-MM-DD) +- RFC PR: [rust-lang/rfcs#0000](https://github.com/rust-lang/rfcs/pull/0000) +- Rust Issue: [rust-lang/rust#0000](https://github.com/rust-lang/rust/issues/0000) + +# Summary +[summary]: #summary + +Introduce a new special lifetime representing an "unchecked" lifetime. calling a method whose signature is generic over any "unchecked" lifetime would require an unsafe operation. + +# Motivation +[motivation]: #motivation + +When creating self referential structs it is often preferred to use pointers over references because the conditions under which the pointer/reference is valid are not evaluated by the borrow checker. The problem with this general approach is that it does not scale well to more complex types. If we have a the following: +```rust +struct A { + item: T + borrower: B<'?> // we want the ref inside this to refer to item +} + +struct B<'a, T> { + actual_ref: &'a T +} +``` +there is no choice for a lifetime to replace `'?` with because `'static` may outlive `T` if it contains lifetimes, and we may not want to replace the ref inside `B` with a pointer, because `B` may have value apart from being stored in a self reference. + +# Guide-level explanation +[guide-level-explanation]: #guide-level-explanation + +There are situations where, when writing unsafe code, you may need to store a type without encoding its lifetime in the type system. The existence of raw pointers in the language is an acknowledgement of this need, but it is not always perfectly ergonomic to use pointers in this scenario. Consider the following self referential struct: +```rust +struct ArrayIter { + buffer: [T; 32] + iter: std::slice::Iter +} +``` +which we create so that `iter` is constructed from a slice of buffer. What should the lifetime parameter `???` be? There are traditionally three choices: +- introduce a new lifetime parameter +- replace all self references with pointers +- use `'static` and transmute our lifetime into it + +First, let's explain why none of these really work, then show the fourth option, proposed in this RFC. + +Introducing a new lifetime parameter has some problems: +```rust +struct ArrayIter<'a, T> { + buffer: [T; 32] + iter: std::slice::Iter<'a, T> +} +``` +while this can work to set your iter up and potentially implement methods on ArrayIter, 'a has no meaning to someone consuming this struct. what do they instantiate this lifetime as? there is not a scope to which this lifetime has any meaningful connection, so it really pollutes your type. + +Replacing all self references with pointers works, but not when you are not the implementor of the type which uses the lifetime. +```rust +struct ArrayIter { + buffer: [T; 32] + iter: MyPointerBasedIterType +} +``` +This approach is unreasonable for all but the simplest borrowing types, as it requires you to fully re-implement anything intended for use with references to work in terms of pointers. + +Using the `'static` lifetime almost works, but has one important failing: +```rust +struct ArrayIter { + buffer: [T; 32] + iter: std::slice::Iter<'static, T> +} +``` +What if T is not `'static`? using the static lifetime here restricts our generic parameter T to being 'static, which is a concession we may not be ok with making. + +So how do we get all of the above? We use the "unchecked lifetime" `'?` +```rust +struct ArrayIter { + buffer: [T; 32] + iter: std::slice::Iter<'?, T> +} +``` + +Note that, like `'static`, `'?` is allowed to appear in struct definitions without being declared. This is because the unchecked lifetime instructs the borrow checker to treat any references with this lifetime like raw pointers. This is very unsafe of course, so as a tradeoff, calling any function whose signature contains `'?` (for example the `next` method of `Iter`) requires an unsafe block. + +In general using replacing a real lifetime with `'?` should be thought of as a similar transformation to replacing a reference with a pointer. If you are doing it, you are doing it because safe rust does not allow for the type of code you are trying to write, and you're trying to encapsulate the unsafe into a compact part of your code. + +If you try to call a method whose arguments or return value include `'?`, that call will need to be wrapped in unsafe, because you are asserting that you know those references are valid despite the borrow checker not knowing. + + +# Reference-level explanation +[reference-level-explanation]: #reference-level-explanation + +This is the technical portion of the RFC. Explain the design in sufficient detail that: + +- Its interaction with other features is clear. +- It is reasonably clear how the feature would be implemented. +- Corner cases are dissected by example. + +The section should return to the examples given in the previous section, and explain more fully how the detailed proposal makes those examples work. + +# Drawbacks +[drawbacks]: #drawbacks + +Why should we *not* do this? + +# Rationale and alternatives +[rationale-and-alternatives]: #rationale-and-alternatives + +- Why is this design the best in the space of possible designs? +- What other designs have been considered and what is the rationale for not choosing them? +- What is the impact of not doing this? + +# Prior art +[prior-art]: #prior-art + +Discuss prior art, both the good and the bad, in relation to this proposal. +A few examples of what this can include are: + +- For language, library, cargo, tools, and compiler proposals: Does this feature exist in other programming languages and what experience have their community had? +- For community proposals: Is this done by some other community and what were their experiences with it? +- For other teams: What lessons can we learn from what other communities have done here? +- Papers: Are there any published papers or great posts that discuss this? If you have some relevant papers to refer to, this can serve as a more detailed theoretical background. + +This section is intended to encourage you as an author to think about the lessons from other languages, provide readers of your RFC with a fuller picture. +If there is no prior art, that is fine - your ideas are interesting to us whether they are brand new or if it is an adaptation from other languages. + +Note that while precedent set by other languages is some motivation, it does not on its own motivate an RFC. +Please also take into consideration that rust sometimes intentionally diverges from common language features. + +# Unresolved questions +[unresolved-questions]: #unresolved-questions + +- What parts of the design do you expect to resolve through the RFC process before this gets merged? +- What parts of the design do you expect to resolve through the implementation of this feature before stabilization? +- What related issues do you consider out of scope for this RFC that could be addressed in the future independently of the solution that comes out of this RFC? + +# Future possibilities +[future-possibilities]: #future-possibilities + +Think about what the natural extension and evolution of your proposal would +be and how it would affect the language and project as a whole in a holistic +way. Try to use this section as a tool to more fully consider all possible +interactions with the project and language in your proposal. +Also consider how this all fits into the roadmap for the project +and of the relevant sub-team. + +This is also a good place to "dump ideas", if they are out of scope for the +RFC you are writing but otherwise related. + +If you have tried and cannot think of any future possibilities, +you may simply state that you cannot think of anything. + +Note that having something written down in the future-possibilities section +is not a reason to accept the current or a future RFC; such notes should be +in the section on motivation or rationale in this or subsequent RFCs. +The section merely provides additional information. From 7a907f95071221687dd5b2ae8aba8ccee054d936 Mon Sep 17 00:00:00 2001 From: Mason Boeman Date: Wed, 24 Nov 2021 18:59:16 -0600 Subject: [PATCH 02/15] progress --- text/0000-unchecked-lifetime.md | 23 ++++++++--------------- 1 file changed, 8 insertions(+), 15 deletions(-) diff --git a/text/0000-unchecked-lifetime.md b/text/0000-unchecked-lifetime.md index 5edad98408b..044699c0c68 100644 --- a/text/0000-unchecked-lifetime.md +++ b/text/0000-unchecked-lifetime.md @@ -6,7 +6,7 @@ # Summary [summary]: #summary -Introduce a new special lifetime representing an "unchecked" lifetime. calling a method whose signature is generic over any "unchecked" lifetime would require an unsafe operation. +Introduce a new special lifetime `'?` representing an "unchecked" lifetime. calling a method whose signature is generic over any "unchecked" lifetime would require an unsafe operation. # Motivation [motivation]: #motivation @@ -82,6 +82,7 @@ In general using replacing a real lifetime with `'?` should be thought of as a s If you try to call a method whose arguments or return value include `'?`, that call will need to be wrapped in unsafe, because you are asserting that you know those references are valid despite the borrow checker not knowing. +The addition of the `'?` lifetime also means the addition of two new reference types, `&'? T` and `&'? mut T`. These are in a sense halfway in between references and pointers. Static references can be coerced into mortal references, which can be coerced into unchecked-lifetime references, which can be coerced into raw pointers. The crucial difference between `&'? T` and `*const T` is that it is considered unsound for `&'? T` to be unaligned at any time, instead of only at the time of dereference for raw pointers. # Reference-level explanation [reference-level-explanation]: #reference-level-explanation @@ -97,14 +98,14 @@ The section should return to the examples given in the previous section, and exp # Drawbacks [drawbacks]: #drawbacks -Why should we *not* do this? +- One more pointer type is potentially confusing. +- Potentially a trap for newer rust developers to just declare structs with unchecked lifetimes without realizing this is just as dangerous as throwing raw pointers around. # Rationale and alternatives [rationale-and-alternatives]: #rationale-and-alternatives -- Why is this design the best in the space of possible designs? -- What other designs have been considered and what is the rationale for not choosing them? -- What is the impact of not doing this? +- There isn't any analogue in the higher type system to the lifetime erasure that occurs when coalescing from reference to pointer. +- An alternative could be a macro-based library. This approach seems unlikely to check all the boxes though # Prior art [prior-art]: #prior-art @@ -112,16 +113,8 @@ Why should we *not* do this? Discuss prior art, both the good and the bad, in relation to this proposal. A few examples of what this can include are: -- For language, library, cargo, tools, and compiler proposals: Does this feature exist in other programming languages and what experience have their community had? -- For community proposals: Is this done by some other community and what were their experiences with it? -- For other teams: What lessons can we learn from what other communities have done here? -- Papers: Are there any published papers or great posts that discuss this? If you have some relevant papers to refer to, this can serve as a more detailed theoretical background. - -This section is intended to encourage you as an author to think about the lessons from other languages, provide readers of your RFC with a fuller picture. -If there is no prior art, that is fine - your ideas are interesting to us whether they are brand new or if it is an adaptation from other languages. - -Note that while precedent set by other languages is some motivation, it does not on its own motivate an RFC. -Please also take into consideration that rust sometimes intentionally diverges from common language features. +- [Oroboros](https://docs.rs/ouroboros/0.13.0/ouroboros/attr.self_referencing.html) is a crate designed to facilitate self referential struct construction via macros. It is limited to references, and attempts to avoid unsafe. +- [rental](https://docs.rs/rental/0.5.6/rental/), another macro based approach, is limited in a few ways and is somewhat clunky to use. # Unresolved questions [unresolved-questions]: #unresolved-questions From 8e4b4e3186c36a8fa09b726f84ed8d3f7aee4b56 Mon Sep 17 00:00:00 2001 From: Mason Boeman Date: Wed, 24 Nov 2021 20:35:40 -0600 Subject: [PATCH 03/15] flesh out some of the reference-level-explanation --- text/0000-unchecked-lifetime.md | 74 ++++++++++++++++++--------------- 1 file changed, 41 insertions(+), 33 deletions(-) diff --git a/text/0000-unchecked-lifetime.md b/text/0000-unchecked-lifetime.md index 044699c0c68..612aefc577b 100644 --- a/text/0000-unchecked-lifetime.md +++ b/text/0000-unchecked-lifetime.md @@ -6,7 +6,7 @@ # Summary [summary]: #summary -Introduce a new special lifetime `'?` representing an "unchecked" lifetime. calling a method whose signature is generic over any "unchecked" lifetime would require an unsafe operation. +Introduce a new special lifetime `'?` representing an "unchecked" lifetime which is outlived by all other lifetimes. Most uses of types with this lifetime are then unsafe, as detailed in this RFC. # Motivation [motivation]: #motivation @@ -15,14 +15,14 @@ When creating self referential structs it is often preferred to use pointers ove ```rust struct A { item: T - borrower: B<'?> // we want the ref inside this to refer to item + borrower: B // we want the ref inside this to refer to item } struct B<'a, T> { actual_ref: &'a T } ``` -there is no choice for a lifetime to replace `'?` with because `'static` may outlive `T` if it contains lifetimes, and we may not want to replace the ref inside `B` with a pointer, because `B` may have value apart from being stored in a self reference. +there is no choice for a lifetime to replace `???` with because `'static` may outlive `T` if it contains lifetimes, and we may not want to replace the ref inside `B` with a pointer, because `B` may have value apart from being stored in a self reference. # Guide-level explanation [guide-level-explanation]: #guide-level-explanation @@ -76,24 +76,53 @@ struct ArrayIter { } ``` -Note that, like `'static`, `'?` is allowed to appear in struct definitions without being declared. This is because the unchecked lifetime instructs the borrow checker to treat any references with this lifetime like raw pointers. This is very unsafe of course, so as a tradeoff, calling any function whose signature contains `'?` (for example the `next` method of `Iter`) requires an unsafe block. +Note that, like `'static`, `'?` is allowed to appear in struct definitions without being declared. This is because the unchecked lifetime instructs the borrow checker to treat any references with this lifetime like raw pointers. This is very unsafe of course, so as a tradeoff, dereferencing a reference with the unchecked lifetime is unsafe, and `'?` is not acceptable when `'a` is expected. In general using replacing a real lifetime with `'?` should be thought of as a similar transformation to replacing a reference with a pointer. If you are doing it, you are doing it because safe rust does not allow for the type of code you are trying to write, and you're trying to encapsulate the unsafe into a compact part of your code. If you try to call a method whose arguments or return value include `'?`, that call will need to be wrapped in unsafe, because you are asserting that you know those references are valid despite the borrow checker not knowing. -The addition of the `'?` lifetime also means the addition of two new reference types, `&'? T` and `&'? mut T`. These are in a sense halfway in between references and pointers. Static references can be coerced into mortal references, which can be coerced into unchecked-lifetime references, which can be coerced into raw pointers. The crucial difference between `&'? T` and `*const T` is that it is considered unsound for `&'? T` to be unaligned at any time, instead of only at the time of dereference for raw pointers. +The addition of the `'?` lifetime also means the addition of two new reference types, `&'? T` and `&'? mut T`. These are in a sense halfway in between references and pointers. Dereferencing them is unsafe. Static references can be coerced into normal references, which can be coerced into unchecked-lifetime references, which can be coerced into raw pointers. The crucial difference between `&'? T` and `*const T` is that it is considered unsound for `&'? T` to be unaligned at any time, instead of only at the time of dereference for raw pointers. # Reference-level explanation [reference-level-explanation]: #reference-level-explanation -This is the technical portion of the RFC. Explain the design in sufficient detail that: +The unchecked lifetime is unique in a few ways: +- It is outlived by all other lifetimes (in the same way that `'static` outlives all other lifetimes). +- The check that a reference does not outlive borrowed content is skipped when the borrowed content has `'?` lifetime. (This is the check that goes *unchecked*). +- It cannot be used where a normal reference is expected. +- assigning to or reading from `&'? T` and `&'? mut T` is unsafe -- Its interaction with other features is clear. -- It is reasonably clear how the feature would be implemented. -- Corner cases are dissected by example. +Let's examine some of the implications of these features. -The section should return to the examples given in the previous section, and explain more fully how the detailed proposal makes those examples work. +It is safe in general to store into a value expecting `'?`, just like with raw pointers + +```rust +struct A { + val: &'? usize +} + +impl A { + fn store(&self, some_ref: &usize) { + self.val = some_ref; // assigning to val, not through val. + } +} +``` + +The only way to use a type that has been stored with `&'?` is to transmute it to a normal lifetime, which of course requires unsafe. +```rust +impl A { + // illegal + fn read(&self) -> usize { + *self.val // throws error due to rule 4 + } + + // legal, requiring unsafe. + fn read(&self) -> usize { + unsafe { *self.val } + } +} +``` # Drawbacks [drawbacks]: #drawbacks @@ -110,36 +139,15 @@ The section should return to the examples given in the previous section, and exp # Prior art [prior-art]: #prior-art -Discuss prior art, both the good and the bad, in relation to this proposal. -A few examples of what this can include are: - - [Oroboros](https://docs.rs/ouroboros/0.13.0/ouroboros/attr.self_referencing.html) is a crate designed to facilitate self referential struct construction via macros. It is limited to references, and attempts to avoid unsafe. - [rental](https://docs.rs/rental/0.5.6/rental/), another macro based approach, is limited in a few ways and is somewhat clunky to use. # Unresolved questions [unresolved-questions]: #unresolved-questions -- What parts of the design do you expect to resolve through the RFC process before this gets merged? -- What parts of the design do you expect to resolve through the implementation of this feature before stabilization? -- What related issues do you consider out of scope for this RFC that could be addressed in the future independently of the solution that comes out of this RFC? +- While this RFC does aim to make self referential structs more possible, it does not aim to make them common or even easy. Pinning is not addressed at all, and must be well understood for any self referential struct, and this RFC aims to be compatable with and separate from the pinning API. # Future possibilities [future-possibilities]: #future-possibilities -Think about what the natural extension and evolution of your proposal would -be and how it would affect the language and project as a whole in a holistic -way. Try to use this section as a tool to more fully consider all possible -interactions with the project and language in your proposal. -Also consider how this all fits into the roadmap for the project -and of the relevant sub-team. - -This is also a good place to "dump ideas", if they are out of scope for the -RFC you are writing but otherwise related. - -If you have tried and cannot think of any future possibilities, -you may simply state that you cannot think of anything. - -Note that having something written down in the future-possibilities section -is not a reason to accept the current or a future RFC; such notes should be -in the section on motivation or rationale in this or subsequent RFCs. -The section merely provides additional information. +This may be valuable to ffi if you know that a pointer is aligned, as then using `&'?` may be more appropriate in this scenario From 1f1f91b752256ab8140e7892b75d29d7b1d3ac54 Mon Sep 17 00:00:00 2001 From: Mason Boeman Date: Wed, 24 Nov 2021 20:37:53 -0600 Subject: [PATCH 04/15] add metadata --- text/0000-unchecked-lifetime.md | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/text/0000-unchecked-lifetime.md b/text/0000-unchecked-lifetime.md index 612aefc577b..5e1e5a0a0ec 100644 --- a/text/0000-unchecked-lifetime.md +++ b/text/0000-unchecked-lifetime.md @@ -1,6 +1,6 @@ -- Feature Name: (fill me in with a unique ident, `my_awesome_feature`) -- Start Date: (fill me in with today's date, YYYY-MM-DD) -- RFC PR: [rust-lang/rfcs#0000](https://github.com/rust-lang/rfcs/pull/0000) +- Feature Name: `unchecked_lifetime` +- Start Date: 2021-11-24 +- RFC PR: [rust-lang/rfcs#3199](https://github.com/rust-lang/rfcs/pull/3199) - Rust Issue: [rust-lang/rust#0000](https://github.com/rust-lang/rust/issues/0000) # Summary From edacab87a43b8d545e0b6f1d2d784369f79941db Mon Sep 17 00:00:00 2001 From: Mason Boeman Date: Wed, 24 Nov 2021 20:41:06 -0600 Subject: [PATCH 05/15] rename with pr number --- text/{0000-unchecked-lifetime.md => 3199-unchecked-lifetime.md} | 0 1 file changed, 0 insertions(+), 0 deletions(-) rename text/{0000-unchecked-lifetime.md => 3199-unchecked-lifetime.md} (100%) diff --git a/text/0000-unchecked-lifetime.md b/text/3199-unchecked-lifetime.md similarity index 100% rename from text/0000-unchecked-lifetime.md rename to text/3199-unchecked-lifetime.md From d55bd2aa18dc9874a97a90580e8560083834d002 Mon Sep 17 00:00:00 2001 From: Mason Boeman Date: Wed, 24 Nov 2021 21:03:06 -0600 Subject: [PATCH 06/15] more tweaks --- text/3199-unchecked-lifetime.md | 26 ++++++++++++++++++++++---- 1 file changed, 22 insertions(+), 4 deletions(-) diff --git a/text/3199-unchecked-lifetime.md b/text/3199-unchecked-lifetime.md index 5e1e5a0a0ec..a64b09e29e2 100644 --- a/text/3199-unchecked-lifetime.md +++ b/text/3199-unchecked-lifetime.md @@ -15,7 +15,7 @@ When creating self referential structs it is often preferred to use pointers ove ```rust struct A { item: T - borrower: B // we want the ref inside this to refer to item + borrower: B // we want the ref inside this to refer to item } struct B<'a, T> { @@ -108,19 +108,36 @@ impl A { } } ``` +An important but subtle point here is that it is only legal to have `&self` because of rule 2. Without it all references to A would outlive their borrowed content. -The only way to use a type that has been stored with `&'?` is to transmute it to a normal lifetime, which of course requires unsafe. +The only way to use a type that has been stored with `&'?` is to use unsafe, or transmute it to a normal lifetime, which of course requires unsafe. ```rust impl A { // illegal - fn read(&self) -> usize { + fn read_illegal(&self) -> usize { *self.val // throws error due to rule 4 } // legal, requiring unsafe. - fn read(&self) -> usize { + fn read_deref(&self) -> usize { unsafe { *self.val } } + + // legal, requiring unsafe. + fn read_transmute(&self) -> usize { + *(unsafe { core::mem::transmute::<&'? usize, &usize>(self.val) }) + } +} +``` +note that read_transmute works for any type which is generic over a lifetime. + +Additionally, an important result of rule 3 is that methods defined on a reference to a type cannot be called on an unchecked reference unless they were specifically written to accept an unchecked reference, as unchecked references do not live long enough to be used where a normal reference would, like the method signature. +```rust +impl A { + // errors + fn get(&self) -> usize { + self.val.clone() // clone expects &self, so this errors as val does not live long enough. + } } ``` @@ -145,6 +162,7 @@ impl A { # Unresolved questions [unresolved-questions]: #unresolved-questions +- The choice for what to use as the lifetime is up in the air. Another idea I had for it was `'*` to imply the relationship to raw pointers, but then it gets a little insane with `&'* T` vs `&T` vs `*T`. - While this RFC does aim to make self referential structs more possible, it does not aim to make them common or even easy. Pinning is not addressed at all, and must be well understood for any self referential struct, and this RFC aims to be compatable with and separate from the pinning API. # Future possibilities From 3c7a572c2add600058d5e8bbb87e92ae1657ec16 Mon Sep 17 00:00:00 2001 From: Mason Date: Thu, 25 Nov 2021 09:25:05 -0600 Subject: [PATCH 07/15] add missing mut Co-authored-by: Jacob Lifshay --- text/3199-unchecked-lifetime.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/text/3199-unchecked-lifetime.md b/text/3199-unchecked-lifetime.md index a64b09e29e2..cff469c0481 100644 --- a/text/3199-unchecked-lifetime.md +++ b/text/3199-unchecked-lifetime.md @@ -103,12 +103,12 @@ struct A { } impl A { - fn store(&self, some_ref: &usize) { + fn store(&mut self, some_ref: &usize) { self.val = some_ref; // assigning to val, not through val. } } ``` -An important but subtle point here is that it is only legal to have `&self` because of rule 2. Without it all references to A would outlive their borrowed content. +An important but subtle point here is that it is only legal to have `&mut self` because of rule 2. Without it all references to A would outlive their borrowed content. The only way to use a type that has been stored with `&'?` is to use unsafe, or transmute it to a normal lifetime, which of course requires unsafe. ```rust From 387a681f3003e7bee1aaacf14a7120aba44dcdf1 Mon Sep 17 00:00:00 2001 From: Mason Boeman Date: Thu, 25 Nov 2021 10:24:11 -0600 Subject: [PATCH 08/15] significant updates, switch to 'unsafe syntax --- text/3199-unchecked-lifetime.md | 171 ------------------------------ text/3199-unsafe-lifetime.md | 180 ++++++++++++++++++++++++++++++++ 2 files changed, 180 insertions(+), 171 deletions(-) delete mode 100644 text/3199-unchecked-lifetime.md create mode 100644 text/3199-unsafe-lifetime.md diff --git a/text/3199-unchecked-lifetime.md b/text/3199-unchecked-lifetime.md deleted file mode 100644 index cff469c0481..00000000000 --- a/text/3199-unchecked-lifetime.md +++ /dev/null @@ -1,171 +0,0 @@ -- Feature Name: `unchecked_lifetime` -- Start Date: 2021-11-24 -- RFC PR: [rust-lang/rfcs#3199](https://github.com/rust-lang/rfcs/pull/3199) -- Rust Issue: [rust-lang/rust#0000](https://github.com/rust-lang/rust/issues/0000) - -# Summary -[summary]: #summary - -Introduce a new special lifetime `'?` representing an "unchecked" lifetime which is outlived by all other lifetimes. Most uses of types with this lifetime are then unsafe, as detailed in this RFC. - -# Motivation -[motivation]: #motivation - -When creating self referential structs it is often preferred to use pointers over references because the conditions under which the pointer/reference is valid are not evaluated by the borrow checker. The problem with this general approach is that it does not scale well to more complex types. If we have a the following: -```rust -struct A { - item: T - borrower: B // we want the ref inside this to refer to item -} - -struct B<'a, T> { - actual_ref: &'a T -} -``` -there is no choice for a lifetime to replace `???` with because `'static` may outlive `T` if it contains lifetimes, and we may not want to replace the ref inside `B` with a pointer, because `B` may have value apart from being stored in a self reference. - -# Guide-level explanation -[guide-level-explanation]: #guide-level-explanation - -There are situations where, when writing unsafe code, you may need to store a type without encoding its lifetime in the type system. The existence of raw pointers in the language is an acknowledgement of this need, but it is not always perfectly ergonomic to use pointers in this scenario. Consider the following self referential struct: -```rust -struct ArrayIter { - buffer: [T; 32] - iter: std::slice::Iter -} -``` -which we create so that `iter` is constructed from a slice of buffer. What should the lifetime parameter `???` be? There are traditionally three choices: -- introduce a new lifetime parameter -- replace all self references with pointers -- use `'static` and transmute our lifetime into it - -First, let's explain why none of these really work, then show the fourth option, proposed in this RFC. - -Introducing a new lifetime parameter has some problems: -```rust -struct ArrayIter<'a, T> { - buffer: [T; 32] - iter: std::slice::Iter<'a, T> -} -``` -while this can work to set your iter up and potentially implement methods on ArrayIter, 'a has no meaning to someone consuming this struct. what do they instantiate this lifetime as? there is not a scope to which this lifetime has any meaningful connection, so it really pollutes your type. - -Replacing all self references with pointers works, but not when you are not the implementor of the type which uses the lifetime. -```rust -struct ArrayIter { - buffer: [T; 32] - iter: MyPointerBasedIterType -} -``` -This approach is unreasonable for all but the simplest borrowing types, as it requires you to fully re-implement anything intended for use with references to work in terms of pointers. - -Using the `'static` lifetime almost works, but has one important failing: -```rust -struct ArrayIter { - buffer: [T; 32] - iter: std::slice::Iter<'static, T> -} -``` -What if T is not `'static`? using the static lifetime here restricts our generic parameter T to being 'static, which is a concession we may not be ok with making. - -So how do we get all of the above? We use the "unchecked lifetime" `'?` -```rust -struct ArrayIter { - buffer: [T; 32] - iter: std::slice::Iter<'?, T> -} -``` - -Note that, like `'static`, `'?` is allowed to appear in struct definitions without being declared. This is because the unchecked lifetime instructs the borrow checker to treat any references with this lifetime like raw pointers. This is very unsafe of course, so as a tradeoff, dereferencing a reference with the unchecked lifetime is unsafe, and `'?` is not acceptable when `'a` is expected. - -In general using replacing a real lifetime with `'?` should be thought of as a similar transformation to replacing a reference with a pointer. If you are doing it, you are doing it because safe rust does not allow for the type of code you are trying to write, and you're trying to encapsulate the unsafe into a compact part of your code. - -If you try to call a method whose arguments or return value include `'?`, that call will need to be wrapped in unsafe, because you are asserting that you know those references are valid despite the borrow checker not knowing. - -The addition of the `'?` lifetime also means the addition of two new reference types, `&'? T` and `&'? mut T`. These are in a sense halfway in between references and pointers. Dereferencing them is unsafe. Static references can be coerced into normal references, which can be coerced into unchecked-lifetime references, which can be coerced into raw pointers. The crucial difference between `&'? T` and `*const T` is that it is considered unsound for `&'? T` to be unaligned at any time, instead of only at the time of dereference for raw pointers. - -# Reference-level explanation -[reference-level-explanation]: #reference-level-explanation - -The unchecked lifetime is unique in a few ways: -- It is outlived by all other lifetimes (in the same way that `'static` outlives all other lifetimes). -- The check that a reference does not outlive borrowed content is skipped when the borrowed content has `'?` lifetime. (This is the check that goes *unchecked*). -- It cannot be used where a normal reference is expected. -- assigning to or reading from `&'? T` and `&'? mut T` is unsafe - -Let's examine some of the implications of these features. - -It is safe in general to store into a value expecting `'?`, just like with raw pointers - -```rust -struct A { - val: &'? usize -} - -impl A { - fn store(&mut self, some_ref: &usize) { - self.val = some_ref; // assigning to val, not through val. - } -} -``` -An important but subtle point here is that it is only legal to have `&mut self` because of rule 2. Without it all references to A would outlive their borrowed content. - -The only way to use a type that has been stored with `&'?` is to use unsafe, or transmute it to a normal lifetime, which of course requires unsafe. -```rust -impl A { - // illegal - fn read_illegal(&self) -> usize { - *self.val // throws error due to rule 4 - } - - // legal, requiring unsafe. - fn read_deref(&self) -> usize { - unsafe { *self.val } - } - - // legal, requiring unsafe. - fn read_transmute(&self) -> usize { - *(unsafe { core::mem::transmute::<&'? usize, &usize>(self.val) }) - } -} -``` -note that read_transmute works for any type which is generic over a lifetime. - -Additionally, an important result of rule 3 is that methods defined on a reference to a type cannot be called on an unchecked reference unless they were specifically written to accept an unchecked reference, as unchecked references do not live long enough to be used where a normal reference would, like the method signature. -```rust -impl A { - // errors - fn get(&self) -> usize { - self.val.clone() // clone expects &self, so this errors as val does not live long enough. - } -} -``` - -# Drawbacks -[drawbacks]: #drawbacks - -- One more pointer type is potentially confusing. -- Potentially a trap for newer rust developers to just declare structs with unchecked lifetimes without realizing this is just as dangerous as throwing raw pointers around. - -# Rationale and alternatives -[rationale-and-alternatives]: #rationale-and-alternatives - -- There isn't any analogue in the higher type system to the lifetime erasure that occurs when coalescing from reference to pointer. -- An alternative could be a macro-based library. This approach seems unlikely to check all the boxes though - -# Prior art -[prior-art]: #prior-art - -- [Oroboros](https://docs.rs/ouroboros/0.13.0/ouroboros/attr.self_referencing.html) is a crate designed to facilitate self referential struct construction via macros. It is limited to references, and attempts to avoid unsafe. -- [rental](https://docs.rs/rental/0.5.6/rental/), another macro based approach, is limited in a few ways and is somewhat clunky to use. - -# Unresolved questions -[unresolved-questions]: #unresolved-questions - -- The choice for what to use as the lifetime is up in the air. Another idea I had for it was `'*` to imply the relationship to raw pointers, but then it gets a little insane with `&'* T` vs `&T` vs `*T`. -- While this RFC does aim to make self referential structs more possible, it does not aim to make them common or even easy. Pinning is not addressed at all, and must be well understood for any self referential struct, and this RFC aims to be compatable with and separate from the pinning API. - -# Future possibilities -[future-possibilities]: #future-possibilities - -This may be valuable to ffi if you know that a pointer is aligned, as then using `&'?` may be more appropriate in this scenario diff --git a/text/3199-unsafe-lifetime.md b/text/3199-unsafe-lifetime.md new file mode 100644 index 00000000000..749aee6ab49 --- /dev/null +++ b/text/3199-unsafe-lifetime.md @@ -0,0 +1,180 @@ +- Feature Name: `unsafe_lifetime` +- Start Date: 2021-11-24 +- RFC PR: [rust-lang/rfcs#3199](https://github.com/rust-lang/rfcs/pull/3199) +- Rust Issue: [rust-lang/rust#0000](https://github.com/rust-lang/rust/issues/0000) + +# Summary +[summary]: #summary + +Introduce a new special lifetime `'unsafe` which is outlived by all other lifetimes. Using a type through a 'unsafe reference, or which is instantiated with an 'unsafe lifetime parameter is rarely possible without unsafe. + +# Motivation +[motivation]: #motivation + +When creating self-referential structs it is often preferred to use pointers over references because the conditions under which the pointer/reference is valid are not evaluated by the borrow checker. The problem with this general approach is that it does not scale well to more complex types. If we have a the following: +```rust +struct A { + item: T + borrower: B<'?, T> // we want the ref inside this to refer to item +} + +struct B<'a, T> { + actual_ref: &'a T +} +``` +there is no choice for a lifetime to replace `'?` with because `'static` may outlive `T` if it contains lifetimes, and we may not want to replace the ref inside `B` with a pointer, because `B` may have value apart from being stored in a self-reference. + +# Guide-level explanation +[guide-level-explanation]: #guide-level-explanation + +There are situations where, when writing unsafe code, you may need to store a type without encoding its lifetime in the type system. The existence of raw pointers in the language is an acknowledgement of this need, but it is not always perfectly ergonomic to use pointers in this scenario. Consider the following self-referential struct: +```rust +struct ArrayIter { + buffer: [T; 32] + iter: std::slice::Iter<'?, T> +} +``` +which we create so that `iter` is constructed from a slice of buffer. What should the lifetime parameter `'?` be? There are traditionally three choices: +- introduce a new lifetime parameter +- replace all self-references with pointers +- use `'static` and transmute our lifetime into it + +First, let's explain why none of these really work, then show the fourth option, proposed in this RFC. + +Introducing a new lifetime parameter has some problems: +```rust +struct ArrayIter<'a, T> { + buffer: [T; 32] + iter: std::slice::Iter<'a, T> +} +``` +while this can work to set your iter up and potentially implement methods on ArrayIter, 'a has no meaning to someone consuming this struct. what do they instantiate this lifetime as? there is not a scope to which this lifetime has any meaningful connection, so it really pollutes your type. + +Replacing all self-references with pointers works, but not when you are not the implementor of the type which uses the lifetime. +```rust +struct ArrayIter { + buffer: [T; 32] + iter: MyPointerBasedReimplementationOfIter +} +``` +This approach is unreasonable for all but the simplest borrowing types, as it requires you to fully re-implement anything intended for use with references to work in terms of pointers. + +Using the `'static` lifetime almost works, but has one important failing: +```rust +struct ArrayIter { + buffer: [T; 32] + iter: std::slice::Iter<'static, T> +} +``` +What if T is not `'static`? using the static lifetime here restricts our generic parameter T to being 'static, which is a concession we may not be ok with making. + +So how do we get all of the above? We use the unsafe lifetime `'unsafe` +```rust +struct ArrayIter { + buffer: [T; 32] + iter: std::slice::Iter<'unsafe, T> +} +``` + +Note that, like `'static`, `'unsafe` is allowed to appear in struct definitions without being declared. This is because the unsafe lifetime, like`'static`, is a specific lifetime and not a generic parameter.`'unsafe` can be thought of as "a lifetime which is outlived by all possible lifetimes. Dereferencing a reference with the unsafe lifetime is unsafe. Additionally, `'unsafe` is not acceptable when `'a` is expected because for any lifetime`'a`, `'unsafe` does not live long enough to satisfy its requirements. + +In general using replacing a real lifetime with `'unsafe` should be thought of as a similar transformation to replacing a reference with a pointer. If you are doing it, you are doing it because safe rust does not allow for the type of code you are trying to write, and you're trying to encapsulate the unsafe into a compact part of your code. + +If you try to call a function which has a lifetime specifier (whether or not it has been elided in the signature) using a `'unsafe` lifetime you will get an error, because any lifetime that the borrow checker could possibly choose for this call to your function would outlive a `'unsafe` reference. + +The addition of the `'unsafe` lifetime also means the addition of two new reference types, `&'unsafe T` and `&'unsafe mut T`. These are in a sense halfway in between references and pointers. Dereferencing them is unsafe. `'static` references can be coerced into `'a` references, which can be coerced into `'unsafe` references, which can be coerced into raw pointers. + +# Reference-level explanation +[reference-level-explanation]: #reference-level-explanation + +The unsafe lifetime is unique in a few ways: +1. It is outlived by all other lifetimes (in the same way that `'static` outlives all other lifetimes). +2. The check that a reference does not outlive borrowed content is skipped when the borrowed content has `'unsafe` lifetime. +3. assigning to or reading from `&'? T` and `&'? mut T` is unsafe + +One important consequence of rule 1, which we will call rule 1.5: `'unsafe` references cannot be used where normal references are expected, because the borrow checker must negotiate a lifetime between the two, but any lifetime that can be assigned to `'a` necessarily outlives `'unsafe`. + +Let's examine some of the implications of these features. + +It is safe in general to store into a value expecting `'unsafe`, just like with raw pointers + +```rust +struct A { + val: &'unsafe usize +} + +impl A { + fn store(&mut self, some_ref: &usize) { + self.val = some_ref; // assigning to val, not through val. + } +} +``` +An important but subtle point here is that it is only legal to have `&mut self` because of rule 2. Without it all references to A would outlive their borrowed content. + +The only way to use a type that has been stored with `&'unsafe` is to use unsafe, or transmute it to a normal lifetime, which of course requires unsafe. +```rust +impl A { + // illegal + fn read_illegal(&self) -> usize { + *self.val // throws error due to rule 4 + } + + // legal, requiring unsafe. + fn read_deref(&self) -> usize { + unsafe { *self.val } + } + + // legal, requiring unsafe. + fn read_transmute(&self) -> usize { + *(unsafe { core::mem::transmute::<&'? usize, &usize>(self.val) }) + } +} +``` +note that read_transmute works for any type which is generic over a lifetime. + +Additionally, an important result of rule 1.5 is that methods defined on a reference to a type cannot be called on an unsafe reference unless they were specifically written to accept an unsafe reference, as unsafe references do not live long enough to be used where a normal reference would, like the method signature. +```rust +impl A { + // errors + fn get(&self) -> usize { + self.val.clone() // clone expects &self, so this errors as val does not live long enough. + } +} +``` + +# Drawbacks +[drawbacks]: #drawbacks + +- One more pointer type is potentially confusing. +- Potentially a trap for newer rust developers to just declare structs with unsafe lifetimes without realizing this is just as dangerous as throwing raw pointers around. + +# Rationale and alternatives +[rationale-and-alternatives]: #rationale-and-alternatives + +- There isn't any analogue in the higher type system to the lifetime erasure that occurs when coalescing from reference to pointer. +- An alternative could be a macro-based library. This approach seems unlikely to check all the boxes though + +# Prior art +[prior-art]: #prior-art + + + +Some crates for dealing with self reference: +- [owning_ref](https://crates.io/crates/owning_ref) is an early attempt to make self references ergonomic but is slightly clunky and not generic enough for some use cases. +- [Oroboros](https://docs.rs/ouroboros/0.13.0/ouroboros/attr.self_referencing.html) is a crate designed to facilitate self-referential struct construction via macros. It is limited to references, and attempts to avoid unsafe. +- [rental](https://docs.rs/rental/0.5.6/rental/), another macro based approach, is limited in a few ways and is somewhat clunky to use. + +# Unresolved questions +[unresolved-questions]: #unresolved-questions + +- What about Drop? it is clearly problematic to call drop on a type like this, as the mutable reference taken in the drop impl outlives the `'unsafe` lifetime. I have a couple possible approaches on this front: + 1. Any type which explicitly contains a `'unsafe` lifetime specifier *must* impl Drop. + - If a type implements drop for the `'unsafe` lifetime instead of simply a declared lifetime, uses of it can be exempt from the above requirement. + 2. Call drop on downgrade from a `'a` to `'unsafe` unless some special unsafe downgrade function is called, or drop is implemented for `'unsafe` of that type. This may be clunky but I think it is the cleanest in terms of maintaining the boundary between safe and unsafe. +- The choice for what to use as the lifetime is up in the air. Another idea I had for it was `'*` to imply the relationship to raw pointers, but then it gets a little insane with `&'* T` vs `&T` vs `*T`. +- While this RFC does aim to make self-referential structs more possible, it does not aim to make them common or even easy. Pinning is not addressed at all, and must be well understood for any self-referential struct, and this RFC aims to be compatable with and separate from the pinning API. + +# Future possibilities +[future-possibilities]: #future-possibilities + +This may be valuable to ffi if you know that a pointer is aligned, as then using `&'?` may be more appropriate in this scenario From 46bd02314a13a8b0aef469728e14635a9e7213d3 Mon Sep 17 00:00:00 2001 From: Mason Boeman Date: Thu, 25 Nov 2021 10:39:55 -0600 Subject: [PATCH 09/15] prior art --- text/3199-unsafe-lifetime.md | 6 +++++- 1 file changed, 5 insertions(+), 1 deletion(-) diff --git a/text/3199-unsafe-lifetime.md b/text/3199-unsafe-lifetime.md index 749aee6ab49..f8350ea1e7f 100644 --- a/text/3199-unsafe-lifetime.md +++ b/text/3199-unsafe-lifetime.md @@ -157,7 +157,11 @@ impl A { # Prior art [prior-art]: #prior-art - +previous rfc: +[unsafe lifetime by jpernst](https://github.com/rust-lang/rfcs/pull/1918/) +Many things are similar between these two RFCs, but there are a couple important differences that solve the problems with the first unsafe lifetime attempt: +- Instead of `'unsafe` satisfying *all* constraints, it satisfies *no* constraints. This makes it so existing functions cannot be called with unsafe lifetimes. +- operations which create values whose types contain `'unsafe` are generally all safe. These values are mostly unusable unless they are transmuted back to a useful lifetime, which is unsafe. Some crates for dealing with self reference: - [owning_ref](https://crates.io/crates/owning_ref) is an early attempt to make self references ergonomic but is slightly clunky and not generic enough for some use cases. From 765ccf56ae5edb737fa1bb15e10215110a8cb4d5 Mon Sep 17 00:00:00 2001 From: Mason Boeman Date: Wed, 1 Dec 2021 19:35:35 -0600 Subject: [PATCH 10/15] tweak semantics to improve Drop behavior --- text/3199-unsafe-lifetime.md | 27 ++++++++++++++------------- 1 file changed, 14 insertions(+), 13 deletions(-) diff --git a/text/3199-unsafe-lifetime.md b/text/3199-unsafe-lifetime.md index f8350ea1e7f..ae753893237 100644 --- a/text/3199-unsafe-lifetime.md +++ b/text/3199-unsafe-lifetime.md @@ -78,19 +78,25 @@ struct ArrayIter { Note that, like `'static`, `'unsafe` is allowed to appear in struct definitions without being declared. This is because the unsafe lifetime, like`'static`, is a specific lifetime and not a generic parameter.`'unsafe` can be thought of as "a lifetime which is outlived by all possible lifetimes. Dereferencing a reference with the unsafe lifetime is unsafe. Additionally, `'unsafe` is not acceptable when `'a` is expected because for any lifetime`'a`, `'unsafe` does not live long enough to satisfy its requirements. -In general using replacing a real lifetime with `'unsafe` should be thought of as a similar transformation to replacing a reference with a pointer. If you are doing it, you are doing it because safe rust does not allow for the type of code you are trying to write, and you're trying to encapsulate the unsafe into a compact part of your code. +In general replacing a real lifetime with `'unsafe` should be thought of as a similar transformation to replacing a reference with a pointer. If you are doing it, you are doing it because safe rust does not allow for the type of code you are trying to write, and you're trying to encapsulate the unsafe into a compact part of your code. -If you try to call a function which has a lifetime specifier (whether or not it has been elided in the signature) using a `'unsafe` lifetime you will get an error, because any lifetime that the borrow checker could possibly choose for this call to your function would outlive a `'unsafe` reference. +If you try to call a function which has a lifetime specifier (whether or not it has been elided in the signature) using a `'unsafe` lifetime you will get a compilation error, because any lifetime that the borrow checker could possibly choose for this call to your function would outlive a `'unsafe` reference. The addition of the `'unsafe` lifetime also means the addition of two new reference types, `&'unsafe T` and `&'unsafe mut T`. These are in a sense halfway in between references and pointers. Dereferencing them is unsafe. `'static` references can be coerced into `'a` references, which can be coerced into `'unsafe` references, which can be coerced into raw pointers. +Now we also need to be able to actually produce a value to assign to our `std::slice::Iter<'unsafe, T>` field, which can be done by assigning to a value of the same type, except with a normal lifetime in place of unsafe, though there's an important catch. Any assignment to a "place" (whether to a variable, through a mutable reference, or setting a field of a struct) is unsafe if the type of the "place" uses 'unsafe instead of a generic lifetime specifier. The user must guarantee that the lifetime replaced by 'unsafe is valid when the value contained in place is dropped. + # Reference-level explanation [reference-level-explanation]: #reference-level-explanation The unsafe lifetime is unique in a few ways: 1. It is outlived by all other lifetimes (in the same way that `'static` outlives all other lifetimes). 2. The check that a reference does not outlive borrowed content is skipped when the borrowed content has `'unsafe` lifetime. -3. assigning to or reading from `&'? T` and `&'? mut T` is unsafe +3. Assigning to or reading from `&'unsafe T` and `&'unsafe mut T` is unsafe. + - The programmer must ensure the references must be valid, or this is UB. +4. Assigning to a variable whose type is instantiated with `'unsafe` in place of at least one of its lifetimes is unsafe. + - The programmer must ensure the `'unsafe` is valid when the variable is dropped (because step 5). +5. Dropping a value whose type is instantiated with `'unsafe` first transmutes the value to the same type with `'unsafe` replaced with `'_`. One important consequence of rule 1, which we will call rule 1.5: `'unsafe` references cannot be used where normal references are expected, because the borrow checker must negotiate a lifetime between the two, but any lifetime that can be assigned to `'a` necessarily outlives `'unsafe`. @@ -105,7 +111,9 @@ struct A { impl A { fn store(&mut self, some_ref: &usize) { - self.val = some_ref; // assigning to val, not through val. + // assigning to val, not through val. + // unsafe due to rule 4. If we were assigning through it would be rule 3. + unsafe { self.val = some_ref; } } } ``` @@ -126,12 +134,10 @@ impl A { // legal, requiring unsafe. fn read_transmute(&self) -> usize { - *(unsafe { core::mem::transmute::<&'? usize, &usize>(self.val) }) + *(unsafe { core::mem::transmute::<&'unsafe usize, &usize>(self.val) }) } } ``` -note that read_transmute works for any type which is generic over a lifetime. - Additionally, an important result of rule 1.5 is that methods defined on a reference to a type cannot be called on an unsafe reference unless they were specifically written to accept an unsafe reference, as unsafe references do not live long enough to be used where a normal reference would, like the method signature. ```rust impl A { @@ -171,14 +177,9 @@ Some crates for dealing with self reference: # Unresolved questions [unresolved-questions]: #unresolved-questions -- What about Drop? it is clearly problematic to call drop on a type like this, as the mutable reference taken in the drop impl outlives the `'unsafe` lifetime. I have a couple possible approaches on this front: - 1. Any type which explicitly contains a `'unsafe` lifetime specifier *must* impl Drop. - - If a type implements drop for the `'unsafe` lifetime instead of simply a declared lifetime, uses of it can be exempt from the above requirement. - 2. Call drop on downgrade from a `'a` to `'unsafe` unless some special unsafe downgrade function is called, or drop is implemented for `'unsafe` of that type. This may be clunky but I think it is the cleanest in terms of maintaining the boundary between safe and unsafe. -- The choice for what to use as the lifetime is up in the air. Another idea I had for it was `'*` to imply the relationship to raw pointers, but then it gets a little insane with `&'* T` vs `&T` vs `*T`. - While this RFC does aim to make self-referential structs more possible, it does not aim to make them common or even easy. Pinning is not addressed at all, and must be well understood for any self-referential struct, and this RFC aims to be compatable with and separate from the pinning API. # Future possibilities [future-possibilities]: #future-possibilities -This may be valuable to ffi if you know that a pointer is aligned, as then using `&'?` may be more appropriate in this scenario +This may be valuable to ffi if you know that a pointer is aligned, as then using `&'unsafe` may be more appropriate in this scenario From 2b95c835f832e81e56a7ab3893244b2df8616651 Mon Sep 17 00:00:00 2001 From: Mason Boeman Date: Fri, 3 Dec 2021 00:15:28 -0600 Subject: [PATCH 11/15] changes from zulip discussion --- text/3199-unsafe-lifetime.md | 164 +++++++++++++++++++++++++++++------ 1 file changed, 136 insertions(+), 28 deletions(-) diff --git a/text/3199-unsafe-lifetime.md b/text/3199-unsafe-lifetime.md index ae753893237..39c2367b23b 100644 --- a/text/3199-unsafe-lifetime.md +++ b/text/3199-unsafe-lifetime.md @@ -44,8 +44,8 @@ First, let's explain why none of these really work, then show the fourth option, Introducing a new lifetime parameter has some problems: ```rust struct ArrayIter<'a, T> { - buffer: [T; 32] iter: std::slice::Iter<'a, T> + buffer: [T; 32] } ``` while this can work to set your iter up and potentially implement methods on ArrayIter, 'a has no meaning to someone consuming this struct. what do they instantiate this lifetime as? there is not a scope to which this lifetime has any meaningful connection, so it really pollutes your type. @@ -53,8 +53,8 @@ while this can work to set your iter up and potentially implement methods on Arr Replacing all self-references with pointers works, but not when you are not the implementor of the type which uses the lifetime. ```rust struct ArrayIter { - buffer: [T; 32] iter: MyPointerBasedReimplementationOfIter + buffer: [T; 32] } ``` This approach is unreasonable for all but the simplest borrowing types, as it requires you to fully re-implement anything intended for use with references to work in terms of pointers. @@ -62,8 +62,8 @@ This approach is unreasonable for all but the simplest borrowing types, as it re Using the `'static` lifetime almost works, but has one important failing: ```rust struct ArrayIter { - buffer: [T; 32] iter: std::slice::Iter<'static, T> + buffer: [T; 32] } ``` What if T is not `'static`? using the static lifetime here restricts our generic parameter T to being 'static, which is a concession we may not be ok with making. @@ -71,12 +71,19 @@ What if T is not `'static`? using the static lifetime here restricts our generic So how do we get all of the above? We use the unsafe lifetime `'unsafe` ```rust struct ArrayIter { + iter: ManuallyDrop> buffer: [T; 32] - iter: std::slice::Iter<'unsafe, T> +} + +impl Drop for ArrayIter { + fn drop(&mut self) { + let iter: &mut ManuallyDrop> = unsafe { core::mem::transmute(&mut self.iter) }; + iter.drop(); + } } ``` -Note that, like `'static`, `'unsafe` is allowed to appear in struct definitions without being declared. This is because the unsafe lifetime, like`'static`, is a specific lifetime and not a generic parameter.`'unsafe` can be thought of as "a lifetime which is outlived by all possible lifetimes. Dereferencing a reference with the unsafe lifetime is unsafe. Additionally, `'unsafe` is not acceptable when `'a` is expected because for any lifetime`'a`, `'unsafe` does not live long enough to satisfy its requirements. +Note that, like `'static`, `'unsafe` is allowed to appear in struct definitions without being declared. This is because the unsafe lifetime, like`'static`, is a specific lifetime and not a generic parameter.`'unsafe` can be thought of as "a lifetime which is outlived by all possible lifetimes. Dereferencing a reference with the unsafe lifetime is unsafe. Additionally, `'unsafe` is not acceptable when `'a` is expected because it never lives long enough. In general replacing a real lifetime with `'unsafe` should be thought of as a similar transformation to replacing a reference with a pointer. If you are doing it, you are doing it because safe rust does not allow for the type of code you are trying to write, and you're trying to encapsulate the unsafe into a compact part of your code. @@ -84,42 +91,83 @@ If you try to call a function which has a lifetime specifier (whether or not it The addition of the `'unsafe` lifetime also means the addition of two new reference types, `&'unsafe T` and `&'unsafe mut T`. These are in a sense halfway in between references and pointers. Dereferencing them is unsafe. `'static` references can be coerced into `'a` references, which can be coerced into `'unsafe` references, which can be coerced into raw pointers. -Now we also need to be able to actually produce a value to assign to our `std::slice::Iter<'unsafe, T>` field, which can be done by assigning to a value of the same type, except with a normal lifetime in place of unsafe, though there's an important catch. Any assignment to a "place" (whether to a variable, through a mutable reference, or setting a field of a struct) is unsafe if the type of the "place" uses 'unsafe instead of a generic lifetime specifier. The user must guarantee that the lifetime replaced by 'unsafe is valid when the value contained in place is dropped. +Now we also need to be able to actually produce a value to assign to our `std::slice::Iter<'unsafe, T>` field, which can be done by assigning to a value of the same type, except with a normal lifetime in place of unsafe. + +Wrapping in `ManuallyDrop` is required when using a type with `'unsafe` substituted for a lifetime parameter, and so to drop the iter we need a drop impl # Reference-level explanation [reference-level-explanation]: #reference-level-explanation -The unsafe lifetime is unique in a few ways: +The unsafe lifetime is unique in a few ways, which we will refer to as the "rules": 1. It is outlived by all other lifetimes (in the same way that `'static` outlives all other lifetimes). + - `'a: 'unsafe` for *all* `'a` + - `'unsafe: 'a` for *no* `'a` 2. The check that a reference does not outlive borrowed content is skipped when the borrowed content has `'unsafe` lifetime. 3. Assigning to or reading from `&'unsafe T` and `&'unsafe mut T` is unsafe. - The programmer must ensure the references must be valid, or this is UB. -4. Assigning to a variable whose type is instantiated with `'unsafe` in place of at least one of its lifetimes is unsafe. - - The programmer must ensure the `'unsafe` is valid when the variable is dropped (because step 5). -5. Dropping a value whose type is instantiated with `'unsafe` first transmutes the value to the same type with `'unsafe` replaced with `'_`. - -One important consequence of rule 1, which we will call rule 1.5: `'unsafe` references cannot be used where normal references are expected, because the borrow checker must negotiate a lifetime between the two, but any lifetime that can be assigned to `'a` necessarily outlives `'unsafe`. +4. Similar union fields, the types of owned values, which have a generic lifetime parameter instantiated to `'unsafe` must either: + - be of the shape `ManuallyDrop<_>` + - have a Drop impl specifically compatible with `'unsafe` for that lifetime parameter. Let's examine some of the implications of these features. -It is safe in general to store into a value expecting `'unsafe`, just like with raw pointers +## Rule 1 +### Coercion +If we want to get a type `T<'unsafe, 'b>` amd we have a type `T<'a, 'b>`, we can simply coerce `T<'a, 'b>` into `T<'unsafe, 'b>` +in this example our coercion target is simply a `&'unsafe usize` ```rust struct A { + // references are COPY, so rule 4 is satisfied val: &'unsafe usize } impl A { fn store(&mut self, some_ref: &usize) { // assigning to val, not through val. - // unsafe due to rule 4. If we were assigning through it would be rule 3. - unsafe { self.val = some_ref; } + // note that '_: 'unsafe by rule 1a, so this is allowed. + self.val = some_ref; } } ``` -An important but subtle point here is that it is only legal to have `&mut self` because of rule 2. Without it all references to A would outlive their borrowed content. -The only way to use a type that has been stored with `&'unsafe` is to use unsafe, or transmute it to a normal lifetime, which of course requires unsafe. +### Unusability +`'unsafe` references cannot be used where normal references are expected, because the borrow checker must negotiate a lifetime between the two, but any lifetime that can be assigned to `'a` necessarily outlives `'unsafe`. This is very important for preventing `'unsafe` from making its way into old functions without clearly using unsafe code for that purpose. + +```rust +fn my_existing_function(&usize) { ... } + +fn main() { + let a = 17; + let b: &usize = &a; + let c: &'unsafe usize = &a; + + // this is fine of course + my_existing_function(b) + // this fails because the lifetime of c ('unsafe) does not satisfy '_ by rule 1b + my_existing_function(c) +} +``` + +## Rule 2 + +lets examine the first example from rule 1 again: +```rust +struct A { + val: &'unsafe usize +} + +impl A { + fn store(&mut self, some_ref: &usize) { + self.val = some_ref; + } +} +``` +An important but subtle point here is that it is only legal to have `&mut self` at all because of rule 2. Without it all references to A would outlive their borrowed content. + +## Rule 3 +This is clearly required as we've thrown away the lifetime information, so we need to assure the compiler that the type is still valid. This is the same idea as casting a ref to a pointer, then needing to use unsafe to dereference. + ```rust impl A { // illegal @@ -128,25 +176,72 @@ impl A { } // legal, requiring unsafe. - fn read_deref(&self) -> usize { + fn read_legal(&self) -> usize { unsafe { *self.val } } +} +``` - // legal, requiring unsafe. - fn read_transmute(&self) -> usize { - *(unsafe { core::mem::transmute::<&'unsafe usize, &usize>(self.val) }) - } +## Rule 4 +Drop poses a problem for types which have had lifetimes replaced with `'unsafe`. Consider the following: + +```rust +struct X { + y: Y<'unsafe> +} + +struct Y<'a> { + some_val: &'a usize +} + +impl<'a> Drop for Y<'a> { + fn drop(&mut self) { ... } } ``` -Additionally, an important result of rule 1.5 is that methods defined on a reference to a type cannot be called on an unsafe reference unless they were specifically written to accept an unsafe reference, as unsafe references do not live long enough to be used where a normal reference would, like the method signature. + +What should happen when an `X` is dropped? we need to drop `y`, but we have a problem because in order to drop `y`, we need to call `drop`, but drop is implemented for `Y<'a>`, which means `Y<'unsafe>` doesn't satisfy the impl block, even though it may still need to run it. There are two approaches we can use: + +### ManuallyDrop +if we want all the special logic to be contained in X, which is often the case when writing self referential code, we use the `ManuallyDrop` approach: ```rust -impl A { - // errors - fn get(&self) -> usize { - self.val.clone() // clone expects &self, so this errors as val does not live long enough. +struct X { + y: ManuallyDrop> +} + +impl Drop for X { + fn drop(&mut self) { + // we need to be sure that our lifetime is valid here. + let y: &mut ManuallyDrop> = unsafe { core::mem::transmute(&mut self.y) }; + y.drop(); } } + +struct Y<'a> { + some_val: &'a usize +} + +impl<'a> Drop for Y<'a> { + fn drop(&mut self) { ... } +} +``` +note that y is now of the shape `ManuallyDrop<_>`, and that we had to use unsafe to properly call the drop, because we needed a `ManuallyDrop>`, not a `ManuallyDrop>` + +### Impl Drop +If we want the type `Y` to be especially convenient for use with the unsafe lifetime, we can use the second approach: +```rust +struct X { + y: Y<'unsafe> +} + +struct Y<'a> { + some_val: &'a usize +} + +impl Drop for Y<'unsafe> { + fn drop(&mut self) { ... } +} ``` +If Y doesn't need to dereference some_val in its drop implementation then `'unsafe` is all it needs, and the default drop behavior on `X` is fine. # Drawbacks [drawbacks]: #drawbacks @@ -182,4 +277,17 @@ Some crates for dealing with self reference: # Future possibilities [future-possibilities]: #future-possibilities -This may be valuable to ffi if you know that a pointer is aligned, as then using `&'unsafe` may be more appropriate in this scenario +This example may solve some of the problems that are currently solved by `#[may_dangle]`: +```rust +struct Y<'a> { + some_val: &'a usize +} + +impl Drop for Y<'unsafe> { + fn drop(&mut self) { ... } +} +``` + +Adding more permissive `Drop` impls throughout the standard library may make these types more ergonomic with no effect to code which does not use the `'unsafe` lifetime. + +This may be valuable to ffi if you know that a pointer is aligned, as using `&'unsafe` may be more appropriate in this scenario From eeb271fa6fb4ad01dc818c93b3c401915ad313c4 Mon Sep 17 00:00:00 2001 From: Mason Date: Fri, 3 Dec 2021 00:16:06 -0600 Subject: [PATCH 12/15] Update text/3199-unsafe-lifetime.md Co-authored-by: Noah Lev --- text/3199-unsafe-lifetime.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/text/3199-unsafe-lifetime.md b/text/3199-unsafe-lifetime.md index 39c2367b23b..39ac8ecbe91 100644 --- a/text/3199-unsafe-lifetime.md +++ b/text/3199-unsafe-lifetime.md @@ -11,7 +11,7 @@ Introduce a new special lifetime `'unsafe` which is outlived by all other lifeti # Motivation [motivation]: #motivation -When creating self-referential structs it is often preferred to use pointers over references because the conditions under which the pointer/reference is valid are not evaluated by the borrow checker. The problem with this general approach is that it does not scale well to more complex types. If we have a the following: +When creating self-referential structs it is often preferred to use pointers over references because the conditions under which the pointer/reference is valid are not evaluated by the borrow checker. The problem with this general approach is that it does not scale well to more complex types. If we have the following: ```rust struct A { item: T From fa1cefa887071251b991bd3bd3b8424db78a534a Mon Sep 17 00:00:00 2001 From: Mason Date: Fri, 3 Dec 2021 21:26:39 -0600 Subject: [PATCH 13/15] Update text/3199-unsafe-lifetime.md Co-authored-by: teor --- text/3199-unsafe-lifetime.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/text/3199-unsafe-lifetime.md b/text/3199-unsafe-lifetime.md index 39ac8ecbe91..14fdd83ecd3 100644 --- a/text/3199-unsafe-lifetime.md +++ b/text/3199-unsafe-lifetime.md @@ -105,7 +105,7 @@ The unsafe lifetime is unique in a few ways, which we will refer to as the "rule 2. The check that a reference does not outlive borrowed content is skipped when the borrowed content has `'unsafe` lifetime. 3. Assigning to or reading from `&'unsafe T` and `&'unsafe mut T` is unsafe. - The programmer must ensure the references must be valid, or this is UB. -4. Similar union fields, the types of owned values, which have a generic lifetime parameter instantiated to `'unsafe` must either: +4. Similar to union fields, the types of owned values which have a generic lifetime parameter instantiated to `'unsafe` must either: - be of the shape `ManuallyDrop<_>` - have a Drop impl specifically compatible with `'unsafe` for that lifetime parameter. From e0d132d83f7c1d102d8f69d42dbd60095a4b8560 Mon Sep 17 00:00:00 2001 From: Mason Boeman Date: Sat, 4 Dec 2021 12:22:47 -0600 Subject: [PATCH 14/15] elaborate on some alternatives --- text/3199-unsafe-lifetime.md | 81 +++++++++++++++++++++++++++++++++++- 1 file changed, 79 insertions(+), 2 deletions(-) diff --git a/text/3199-unsafe-lifetime.md b/text/3199-unsafe-lifetime.md index 14fdd83ecd3..88e821db617 100644 --- a/text/3199-unsafe-lifetime.md +++ b/text/3199-unsafe-lifetime.md @@ -66,7 +66,16 @@ struct ArrayIter { buffer: [T; 32] } ``` -What if T is not `'static`? using the static lifetime here restricts our generic parameter T to being 'static, which is a concession we may not be ok with making. +What if T is not `'static`? using the static lifetime here restricts our generic parameter T to being 'static, which is a concession we may not be ok with making. It turns out this is extremely pervasive. Even if we went with the nuclear approach: +```rust +#![feature(generic_const_exprs)] + +struct ArrayIter { + iter: [u8; std::mem::size_of::>()] + buffer: [T; 32] +} +``` +and just transmuted whenever needed, this still requires T to be `'static.` So how do we get all of the above? We use the unsafe lifetime `'unsafe` ```rust @@ -253,7 +262,75 @@ If Y doesn't need to dereference some_val in its drop implementation then `'unsa [rationale-and-alternatives]: #rationale-and-alternatives - There isn't any analogue in the higher type system to the lifetime erasure that occurs when coalescing from reference to pointer. -- An alternative could be a macro-based library. This approach seems unlikely to check all the boxes though + +## Alternatives +### Do nothing +It is not technically possible to have self reference to non `'static` types unless you reimplement with pointers, but this is also a niche thing to want to do. adding language complexity may not be worth it for this niche. + +### Add special size_of operator syntax +Add a way to perform `std::mem::size_of::>()` as a const fn. If there is a way to get the size of a type regardless of its lifetime specifiers in a const context, then everything here is possible (with use of the `generic_const_exprs` feature). The following would work: +```rust +#![feature(generic_const_exprs)] + +use core::pin::Pin; +use core::slice::Iter; + +// the only reason we need this 'static bound is because there is no way to ask rust +// what the size of Iter<'anything, T> is, which doesn't change depending on the lifetime. +// if we could do core::mem::size_of::>(), we would not need the bound on T. +struct Test +where [u8; core::mem::size_of::>()]: Sized +{ + // this is an Iter<'self, T> with the type fully erased. + // this is absurdly unsafe. + my_iter: [u8; core::mem::size_of::>()], + + // this must be treated as immutably borrowed. + my_vec: Vec, +} + +impl Test +where [u8; core::mem::size_of::>()]: Sized +{ + // using this function is UB if init is not called before any other methods, or before dropping. + pub unsafe fn new(my_vec: Vec) -> Self { + Self { + my_iter: [0; core::mem::size_of::>()], + my_vec, + } + } + + pub fn init(self: Pin<&mut Self>) { + let this = unsafe { self.get_unchecked_mut() }; + let iter = this.my_vec.iter(); + this.my_iter = unsafe {core::mem::transmute(iter) };; + } + + fn get_my_iter(&mut self) -> &mut Iter<'_, T> { + unsafe { core::mem::transmute(&mut self.my_iter) } + } + + pub fn next(self: Pin<&mut Self>) -> Option<&T> { + let this = unsafe { self.get_unchecked_mut() }; + this.get_my_iter().next() + } +} + +impl Drop for Test + +where [u8; core::mem::size_of::>()]: Sized{ + fn drop(&mut self) { + unsafe { + let iter: &mut core::mem::ManuallyDrop> = core::mem::transmute(&mut self.my_iter); + core::mem::ManuallyDrop::drop(iter); + } + } +} +``` + +### Add some sort of `'self` lifetime + +As the primary use case this is trying to support is self reference, maybe it should just be made explicit. I suspect this would work similarly to `'unsafe` as it has been layed out here, just with a more restricted domain. # Prior art [prior-art]: #prior-art From 1a3da1185e0eaeb6ebf248b8c62abfd28d5589de Mon Sep 17 00:00:00 2001 From: Mason Boeman Date: Thu, 24 Mar 2022 19:42:00 -0500 Subject: [PATCH 15/15] reword as erased --- text/3199-unsafe-lifetime.md | 181 +++++++++++++++++++++++++++-------- 1 file changed, 140 insertions(+), 41 deletions(-) diff --git a/text/3199-unsafe-lifetime.md b/text/3199-unsafe-lifetime.md index 88e821db617..57035f571e9 100644 --- a/text/3199-unsafe-lifetime.md +++ b/text/3199-unsafe-lifetime.md @@ -6,26 +6,125 @@ # Summary [summary]: #summary -Introduce a new special lifetime `'unsafe` which is outlived by all other lifetimes. Using a type through a 'unsafe reference, or which is instantiated with an 'unsafe lifetime parameter is rarely possible without unsafe. +Introduce a new special lifetime `'erased` which represents an erased lifetime parameter. Dereferencing a `&'erased T` or `&'erased mut T` is unsafe, and all generic parameters (whether lifetime or type) have the implicit bound `'a: '!erased` or `T: '!erased` unless they explicitly opt out with `'a: '?erased`. Additionally (and perhaps optionally) introduce a restricted version of transmute called `unerase_lifetimes` which takes `T<'a, 'erased, 'erased>` and transmutes to `T<'a, 'b, 'c>`. # Motivation [motivation]: #motivation -When creating self-referential structs it is often preferred to use pointers over references because the conditions under which the pointer/reference is valid are not evaluated by the borrow checker. The problem with this general approach is that it does not scale well to more complex types. If we have the following: +There are two particularly useful applications of a `'erased` lifetime: +- Enabling self referential structs, which are only really possible with +- Resolving the dropcheck eyepatch by eliminating the requirement for the `#[may_dangle]` attribute. + +# Guide-level explanation +[guide-level-explanation]: #guide-level-explanation + +## Self Reference + +The borrow checker does not understand self referential types. If we want to write the following: ```rust struct A { item: T borrower: B<'?, T> // we want the ref inside this to refer to item } +impl A { + pub fn a_func(&self) -> &'a T { + let t: &T = self.borrower.b_func() + todo!() + } +} + struct B<'a, T> { actual_ref: &'a T } + +impl<'a, T> B { + pub fn b_func(&self) -> &'a T { todo!() } +} ``` -there is no choice for a lifetime to replace `'?` with because `'static` may outlive `T` if it contains lifetimes, and we may not want to replace the ref inside `B` with a pointer, because `B` may have value apart from being stored in a self-reference. +What should the lifetime parameter `'?` be? There are traditionally three choices: +- introduce a new lifetime parameter +- replace all self-references with pointers +- use `'static` and transmute our lifetime into it -# Guide-level explanation -[guide-level-explanation]: #guide-level-explanation +Right now, there is only one way to really achieve this, which is to replace the refs with pointers: +```rust +struct A { + item: T + borrower: B // we want the ref inside this to refer to item +} + +impl A { + pub fn a_func(&self) -> &'a T { + let t_1: &T = unsafe { self.borrower.b_func_1().as_ref().unwrap() } + let t_2: &T = self.borrower.b_func_2() + todo!() + } +} + +struct B { + actual_ref: *const T +} + +impl B { + // which of these is best? + pub fn b_func_1<'a>(&self) -> &'a T { todo!() } + pub fn b_func_2(&self) -> *const T { todo!() } +} +``` +This works, but the we have shifted the burden of unsafe away from where the requirements are actually being upheld, which is in struct `A`. The original lifetime based implementation of `B` doesn't care that the lifetime is self referential; only that it is valid whenever `B` is used, and therefore can be verified by the borrow checker. It makes more sense for `A`, the struct creating and storing self references, to bear the unsafe burden of dereferencing than for `B` to. + +How do we use the `'erased` lifetime to resolve this? +```rust +struct A { + item: T + borrower: B<'erased, T> // we want the ref inside this to refer to item +} + +impl A { + pub fn a_func(&self) -> &'a T { + let t: &T = unsafe { self.borrower.unerase_lifetimes() }.b_func() + todo!() + } +} + +struct B<'a: '?erased, T> { + actual_ref: &'a T +} + +impl<'a, T> B { + pub fn b_func(&self) -> &'a T { todo!() } +} +``` + +Note that the use of unsafe is confined to `A` and it's `impl`s, which is where the unsafe action of creating self references actually occurs. The only modification made to `B` is marking `'a` as `'?erased`. The same modification is not made to the `impl` for `B`, which means that a value of type `B<'erased, T>` cannot be used as the self parameter of `b_func` without first being unerased. + +## Dropcheck Eyepatch + +Right now the implementation of box looks something like this: +```rust +pub struct Box { + pointer: *mut T, +}; + +unsafe impl<#[may_dangle] T: ?Sized> Drop for Box { + fn drop(&mut self) { + // FIXME: Do nothing, drop is currently performed by compiler. + } +} +``` +but the same effect could be achieved by this: +```rust +pub struct Box { + pointer: *mut T, +}; + +impl Drop for Box { + fn drop(&mut self) { + // FIXME: Do nothing, drop is currently performed by compiler. + } +} +``` There are situations where, when writing unsafe code, you may need to store a type without encoding its lifetime in the type system. The existence of raw pointers in the language is an acknowledgement of this need, but it is not always perfectly ergonomic to use pointers in this scenario. Consider the following self-referential struct: ```rust @@ -77,10 +176,10 @@ struct ArrayIter { ``` and just transmuted whenever needed, this still requires T to be `'static.` -So how do we get all of the above? We use the unsafe lifetime `'unsafe` +So how do we get all of the above? We use the unsafe lifetime `'erased` ```rust struct ArrayIter { - iter: ManuallyDrop> + iter: ManuallyDrop> buffer: [T; 32] } @@ -92,56 +191,56 @@ impl Drop for ArrayIter { } ``` -Note that, like `'static`, `'unsafe` is allowed to appear in struct definitions without being declared. This is because the unsafe lifetime, like`'static`, is a specific lifetime and not a generic parameter.`'unsafe` can be thought of as "a lifetime which is outlived by all possible lifetimes. Dereferencing a reference with the unsafe lifetime is unsafe. Additionally, `'unsafe` is not acceptable when `'a` is expected because it never lives long enough. +Note that, like `'static`, `'erased` is allowed to appear in struct definitions without being declared. This is because the unsafe lifetime, like`'static`, is a specific lifetime and not a generic parameter.`'erased` can be thought of as "a lifetime which is outlived by all possible lifetimes. Dereferencing a reference with the unsafe lifetime is unsafe. Additionally, `'erased` is not acceptable when `'a` is expected because it never lives long enough. -In general replacing a real lifetime with `'unsafe` should be thought of as a similar transformation to replacing a reference with a pointer. If you are doing it, you are doing it because safe rust does not allow for the type of code you are trying to write, and you're trying to encapsulate the unsafe into a compact part of your code. +In general replacing a real lifetime with `'erased` should be thought of as a similar transformation to replacing a reference with a pointer. If you are doing it, you are doing it because safe rust does not allow for the type of code you are trying to write, and you're trying to encapsulate the unsafe into a compact part of your code. -If you try to call a function which has a lifetime specifier (whether or not it has been elided in the signature) using a `'unsafe` lifetime you will get a compilation error, because any lifetime that the borrow checker could possibly choose for this call to your function would outlive a `'unsafe` reference. +If you try to call a function which has a lifetime specifier (whether or not it has been elided in the signature) using a `'erased` lifetime you will get a compilation error, because any lifetime that the borrow checker could possibly choose for this call to your function would outlive a `'erased` reference. -The addition of the `'unsafe` lifetime also means the addition of two new reference types, `&'unsafe T` and `&'unsafe mut T`. These are in a sense halfway in between references and pointers. Dereferencing them is unsafe. `'static` references can be coerced into `'a` references, which can be coerced into `'unsafe` references, which can be coerced into raw pointers. +The addition of the `'erased` lifetime also means the addition of two new reference types, `&'erased T` and `&'erased mut T`. These are in a sense halfway in between references and pointers. Dereferencing them is unsafe. `'static` references can be coerced into `'a` references, which can be coerced into `'erased` references, which can be coerced into raw pointers. -Now we also need to be able to actually produce a value to assign to our `std::slice::Iter<'unsafe, T>` field, which can be done by assigning to a value of the same type, except with a normal lifetime in place of unsafe. +Now we also need to be able to actually produce a value to assign to our `std::slice::Iter<'erased, T>` field, which can be done by assigning to a value of the same type, except with a normal lifetime in place of unsafe. -Wrapping in `ManuallyDrop` is required when using a type with `'unsafe` substituted for a lifetime parameter, and so to drop the iter we need a drop impl +Wrapping in `ManuallyDrop` is required when using a type with `'erased` substituted for a lifetime parameter, and so to drop the iter we need a drop impl # Reference-level explanation [reference-level-explanation]: #reference-level-explanation The unsafe lifetime is unique in a few ways, which we will refer to as the "rules": 1. It is outlived by all other lifetimes (in the same way that `'static` outlives all other lifetimes). - - `'a: 'unsafe` for *all* `'a` - - `'unsafe: 'a` for *no* `'a` -2. The check that a reference does not outlive borrowed content is skipped when the borrowed content has `'unsafe` lifetime. -3. Assigning to or reading from `&'unsafe T` and `&'unsafe mut T` is unsafe. + - `'a: 'erased` for *all* `'a` + - `'erased: 'a` for *no* `'a` +2. The check that a reference does not outlive borrowed content is skipped when the borrowed content has `'erased` lifetime. +3. Assigning to or reading from `&'erased T` and `&'erased mut T` is unsafe. - The programmer must ensure the references must be valid, or this is UB. -4. Similar to union fields, the types of owned values which have a generic lifetime parameter instantiated to `'unsafe` must either: +4. Similar to union fields, the types of owned values which have a generic lifetime parameter instantiated to `'erased` must either: - be of the shape `ManuallyDrop<_>` - - have a Drop impl specifically compatible with `'unsafe` for that lifetime parameter. + - have a Drop impl specifically compatible with `'erased` for that lifetime parameter. Let's examine some of the implications of these features. ## Rule 1 ### Coercion -If we want to get a type `T<'unsafe, 'b>` amd we have a type `T<'a, 'b>`, we can simply coerce `T<'a, 'b>` into `T<'unsafe, 'b>` +If we want to get a type `T<'erased, 'b>` amd we have a type `T<'a, 'b>`, we can simply coerce `T<'a, 'b>` into `T<'erased, 'b>` -in this example our coercion target is simply a `&'unsafe usize` +in this example our coercion target is simply a `&'erased usize` ```rust struct A { // references are COPY, so rule 4 is satisfied - val: &'unsafe usize + val: &'erased usize } impl A { fn store(&mut self, some_ref: &usize) { // assigning to val, not through val. - // note that '_: 'unsafe by rule 1a, so this is allowed. + // note that '_: 'erased by rule 1a, so this is allowed. self.val = some_ref; } } ``` ### Unusability -`'unsafe` references cannot be used where normal references are expected, because the borrow checker must negotiate a lifetime between the two, but any lifetime that can be assigned to `'a` necessarily outlives `'unsafe`. This is very important for preventing `'unsafe` from making its way into old functions without clearly using unsafe code for that purpose. +`'erased` references cannot be used where normal references are expected, because the borrow checker must negotiate a lifetime between the two, but any lifetime that can be assigned to `'a` necessarily outlives `'erased`. This is very important for preventing `'erased` from making its way into old functions without clearly using unsafe code for that purpose. ```rust fn my_existing_function(&usize) { ... } @@ -149,11 +248,11 @@ fn my_existing_function(&usize) { ... } fn main() { let a = 17; let b: &usize = &a; - let c: &'unsafe usize = &a; + let c: &'erased usize = &a; // this is fine of course my_existing_function(b) - // this fails because the lifetime of c ('unsafe) does not satisfy '_ by rule 1b + // this fails because the lifetime of c ('erased) does not satisfy '_ by rule 1b my_existing_function(c) } ``` @@ -163,7 +262,7 @@ fn main() { lets examine the first example from rule 1 again: ```rust struct A { - val: &'unsafe usize + val: &'erased usize } impl A { @@ -192,11 +291,11 @@ impl A { ``` ## Rule 4 -Drop poses a problem for types which have had lifetimes replaced with `'unsafe`. Consider the following: +Drop poses a problem for types which have had lifetimes replaced with `'erased`. Consider the following: ```rust struct X { - y: Y<'unsafe> + y: Y<'erased> } struct Y<'a> { @@ -208,13 +307,13 @@ impl<'a> Drop for Y<'a> { } ``` -What should happen when an `X` is dropped? we need to drop `y`, but we have a problem because in order to drop `y`, we need to call `drop`, but drop is implemented for `Y<'a>`, which means `Y<'unsafe>` doesn't satisfy the impl block, even though it may still need to run it. There are two approaches we can use: +What should happen when an `X` is dropped? we need to drop `y`, but we have a problem because in order to drop `y`, we need to call `drop`, but drop is implemented for `Y<'a>`, which means `Y<'erased>` doesn't satisfy the impl block, even though it may still need to run it. There are two approaches we can use: ### ManuallyDrop if we want all the special logic to be contained in X, which is often the case when writing self referential code, we use the `ManuallyDrop` approach: ```rust struct X { - y: ManuallyDrop> + y: ManuallyDrop> } impl Drop for X { @@ -233,24 +332,24 @@ impl<'a> Drop for Y<'a> { fn drop(&mut self) { ... } } ``` -note that y is now of the shape `ManuallyDrop<_>`, and that we had to use unsafe to properly call the drop, because we needed a `ManuallyDrop>`, not a `ManuallyDrop>` +note that y is now of the shape `ManuallyDrop<_>`, and that we had to use unsafe to properly call the drop, because we needed a `ManuallyDrop>`, not a `ManuallyDrop>` ### Impl Drop If we want the type `Y` to be especially convenient for use with the unsafe lifetime, we can use the second approach: ```rust struct X { - y: Y<'unsafe> + y: Y<'erased> } struct Y<'a> { some_val: &'a usize } -impl Drop for Y<'unsafe> { +impl Drop for Y<'erased> { fn drop(&mut self) { ... } } ``` -If Y doesn't need to dereference some_val in its drop implementation then `'unsafe` is all it needs, and the default drop behavior on `X` is fine. +If Y doesn't need to dereference some_val in its drop implementation then `'erased` is all it needs, and the default drop behavior on `X` is fine. # Drawbacks [drawbacks]: #drawbacks @@ -330,7 +429,7 @@ where [u8; core::mem::size_of::>()]: Sized{ ### Add some sort of `'self` lifetime -As the primary use case this is trying to support is self reference, maybe it should just be made explicit. I suspect this would work similarly to `'unsafe` as it has been layed out here, just with a more restricted domain. +As the primary use case this is trying to support is self reference, maybe it should just be made explicit. I suspect this would work similarly to `'erased` as it has been layed out here, just with a more restricted domain. # Prior art [prior-art]: #prior-art @@ -338,8 +437,8 @@ As the primary use case this is trying to support is self reference, maybe it sh previous rfc: [unsafe lifetime by jpernst](https://github.com/rust-lang/rfcs/pull/1918/) Many things are similar between these two RFCs, but there are a couple important differences that solve the problems with the first unsafe lifetime attempt: -- Instead of `'unsafe` satisfying *all* constraints, it satisfies *no* constraints. This makes it so existing functions cannot be called with unsafe lifetimes. -- operations which create values whose types contain `'unsafe` are generally all safe. These values are mostly unusable unless they are transmuted back to a useful lifetime, which is unsafe. +- Instead of `'erased` satisfying *all* constraints, it satisfies *no* constraints. This makes it so existing functions cannot be called with unsafe lifetimes. +- operations which create values whose types contain `'erased` are generally all safe. These values are mostly unusable unless they are transmuted back to a useful lifetime, which is unsafe. Some crates for dealing with self reference: - [owning_ref](https://crates.io/crates/owning_ref) is an early attempt to make self references ergonomic but is slightly clunky and not generic enough for some use cases. @@ -360,11 +459,11 @@ struct Y<'a> { some_val: &'a usize } -impl Drop for Y<'unsafe> { +impl Drop for Y<'erased> { fn drop(&mut self) { ... } } ``` -Adding more permissive `Drop` impls throughout the standard library may make these types more ergonomic with no effect to code which does not use the `'unsafe` lifetime. +Adding more permissive `Drop` impls throughout the standard library may make these types more ergonomic with no effect to code which does not use the `'erased` lifetime. -This may be valuable to ffi if you know that a pointer is aligned, as using `&'unsafe` may be more appropriate in this scenario +This may be valuable to ffi if you know that a pointer is aligned, as using `&'erased` may be more appropriate in this scenario