diff --git a/src/types/closure.md b/src/types/closure.md index 0f59f3f64..d46d7f7ae 100644 --- a/src/types/closure.md +++ b/src/types/closure.md @@ -2,40 +2,51 @@ r[type.closure] -A [closure expression] produces a closure value with a unique, anonymous type -that cannot be written out. A closure type is approximately equivalent to a -struct which contains the captured variables. For instance, the following -closure: +r[type.closure.intro] +A [closure expression] produces a closure value with a unique, anonymous type that cannot be written out. +A closure type is approximately equivalent to a struct which contains the captured values. +For instance, the following closure: ```rust +#[derive(Debug)] +struct Point { x: i32, y: i32 } +struct Rectangle { left_top: Point, right_bottom: Point } + fn f String> (g: F) { println!("{}", g()); } -let mut s = String::from("foo"); -let t = String::from("bar"); - -f(|| { - s += &t; - s -}); -// Prints "foobar". +let mut rect = Rectangle { + left_top: Point { x: 1, y: 1 }, + right_bottom: Point { x: 0, y: 0 } +}; + +let c = || { + rect.left_top.x += 1; + rect.right_bottom.x += 1; + format!("{:?}", rect.left_top) +}; +f(c); // Prints "Point { x: 2, y: 1 }". ``` generates a closure type roughly like the following: - + ```rust,ignore +// Note: This is not exactly how it is translated, this is only for +// illustration. + struct Closure<'a> { - s : String, - t : &'a String, + left_top : &'a mut Point, + right_bottom_x : &'a mut i32, } impl<'a> FnOnce<()> for Closure<'a> { type Output = String; - fn call_once(self) -> String { - self.s += &*self.t; - self.s + extern "rust-call" fn call_once(self, args: ()) -> String { + self.left_top.x += 1; + *self.right_bottom_x += 1; + format!("{:?}", self.left_top) } } ``` @@ -44,84 +55,417 @@ so that the call to `f` works as if it were: ```rust,ignore -f(Closure{s: s, t: &t}); +// Note: This is not valid Rust due to the duplicate mutable borrows. +// This is only provided as an illustration. +f(Closure{ left_top: &mut rect.left_top, right_bottom_x: &mut rect.left_top.x }); ``` ## Capture modes r[type.closure.capture] -r[type.closure.capture.order] -The compiler prefers to capture a closed-over variable by immutable borrow, -followed by unique immutable borrow (see below), by mutable borrow, and finally -by move. It will pick the first choice of these that is compatible with how the -captured variable is used inside the closure body. The compiler does not take -surrounding code into account, such as the lifetimes of involved variables, or -of the closure itself. +r[type.closure.capture.intro] +A *capture mode* determines how a [place expression] from the environment is borrowed or moved into the closure. +The capture modes are: + +1. Immutable borrow (`ImmBorrow`) --- The place expression is captured as a [shared reference]. +2. Unique immutable borrow (`UniqueImmBorrow`) --- This is similar to an immutable borrow, but must be unique as described [below](#unique-immutable-borrows-in-captures). +3. Mutable borrow (`MutBorrow`) --- The place expression is captured as a [mutable reference]. +4. Move (`ByValue`) --- The place expression is captured by [moving the value] into the closure. + +r[type.closure.capture.precedence] +Place expressions from the environment are captured from the first mode that is compatible with how the captured value is used inside the closure body. +The mode is not affected by the code surrounding the closure, such as the lifetimes of involved variables or fields, or of the closure itself. + +[moving the value]: ../expressions.md#moved-and-copied-types +[mutable reference]: pointer.md#mutable-references-mut +[place expression]: ../expressions.md#place-expressions-and-value-expressions +[shared reference]: pointer.md#references--and-mut -r[type.closure.capture.move] -If the `move` keyword is used, then all captures are by move or, for `Copy` -types, by copy, regardless of whether a borrow would work. The `move` keyword is -usually used to allow the closure to outlive the captured values, such as if the -closure is being returned or used to spawn a new thread. +### `Copy` values -r[type.closure.capture.composite] -Composite types such as structs, tuples, and enums are always captured entirely, -not by individual fields. It may be necessary to borrow into a local variable in -order to capture a single field: - +r[type.closure.capture.copy] +Values that implement [`Copy`] that are moved into the closure are captured with the `ImmBorrow` mode. ```rust -# use std::collections::HashSet; -# -struct SetVec { - set: HashSet, - vec: Vec +let x = [0; 1024]; +let c = || { + let y = x; // x captured by ImmBorrow +}; +``` + +## Capture Precision + +r[type.closure.capture.precision.capture-path] +A *capture path* is a sequence starting with a variable from the environment followed by zero or more place projections that were applied to that variable. + +r[type.closure.capture.precision.place-projection] +A *place projection* is a [field access], [tuple index], [dereference] (and automatic dereferences), or [array or slice index] expression applied to a variable. + +r[type.closure.capture.precision.intro] +The closure borrows or moves the capture path, which may be truncated based on the rules described below. + +For example: + +```rust +struct SomeStruct { + f1: (i32, i32), } +let s = SomeStruct { f1: (1, 2) }; -impl SetVec { - fn populate(&mut self) { - let vec = &mut self.vec; - self.set.iter().for_each(|&n| { - vec.push(n); - }) - } +let c = || { + let x = s.f1.1; // s.f1.1 captured by ImmBorrow +}; +c(); +``` + +Here the capture path is the local variable `s`, followed by a field access `.f1`, and then a tuple index `.1`. +This closure captures an immutable borrow of `s.f1.1`. + +[field access]: ../expressions/field-expr.md +[tuple index]: ../expressions/tuple-expr.md#tuple-indexing-expressions +[dereference]: ../expressions/operator-expr.md#the-dereference-operator +[array or slice index]: ../expressions/array-expr.md#array-and-slice-indexing-expressions + +### Shared prefix + +r[type.closure.capture.precision.shared-prefix] +In the case where a capture path and one of the ancestor’s of that path are both captured by a closure, the ancestor path is captured with the highest capture mode among the two captures, `CaptureMode = max(AncestorCaptureMode, DescendantCaptureMode)`, using the strict weak ordering: + +`ImmBorrow < UniqueImmBorrow < MutBorrow < ByValue` + +Note that this might need to be applied recursively. + +```rust +// In this example, there are three different capture paths with a shared ancestor: +# fn move_value(_: T){} +let s = String::from("S"); +let t = (s, String::from("T")); +let mut u = (t, String::from("U")); + +let c = || { + println!("{:?}", u); // u captured by ImmBorrow + u.1.truncate(0); // u.0 captured by MutBorrow + move_value(u.0.0); // u.0.0 captured by ByValue +}; +c(); +``` + +Overall this closure will capture `u` by `ByValue`. + +### Rightmost shared reference truncation + +r[type.closure.capture.precision.dereference-shared] +The capture path is truncated at the rightmost dereference in the capture path if the dereference is applied to a shared reference. + +This truncation is allowed because fields that are read through a shared reference will always be read via a shared reference or a copy. +This helps reduce the size of the capture when the extra precision does not yield any benefit from a borrow checking perspective. + +The reason it is the *rightmost* dereference is to help avoid a shorter lifetime than is necessary. +Consider the following example: + +```rust +struct Int(i32); +struct B<'a>(&'a i32); + +struct MyStruct<'a> { + a: &'static Int, + b: B<'a>, +} + +fn foo<'a, 'b>(m: &'a MyStruct<'b>) -> impl FnMut() + 'static { + let c = || drop(&m.a.0); + c } ``` -If, instead, the closure were to use `self.vec` directly, then it would attempt -to capture `self` by mutable reference. But since `self.set` is already -borrowed to iterate over, the code would not compile. +If this were to capture `m`, then the closure would no longer outlive `'static`, since `m` is constrained to `'a`. Instead, it captures `(*(*m).a)` by `ImmBorrow`. + +### Wildcard pattern bindings + +r[type.closure.capture.precision.wildcard] +Closures only capture data that needs to be read. +Binding a value with a [wildcard pattern] does not count as a read, and thus won't be captured. +For example, the following closures will not capture `x`: + +```rust +let x = String::from("hello"); +let c = || { + let _ = x; // x is not captured +}; +c(); + +let c = || match x { // x is not captured + _ => println!("Hello World!") +}; +c(); +``` + +This also includes destructuring of tuples, structs, and enums. +Fields matched with the [_RestPattern_] or [_StructPatternEtCetera_] are also not considered as read, and thus those fields will not be captured. +The following illustrates some of these: + +```rust +let x = (String::from("a"), String::from("b")); +let c = || { + let (first, ..) = x; // captures `x.0` ByValue +}; +// The first tuple field has been moved into the closure. +// The second tuple field is still accessible. +println!("{:?}", x.1); +c(); +``` + +```rust +struct Example { + f1: String, + f2: String, +} + +let e = Example { + f1: String::from("first"), + f2: String::from("second"), +}; +let c = || { + let Example { f2, .. } = e; // captures `e.f2` ByValue +}; +// Field f2 cannot be accessed since it is moved into the closure. +// Field f1 is still accessible. +println!("{:?}", e.f1); +c(); +``` + +r[type.closure.capture.precision.wildcard.array-slice] +Partial captures of arrays and slices are not supported; the entire slice or array is always captured even if used with wildcard pattern matching, indexing, or sub-slicing. +For example: + +```rust,compile_fail,E0382 +#[derive(Debug)] +struct Example; +let x = [Example, Example]; + +let c = || { + let [first, _] = x; // captures all of `x` ByValue +}; +c(); +println!("{:?}", x[1]); // ERROR: borrow of moved value: `x` +``` + +r[type.closure.capture.precision.wildcard.initialized] +Values that are matched with wildcards must still be initialized. + +```rust,compile_fail,E0381 +let x: i32; +let c = || { + let _ = x; // ERROR: used binding `x` isn't initialized +}; +``` + +[_RestPattern_]: ../patterns.md#rest-patterns +[_StructPatternEtCetera_]: ../patterns.md#struct-patterns +[wildcard pattern]: ../patterns.md#wildcard-pattern + +### Capturing references in move contexts + +r[type.closure.capture.precision.move-dereference] +Because it is not allowed to move fields out of a reference, `move` closures will only capture the prefix of a capture path that runs up to, but not including, the first dereference of a reference. +The reference itself will be moved into the closure. + +```rust +struct T(String, String); + +let mut t = T(String::from("foo"), String::from("bar")); +let t_mut_ref = &mut t; +let mut c = move || { + t_mut_ref.0.push_str("123"); // captures `t_mut_ref` ByValue +}; +c(); +``` + +### Raw pointer dereference + +r[type.closure.capture.precision.raw-pointer-dereference] +Because it is `unsafe` to dereference a raw pointer, closures will only capture the prefix of a capture path that runs up to, but not including, the first dereference of a raw pointer. + +```rust +struct T(String, String); + +let t = T(String::from("foo"), String::from("bar")); +let t_ptr = &t as *const T; + +let c = || unsafe { + println!("{}", (*t_ptr).0); // captures `t_ptr` by ImmBorrow +}; +c(); +``` + +### Union fields + +r[type.closure.capture.precision.union] +Because it is `unsafe` to access a union field, closures will only capture the prefix of a capture path that runs up to the union itself. + +```rust +union U { + a: (i32, i32), + b: bool, +} +let u = U { a: (123, 456) }; + +let c = || { + let x = unsafe { u.a.0 }; // captures `u` ByValue +}; +c(); + +// This also includes writing to fields. +let mut u = U { a: (123, 456) }; + +let mut c = || { + u.b = true; // captures `u` with MutBorrow +}; +c(); +``` + +### Reference into unaligned `struct`s + +r[type.closure.capture.precision.unaligned] +Because it is [undefined behavior] to create references to unaligned fields in a structure, +closures will only capture the prefix of the capture path that runs up to, but not including, the first field access into a structure that uses [the `packed` representation]. +This includes all fields, even those that are aligned, to protect against compatibility concerns should any of the fields in the structure change in the future. + +```rust +#[repr(packed)] +struct T(i32, i32); + +let t = T(2, 5); +let c = || { + let a = t.0; // captures `t` with ImmBorrow +}; +// Copies out of `t` are ok. +let (a, b) = (t.0, t.1); +c(); +``` + +Similarly, taking the address of an unaligned field also captures the entire struct: + +```rust,compile_fail,E0505 +#[repr(packed)] +struct T(String, String); + +let mut t = T(String::new(), String::new()); +let c = || { + let a = std::ptr::addr_of!(t.1); // captures `t` with ImmBorrow +}; +let a = t.0; // ERROR: cannot move out of `t.0` because it is borrowed +c(); +``` + +but the above works if it is not packed since it captures the field precisely: + +```rust +struct T(String, String); + +let mut t = T(String::new(), String::new()); +let c = || { + let a = std::ptr::addr_of!(t.1); // captures `t.1` with ImmBorrow +}; +// The move here is allowed. +let a = t.0; +c(); +``` + +[undefined behavior]: ../behavior-considered-undefined.md +[the `packed` representation]: ../type-layout.md#the-alignment-modifiers + +### `Box` vs other `Deref` implementations + +r[type.closure.capture.precision.box-deref] +The implementation of the [`Deref`] trait for [`Box`] is treated differently from other `Deref` implementations, as it is considered a special entity. + +For example, let us look at examples involving `Rc` and `Box`. The `*rc` is desugared to a call to the trait method `deref` defined on `Rc`, but since `*box` is treated differently, it is possible to do a precise capture of the contents of the `Box`. + +[`Box`]: ../special-types-and-traits.md#boxt +[`Deref`]: ../special-types-and-traits.md#deref-and-derefmut + +#### `Box` with non-`move` closure + +r[type.closure.capture.precision.box-non-move.not-moved] +In a non-`move` closure, if the contents of the `Box` are not moved into the closure body, the contents of the `Box` are precisely captured. + +```rust +struct S(String); + +let b = Box::new(S(String::new())); +let c_box = || { + let x = &(*b).0; // captures `(*b).0` by ImmBorrow +}; +c_box(); + +// Contrast `Box` with another type that implements Deref: +let r = std::rc::Rc::new(S(String::new())); +let c_rc = || { + let x = &(*r).0; // captures `r` by ImmBorrow +}; +c_rc(); +``` + +r[type.closure.capture.precision.box-non-move.moved] +However, if the contents of the `Box` are moved into the closure, then the box is entirely captured. This is done so the amount of data that needs to be moved into the closure is minimized. + +```rust +// This is the same as the example above except the closure +// moves the value instead of taking a reference to it. + +struct S(String); + +let b = Box::new(S(String::new())); +let c_box = || { + let x = (*b).0; // captures `b` with ByValue +}; +c_box(); +``` + +#### `Box` with move closure + +r[type.closure.capture.precision.box-move.read] +Similarly to moving contents of a `Box` in a non-`move` closure, reading the contents of a `Box` in a `move` closure will capture the `Box` entirely. + +```rust +struct S(i32); + +let b = Box::new(S(10)); +let c_box = move || { + let x = (*b).0; // captures `b` with ByValue +}; +``` ## Unique immutable borrows in captures r[type.closure.unique-immutable] - -Captures can occur by a special kind of borrow called a _unique immutable -borrow_, which cannot be used anywhere else in the language and cannot be -written out explicitly. It occurs when modifying the referent of a mutable -reference, as in the following example: +Captures can occur by a special kind of borrow called a _unique immutable borrow_, +which cannot be used anywhere else in the language and cannot be written out explicitly. +It occurs when modifying the referent of a mutable reference, as in the following example: ```rust let mut b = false; let x = &mut b; -{ - let mut c = || { *x = true; }; - // The following line is an error: - // let y = &x; - c(); -} +let mut c = || { + // An ImmBorrow and a MutBorrow of `x`. + let a = &x; + *x = true; // `x` captured by UniqueImmBorrow +}; +// The following line is an error: +// let y = &x; +c(); +// However, the following is OK. let z = &x; ``` In this case, borrowing `x` mutably is not possible, because `x` is not `mut`. But at the same time, borrowing `x` immutably would make the assignment illegal, -because a `& &mut` reference might not be unique, so it cannot safely be used to -modify a value. So a unique immutable borrow is used: it borrows `x` immutably, -but like a mutable borrow, it must be unique. In the above example, uncommenting -the declaration of `y` will produce an error because it would violate the -uniqueness of the closure's borrow of `x`; the declaration of z is valid because -the closure's lifetime has expired at the end of the block, releasing the borrow. +because a `& &mut` reference might not be unique, so it cannot safely be used to modify a value. +So a unique immutable borrow is used: it borrows `x` immutably, but like a mutable borrow, it must be unique. + +In the above example, uncommenting the declaration of `y` will produce an error because it would violate the uniqueness of the closure's borrow of `x`; the declaration of z is valid because the closure's lifetime has expired at the end of the block, releasing the borrow. ## Call traits and coercions @@ -176,12 +520,13 @@ following traits if allowed to do so by the types of the captures it stores: r[type.closure.traits.behavior] The rules for [`Send`] and [`Sync`] match those for normal struct types, while [`Clone`] and [`Copy`] behave as if [derived]. For [`Clone`], the order of -cloning of the captured variables is left unspecified. +cloning of the captured values is left unspecified. + Because captures are often by reference, the following general rules arise: -* A closure is [`Sync`] if all captured variables are [`Sync`]. -* A closure is [`Send`] if all variables captured by non-unique immutable +* A closure is [`Sync`] if all captured values are [`Sync`]. +* A closure is [`Send`] if all values captured by non-unique immutable reference are [`Sync`], and all values captured by unique immutable or mutable reference, copy, or move are [`Send`]. * A closure is [`Clone`] or [`Copy`] if it does not capture any values by @@ -195,3 +540,101 @@ Because captures are often by reference, the following general rules arise: [`Sync`]: ../special-types-and-traits.md#sync [closure expression]: ../expressions/closure-expr.md [derived]: ../attributes/derive.md + +## Drop Order + +r[type.closure.drop-order] +If a closure captures a field of a composite types such as structs, tuples, and enums by value, the field's lifetime would now be tied to the closure. As a result, it is possible for disjoint fields of a composite types to be dropped at different times. + +```rust +{ + let tuple = + (String::from("foo"), String::from("bar")); // --+ + { // | + let c = || { // ----------------------------+ | + // tuple.0 is captured into the closure | | + drop(tuple.0); // | | + }; // | | + } // 'c' and 'tuple.0' dropped here ------------+ | +} // tuple.1 dropped here -----------------------------+ +``` + +## Edition 2018 and before + +### Closure types difference + +r[type.closure.capture.precision.edition2018.entirety] +In Edition 2018 and before, closures always capture a variable in its entirety, without its precise capture path. This means that for the example used in the [Closure types](#closure-types) section, the generated closure type would instead look something like this: + + +```rust,ignore +struct Closure<'a> { + rect : &'a mut Rectangle, +} + +impl<'a> FnOnce<()> for Closure<'a> { + type Output = String; + extern "rust-call" fn call_once(self, args: ()) -> String { + self.rect.left_top.x += 1; + self.rect.right_bottom.x += 1; + format!("{:?}", self.rect.left_top) + } +} +``` + +and the call to `f` would work as follows: + + +```rust,ignore +f(Closure { rect: rect }); +``` + +### Capture precision difference + +r[type.closure.capture.precision.edition2018.composite] +Composite types such as structs, tuples, and enums are always captured in its entirety, +not by individual fields. As a result, it may be necessary to borrow into a local variable in order to capture a single field: + +```rust +# use std::collections::HashSet; +# +struct SetVec { + set: HashSet, + vec: Vec +} + +impl SetVec { + fn populate(&mut self) { + let vec = &mut self.vec; + self.set.iter().for_each(|&n| { + vec.push(n); + }) + } +} +``` + +If, instead, the closure were to use `self.vec` directly, then it would attempt to capture `self` by mutable reference. But since `self.set` is already borrowed to iterate over, the code would not compile. + +r[type.closure.capture.precision.edition2018.move] +If the `move` keyword is used, then all captures are by move or, for `Copy` types, by copy, regardless of whether a borrow would work. The `move` keyword is usually used to allow the closure to outlive the captured values, such as if the closure is being returned or used to spawn a new thread. + +r[type.closure.capture.precision.edition2018.wildcard] +Regardless of if the data will be read by the closure, i.e. in case of wild card patterns, if a variable defined outside the closure is mentioned within the closure the variable will be captured in its entirety. + +### Drop order difference + +r[type.closure.capture.precision.edition2018.drop-order] +As composite types are captured in their entirety, a closure which captures one of those composite types by value would drop the entire captured variable at the same time as the closure gets dropped. + +```rust +{ + let tuple = + (String::from("foo"), String::from("bar")); + { + let c = || { // --------------------------+ + // tuple is captured into the closure | + drop(tuple.0); // | + }; // | + } // 'c' and 'tuple' dropped here ------------+ +} +```