From f1d32822c950f50b8539eb726224c7f1d9f28a77 Mon Sep 17 00:00:00 2001 From: Demi Obenour Date: Tue, 7 Feb 2017 09:57:54 -0500 Subject: [PATCH 1/7] RFC for proper tail calls --- 0000-template.md | 143 ++++++++++++++++++++++++++++++++++++++++++----- 1 file changed, 130 insertions(+), 13 deletions(-) diff --git a/0000-template.md b/0000-template.md index ef898e3360a..f42017f2a6b 100644 --- a/0000-template.md +++ b/0000-template.md @@ -6,42 +6,159 @@ # Summary [summary]: #summary -One para explanation of the feature. +This RFC adds explicit proper tail calls to Rust via the `become` keyword. +It also specifies the semantics of proper tail calls and their interaction +with destructors. # Motivation [motivation]: #motivation -Why are we doing this? What use cases does it support? What is the expected outcome? +Proper tail calls are commonly found in functional languages, such as OCaml, +SML, Haskell, Scheme, and F#. They also can be used to easily encode state +machines and jump tables. Furthermore, they allow for code to be written in +continuation-passing style, which may be useful for some functional programming +patterns. # Detailed design [design]: #detailed-design -This is the bulk of the RFC. Explain the design in enough detail for somebody familiar -with the language to understand, and for somebody familiar with the compiler to implement. -This should get into specifics and corner-cases, and include examples of how the feature is used. +## Syntax +[syntax]: #syntax + +A proper tail call is written using the `become` keyword. This is already +reserved, so there is no backwards-compatibility break. + +The `become` keyword must be followed by a function or method calls. +Invocations of overloaded operators with at least one non-primitive argument +were considered as valid targets, but were rejected on grounds of being too +error-prone. In any case, these can still be called as methods. + +A future RFC may allow `become` in more places. + +## Type checking +[typechecking]: #typechecking +A `become` statement is type-checked exactly like a `return` statement. In the +current implementation, the syntactic restrictions on `become` noted above are +enforced during this phase. Additionally, the callee **must** use either the +`rust` calling convention or the `rust-call` calling convention. + +## Borrowchecking and Runtime Semantics +[semantics]: #semantics + +A `become` expression acts as if the following events occurred in-order: + +1. All variables that are being passed by-value are moved to temporary storage. +2. All local variables in the caller are destroyed according to usual Rust + semantics. Destructors are called where necessary. Note that values + moved from in step 1 are _not_ dropped. +3. The caller's stack frame is removed from the stack. +4. Control is transferred to the callee's entry point. + +This implies that it is invalid for any references into the caller's stack frame +to outlive the call. + +The borrow checker ensures that none of the above steps will result in the use +of a value that has gone out of scope. + +An implementation is required to support an unlimited number of proper tail +calls without exhausting any resources. + +## Implementation +[implementation]: #implementation + +The parser parses `become` exactly how it parses the `return` keyword. The +difference in semantics is handled later. + +During type checking, the following are checked: + +1. The target of the tail call is, in fact, a call. +2. The target of the tail call has the proper ABI. + +Later phases in the compiler assert that these requirements are met. + +New nodes are added in HIR and HAIR to correspond to `become`. In MIR, however, +a new flag is added to the `TerminatorKind::Call` varient. This flag is only +allowed to be set if all of the following are true: + +1. The destination is `RETURN_POINTER`. +2. There are no cleanups. +3. The basic block being branched into has length zero. +4. The basic block being branched into terminates with a return. + +Trans will assert that the above are in fact true. + +Finally, the use of proper tail calls must be propogated to LLVM. This is done +in two ways: + +1. Turn on tail call optimization. This is done by setting + `Options.GuaranteedTailCallOpt` in + [PassWrapper.cpp](src/rustllvm/PassWrapper.cpp). +2. Make the actual call a tail call. This is done by means of the following + function, added to [RustWrapper.cpp](src/rustllvm/RustWrapper.cpp): + + ```c++ + extern "C" void LLVMRustSetTailCall(LLVMValueRef Instr) { + CallInst *Call = cast(unwrap(Instr)); + Call->setTailCall(); + } + ``` # How We Teach This [how-we-teach-this]: #how-we-teach-this -What names and terminology work best for these concepts and why? -How is this idea best presented—as a continuation of existing Rust patterns, or as a wholly new one? +Tail calls are essentially disciplined cross-function `goto` – a unidirectional +transfer of control. They are a wholly new pattern for Rust, and make others +possible, such as continuation-passing style. -Would the acceptance of this proposal change how Rust is taught to new users at any level? -How should this feature be introduced and taught to existing Rust users? +Nevertheless, tail calls are an advanced feature that can produce code which is +difficult to debug. As such, they should only be used when other techniques are +not suitable. -What additions or changes to the Rust Reference, _The Rust Programming Language_, and/or _Rust by Example_ does it entail? +I believe that the first three paragraphs under #detailed-design could be added +to the Rust Reference with only minor changes. New material containing some +use cases would need to be added to _The Rust Programming Language_ and +_Rust by Example_. # Drawbacks [drawbacks]: #drawbacks -Why should we *not* do this? +## Runtime overhead +[runtime overhead]: #runtime-overhead + +One major drawback of proper tail calls is that their current implementation in +LLVM is not zero-cost: it forces a callee-pops calling convention, and thus +causes a stack adjustment after each non-tail call. This could be a performance +penalty. + +## Portability +[portability]: #portability + +An even greater drawback of proper tail calls is lack of cross-platform support: +LLVM does not support proper tail calls when targeting MIPS or WebAssembly, and +a compiler that generated C code would be hard-pressed to support them. While +relying on sibling call optimization in the C compiler might be possible with +whole-program compilation, it would still be tricky. WebAssembly does not +support tail calls at all yet, so stablization of this feature will need to wait +until this changes, which could take years. + +In fact, this is such a drawback that I (Demi Marie Obenour) considered not +making this RFC at all. Rust language features (as opposed to library features) +should work everywhere. That this does not is unfortunate. + +## Debugability +[debugability]: #debugability + +Proper tail calls can make debugging difficult, by overwriting stack frames. # Alternatives [alternatives]: #alternatives -What other designs have been considered? What is the impact of not doing this? +Proper tail calls are not necessary. Rust has done fine without them, and will +continue to do so if this RFC is not accepted. # Unresolved questions [unresolved]: #unresolved-questions -What parts of the design are still TBD? +Is there a way to emulate proper tail calls when compiling to targets that don't +support them? Is it possible to eliminate the overhead imposed on every +non-tail call? From 7fbf2c10ba98c1b99d865d5b50ec247ba25e32ea Mon Sep 17 00:00:00 2001 From: Demi Obenour Date: Tue, 7 Feb 2017 09:59:54 -0500 Subject: [PATCH 2/7] Provide link to implementation --- 0000-template.md | 3 +++ 1 file changed, 3 insertions(+) diff --git a/0000-template.md b/0000-template.md index f42017f2a6b..40886b2fcef 100644 --- a/0000-template.md +++ b/0000-template.md @@ -66,6 +66,9 @@ calls without exhausting any resources. ## Implementation [implementation]: #implementation +A current, mostly-functioning implementation can be found at +[DemiMarie/rust](/DemiMarie/rust). + The parser parses `become` exactly how it parses the `return` keyword. The difference in semantics is handled later. From 55e405682617530b58ccb72f505a50e2fb35db06 Mon Sep 17 00:00:00 2001 From: Demi Obenour Date: Tue, 7 Feb 2017 10:02:37 -0500 Subject: [PATCH 3/7] Add feature name --- 0000-template.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/0000-template.md b/0000-template.md index 40886b2fcef..2bb7ddc025d 100644 --- a/0000-template.md +++ b/0000-template.md @@ -1,4 +1,4 @@ -- Feature Name: (fill me in with a unique ident, my_awesome_feature) +- Feature Name: proper_tail_calls - Start Date: (fill me in with today's date, YYYY-MM-DD) - RFC PR: (leave this empty) - Rust Issue: (leave this empty) @@ -67,7 +67,7 @@ calls without exhausting any resources. [implementation]: #implementation A current, mostly-functioning implementation can be found at -[DemiMarie/rust](/DemiMarie/rust). +[DemiMarie/rust/tree/explicit-tailcalls](/DemiMarie/rust/tree/explicit-tailcalls). The parser parses `become` exactly how it parses the `return` keyword. The difference in semantics is handled later. From 7f9fb23bda519741d07ba08ea093760610383b80 Mon Sep 17 00:00:00 2001 From: Demi Obenour Date: Tue, 7 Feb 2017 15:59:00 -0500 Subject: [PATCH 4/7] typo: varient is spelled variant --- 0000-template.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/0000-template.md b/0000-template.md index 2bb7ddc025d..63dd278ea50 100644 --- a/0000-template.md +++ b/0000-template.md @@ -80,7 +80,7 @@ During type checking, the following are checked: Later phases in the compiler assert that these requirements are met. New nodes are added in HIR and HAIR to correspond to `become`. In MIR, however, -a new flag is added to the `TerminatorKind::Call` varient. This flag is only +a new flag is added to the `TerminatorKind::Call` variant. This flag is only allowed to be set if all of the following are true: 1. The destination is `RETURN_POINTER`. From c7b83dc46bd92e2e30733485d053b1400ef40195 Mon Sep 17 00:00:00 2001 From: Demi Obenour Date: Tue, 7 Feb 2017 20:35:28 -0500 Subject: [PATCH 5/7] Create a new file with RFC instead of changing template --- 0000-proper-tail-calls.md | 167 ++++++++++++++++++++++++++++++++++++++ 0000-template.md | 148 ++++----------------------------- 2 files changed, 181 insertions(+), 134 deletions(-) create mode 100644 0000-proper-tail-calls.md diff --git a/0000-proper-tail-calls.md b/0000-proper-tail-calls.md new file mode 100644 index 00000000000..63dd278ea50 --- /dev/null +++ b/0000-proper-tail-calls.md @@ -0,0 +1,167 @@ +- Feature Name: proper_tail_calls +- Start Date: (fill me in with today's date, YYYY-MM-DD) +- RFC PR: (leave this empty) +- Rust Issue: (leave this empty) + +# Summary +[summary]: #summary + +This RFC adds explicit proper tail calls to Rust via the `become` keyword. +It also specifies the semantics of proper tail calls and their interaction +with destructors. + +# Motivation +[motivation]: #motivation + +Proper tail calls are commonly found in functional languages, such as OCaml, +SML, Haskell, Scheme, and F#. They also can be used to easily encode state +machines and jump tables. Furthermore, they allow for code to be written in +continuation-passing style, which may be useful for some functional programming +patterns. + +# Detailed design +[design]: #detailed-design + +## Syntax +[syntax]: #syntax + +A proper tail call is written using the `become` keyword. This is already +reserved, so there is no backwards-compatibility break. + +The `become` keyword must be followed by a function or method calls. +Invocations of overloaded operators with at least one non-primitive argument +were considered as valid targets, but were rejected on grounds of being too +error-prone. In any case, these can still be called as methods. + +A future RFC may allow `become` in more places. + +## Type checking +[typechecking]: #typechecking +A `become` statement is type-checked exactly like a `return` statement. In the +current implementation, the syntactic restrictions on `become` noted above are +enforced during this phase. Additionally, the callee **must** use either the +`rust` calling convention or the `rust-call` calling convention. + +## Borrowchecking and Runtime Semantics +[semantics]: #semantics + +A `become` expression acts as if the following events occurred in-order: + +1. All variables that are being passed by-value are moved to temporary storage. +2. All local variables in the caller are destroyed according to usual Rust + semantics. Destructors are called where necessary. Note that values + moved from in step 1 are _not_ dropped. +3. The caller's stack frame is removed from the stack. +4. Control is transferred to the callee's entry point. + +This implies that it is invalid for any references into the caller's stack frame +to outlive the call. + +The borrow checker ensures that none of the above steps will result in the use +of a value that has gone out of scope. + +An implementation is required to support an unlimited number of proper tail +calls without exhausting any resources. + +## Implementation +[implementation]: #implementation + +A current, mostly-functioning implementation can be found at +[DemiMarie/rust/tree/explicit-tailcalls](/DemiMarie/rust/tree/explicit-tailcalls). + +The parser parses `become` exactly how it parses the `return` keyword. The +difference in semantics is handled later. + +During type checking, the following are checked: + +1. The target of the tail call is, in fact, a call. +2. The target of the tail call has the proper ABI. + +Later phases in the compiler assert that these requirements are met. + +New nodes are added in HIR and HAIR to correspond to `become`. In MIR, however, +a new flag is added to the `TerminatorKind::Call` variant. This flag is only +allowed to be set if all of the following are true: + +1. The destination is `RETURN_POINTER`. +2. There are no cleanups. +3. The basic block being branched into has length zero. +4. The basic block being branched into terminates with a return. + +Trans will assert that the above are in fact true. + +Finally, the use of proper tail calls must be propogated to LLVM. This is done +in two ways: + +1. Turn on tail call optimization. This is done by setting + `Options.GuaranteedTailCallOpt` in + [PassWrapper.cpp](src/rustllvm/PassWrapper.cpp). +2. Make the actual call a tail call. This is done by means of the following + function, added to [RustWrapper.cpp](src/rustllvm/RustWrapper.cpp): + + ```c++ + extern "C" void LLVMRustSetTailCall(LLVMValueRef Instr) { + CallInst *Call = cast(unwrap(Instr)); + Call->setTailCall(); + } + ``` + +# How We Teach This +[how-we-teach-this]: #how-we-teach-this + +Tail calls are essentially disciplined cross-function `goto` – a unidirectional +transfer of control. They are a wholly new pattern for Rust, and make others +possible, such as continuation-passing style. + +Nevertheless, tail calls are an advanced feature that can produce code which is +difficult to debug. As such, they should only be used when other techniques are +not suitable. + +I believe that the first three paragraphs under #detailed-design could be added +to the Rust Reference with only minor changes. New material containing some +use cases would need to be added to _The Rust Programming Language_ and +_Rust by Example_. + +# Drawbacks +[drawbacks]: #drawbacks + +## Runtime overhead +[runtime overhead]: #runtime-overhead + +One major drawback of proper tail calls is that their current implementation in +LLVM is not zero-cost: it forces a callee-pops calling convention, and thus +causes a stack adjustment after each non-tail call. This could be a performance +penalty. + +## Portability +[portability]: #portability + +An even greater drawback of proper tail calls is lack of cross-platform support: +LLVM does not support proper tail calls when targeting MIPS or WebAssembly, and +a compiler that generated C code would be hard-pressed to support them. While +relying on sibling call optimization in the C compiler might be possible with +whole-program compilation, it would still be tricky. WebAssembly does not +support tail calls at all yet, so stablization of this feature will need to wait +until this changes, which could take years. + +In fact, this is such a drawback that I (Demi Marie Obenour) considered not +making this RFC at all. Rust language features (as opposed to library features) +should work everywhere. That this does not is unfortunate. + +## Debugability +[debugability]: #debugability + +Proper tail calls can make debugging difficult, by overwriting stack frames. + +# Alternatives +[alternatives]: #alternatives + +Proper tail calls are not necessary. Rust has done fine without them, and will +continue to do so if this RFC is not accepted. + +# Unresolved questions +[unresolved]: #unresolved-questions + +Is there a way to emulate proper tail calls when compiling to targets that don't +support them? Is it possible to eliminate the overhead imposed on every +non-tail call? diff --git a/0000-template.md b/0000-template.md index 63dd278ea50..ef898e3360a 100644 --- a/0000-template.md +++ b/0000-template.md @@ -1,4 +1,4 @@ -- Feature Name: proper_tail_calls +- Feature Name: (fill me in with a unique ident, my_awesome_feature) - Start Date: (fill me in with today's date, YYYY-MM-DD) - RFC PR: (leave this empty) - Rust Issue: (leave this empty) @@ -6,162 +6,42 @@ # Summary [summary]: #summary -This RFC adds explicit proper tail calls to Rust via the `become` keyword. -It also specifies the semantics of proper tail calls and their interaction -with destructors. +One para explanation of the feature. # Motivation [motivation]: #motivation -Proper tail calls are commonly found in functional languages, such as OCaml, -SML, Haskell, Scheme, and F#. They also can be used to easily encode state -machines and jump tables. Furthermore, they allow for code to be written in -continuation-passing style, which may be useful for some functional programming -patterns. +Why are we doing this? What use cases does it support? What is the expected outcome? # Detailed design [design]: #detailed-design -## Syntax -[syntax]: #syntax - -A proper tail call is written using the `become` keyword. This is already -reserved, so there is no backwards-compatibility break. - -The `become` keyword must be followed by a function or method calls. -Invocations of overloaded operators with at least one non-primitive argument -were considered as valid targets, but were rejected on grounds of being too -error-prone. In any case, these can still be called as methods. - -A future RFC may allow `become` in more places. - -## Type checking -[typechecking]: #typechecking -A `become` statement is type-checked exactly like a `return` statement. In the -current implementation, the syntactic restrictions on `become` noted above are -enforced during this phase. Additionally, the callee **must** use either the -`rust` calling convention or the `rust-call` calling convention. - -## Borrowchecking and Runtime Semantics -[semantics]: #semantics - -A `become` expression acts as if the following events occurred in-order: - -1. All variables that are being passed by-value are moved to temporary storage. -2. All local variables in the caller are destroyed according to usual Rust - semantics. Destructors are called where necessary. Note that values - moved from in step 1 are _not_ dropped. -3. The caller's stack frame is removed from the stack. -4. Control is transferred to the callee's entry point. - -This implies that it is invalid for any references into the caller's stack frame -to outlive the call. - -The borrow checker ensures that none of the above steps will result in the use -of a value that has gone out of scope. - -An implementation is required to support an unlimited number of proper tail -calls without exhausting any resources. - -## Implementation -[implementation]: #implementation - -A current, mostly-functioning implementation can be found at -[DemiMarie/rust/tree/explicit-tailcalls](/DemiMarie/rust/tree/explicit-tailcalls). - -The parser parses `become` exactly how it parses the `return` keyword. The -difference in semantics is handled later. - -During type checking, the following are checked: - -1. The target of the tail call is, in fact, a call. -2. The target of the tail call has the proper ABI. - -Later phases in the compiler assert that these requirements are met. - -New nodes are added in HIR and HAIR to correspond to `become`. In MIR, however, -a new flag is added to the `TerminatorKind::Call` variant. This flag is only -allowed to be set if all of the following are true: - -1. The destination is `RETURN_POINTER`. -2. There are no cleanups. -3. The basic block being branched into has length zero. -4. The basic block being branched into terminates with a return. - -Trans will assert that the above are in fact true. - -Finally, the use of proper tail calls must be propogated to LLVM. This is done -in two ways: - -1. Turn on tail call optimization. This is done by setting - `Options.GuaranteedTailCallOpt` in - [PassWrapper.cpp](src/rustllvm/PassWrapper.cpp). -2. Make the actual call a tail call. This is done by means of the following - function, added to [RustWrapper.cpp](src/rustllvm/RustWrapper.cpp): - - ```c++ - extern "C" void LLVMRustSetTailCall(LLVMValueRef Instr) { - CallInst *Call = cast(unwrap(Instr)); - Call->setTailCall(); - } - ``` +This is the bulk of the RFC. Explain the design in enough detail for somebody familiar +with the language to understand, and for somebody familiar with the compiler to implement. +This should get into specifics and corner-cases, and include examples of how the feature is used. # How We Teach This [how-we-teach-this]: #how-we-teach-this -Tail calls are essentially disciplined cross-function `goto` – a unidirectional -transfer of control. They are a wholly new pattern for Rust, and make others -possible, such as continuation-passing style. +What names and terminology work best for these concepts and why? +How is this idea best presented—as a continuation of existing Rust patterns, or as a wholly new one? -Nevertheless, tail calls are an advanced feature that can produce code which is -difficult to debug. As such, they should only be used when other techniques are -not suitable. +Would the acceptance of this proposal change how Rust is taught to new users at any level? +How should this feature be introduced and taught to existing Rust users? -I believe that the first three paragraphs under #detailed-design could be added -to the Rust Reference with only minor changes. New material containing some -use cases would need to be added to _The Rust Programming Language_ and -_Rust by Example_. +What additions or changes to the Rust Reference, _The Rust Programming Language_, and/or _Rust by Example_ does it entail? # Drawbacks [drawbacks]: #drawbacks -## Runtime overhead -[runtime overhead]: #runtime-overhead - -One major drawback of proper tail calls is that their current implementation in -LLVM is not zero-cost: it forces a callee-pops calling convention, and thus -causes a stack adjustment after each non-tail call. This could be a performance -penalty. - -## Portability -[portability]: #portability - -An even greater drawback of proper tail calls is lack of cross-platform support: -LLVM does not support proper tail calls when targeting MIPS or WebAssembly, and -a compiler that generated C code would be hard-pressed to support them. While -relying on sibling call optimization in the C compiler might be possible with -whole-program compilation, it would still be tricky. WebAssembly does not -support tail calls at all yet, so stablization of this feature will need to wait -until this changes, which could take years. - -In fact, this is such a drawback that I (Demi Marie Obenour) considered not -making this RFC at all. Rust language features (as opposed to library features) -should work everywhere. That this does not is unfortunate. - -## Debugability -[debugability]: #debugability - -Proper tail calls can make debugging difficult, by overwriting stack frames. +Why should we *not* do this? # Alternatives [alternatives]: #alternatives -Proper tail calls are not necessary. Rust has done fine without them, and will -continue to do so if this RFC is not accepted. +What other designs have been considered? What is the impact of not doing this? # Unresolved questions [unresolved]: #unresolved-questions -Is there a way to emulate proper tail calls when compiling to targets that don't -support them? Is it possible to eliminate the overhead imposed on every -non-tail call? +What parts of the design are still TBD? From 788fd7e25b9948a7615672df4bff2f904a711cc0 Mon Sep 17 00:00:00 2001 From: Demi Obenour Date: Tue, 7 Feb 2017 20:37:51 -0500 Subject: [PATCH 6/7] Add license --- 0000-proper-tail-calls.md | 5 +++++ 1 file changed, 5 insertions(+) diff --git a/0000-proper-tail-calls.md b/0000-proper-tail-calls.md index 63dd278ea50..c4ff7f4ba3e 100644 --- a/0000-proper-tail-calls.md +++ b/0000-proper-tail-calls.md @@ -165,3 +165,8 @@ continue to do so if this RFC is not accepted. Is there a way to emulate proper tail calls when compiling to targets that don't support them? Is it possible to eliminate the overhead imposed on every non-tail call? + +# License of this RFC +[license]: #license + +This RFC is licensed under the same terms as Rust itself. From 1e7d44859b5493109b47f2fc856977d45a6c4bf9 Mon Sep 17 00:00:00 2001 From: Jean-Marie Comets Date: Fri, 10 Feb 2017 12:43:28 +0100 Subject: [PATCH 7/7] Fix 404 link in RFC --- 0000-proper-tail-calls.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/0000-proper-tail-calls.md b/0000-proper-tail-calls.md index c4ff7f4ba3e..1036f52402d 100644 --- a/0000-proper-tail-calls.md +++ b/0000-proper-tail-calls.md @@ -67,7 +67,7 @@ calls without exhausting any resources. [implementation]: #implementation A current, mostly-functioning implementation can be found at -[DemiMarie/rust/tree/explicit-tailcalls](/DemiMarie/rust/tree/explicit-tailcalls). +[DemiMarie/rust/tree/explicit-tailcalls](https://github.com/DemiMarie/rust/tree/explicit-tailcalls). The parser parses `become` exactly how it parses the `return` keyword. The difference in semantics is handled later.