WhenExpressions in Finally Tasks

Users can guard execution of `Tasks` using `WhenExpressions`, but that is currently not supported in `Finally Tasks`. This change adds support for `WhenExpressions` in `Finally Tasks` not only to provide efficient guarded execution but also to improve the reusability of `Tasks` in `Finally`. The proposal is described further in the [WhenExpressions in Finally Tasks TEP](https://github.com/tektoncd/community/blob/master/teps/0045-whenexpressions-in-finally-tasks.md). Given we've recently added support for `Results` and `Status` in `Finally Tasks`, this is an opportune time to enable `WhenExpressions` in `Finally Tasks`.
tektoncd · Feb 4, 2021 · 62d8621 · 62d8621
1 parent 8c5a751
commit 62d8621
Show file tree

Hide file tree

Showing 12 changed files with 782 additions and 56 deletions.
diff --git a/docs/pipelines.md b/docs/pipelines.md
@@ -31,6 +31,10 @@ weight: 3
     - [Consuming `Task` execution results in `finally`](#consuming-task-execution-results-in-finally)
     - [`PipelineRun` Status with `finally`](#pipelinerun-status-with-finally)
     - [Using Execution `Status` of `pipelineTask`](#using-execution-status-of-pipelinetask)
+    - [Guard `Finally Task` execution using `WhenExpressions`](#guard-finally-task-execution-using-whenexpressions)
+      - [`WhenExpressions` using `Parameters` in `Finally Tasks`](#whenexpressions-using-parameters-in-finally-tasks)
+      - [`WhenExpressions` using `Results` in `Finally Tasks`](#whenexpressions-using-results-in-finally-tasks)
+      - [`WhenExpressions` using `Execution Status` of `PipelineTask` in `Finally Tasks`](#whenexpressions-using-execution-status-of-pipelinetask-in-finally-tasks)  
     - [Known Limitations](#known-limitations)
       - [Specifying `Resources` in Final Tasks](#specifying-resources-in-final-tasks)
       - [Cannot configure the Final Task execution order](#cannot-configure-the-final-task-execution-order)
@@ -895,6 +899,118 @@ This kind of variable can have any one of the values from the following table:
 
 For an end-to-end example, see [`status` in a `PipelineRun`](../examples/v1beta1/pipelineruns/pipelinerun-task-execution-status.yaml).
 
+### Guard `Finally Task` execution using `WhenExpressions`
+
+Similar to `Tasks`, `Finally Tasks` can be guarded using [`WhenExpressions`](#guard-task-execution-using-whenexpressions)
+that operate on static inputs or variables. Like in `Tasks`, `WhenExpressions` in `Finally Tasks` can operate on
+`Parameters` and `Results`. Unlike in `Tasks`, `WhenExpressions` in `Finally Tasks` can also operate on the [`Execution
+Status`](#using-execution-status-of-pipelinetask) of `Tasks`.
+
+#### `WhenExpressions` using `Parameters` in `Finally Tasks`
+
+`WhenExpressions` in `Finally Tasks` can utilize `Parameters` as demonstrated using [`golang-build`](https://github.com/tektoncd/catalog/tree/master/task/golang-build/0.1)
+and [`send-to-channel-slack`](https://github.com/tektoncd/catalog/tree/master/task/send-to-channel-slack/0.1) Catalog
+`Tasks`:
+
+```yaml
+apiVersion: tekton.dev/v1beta1
+kind: PipelineRun
+metadata:
+  generateName: pipelinerun-
+spec:
+  pipelineSpec:
+    params:
+    - name: enable-notifications
+      type: string
+      description: a boolean indicating whether the notifications should be sent
+    tasks:
+      - name: golang-build
+        taskRef:
+          name: golang-build
+      # […]
+    finally:
+      - name: notify-build-failure # executed only when build task fails and notifications are enabled
+        when:
+          - input: $(tasks.golang-build.status)
+            operator: in
+            values: ["Failed"]
+          - input: $(params.enable-notifications)
+            operator: in
+            values: ["true"]
+        taskRef:
+          name: send-to-slack-channel
+      # […]
+  params:
+    - name: enable-notifications
+      value: true
+```
+
+#### `WhenExpressions` using `Results` in `Finally Tasks`
+
+`WhenExpressions` in `Finally Tasks` can utilize `Results`, as demonstrated using [`git-clone`](https://github.com/tektoncd/catalog/tree/master/task/git-clone/0.2)
+and [`github-add-comment`](https://github.com/tektoncd/catalog/tree/master/task/github-add-comment/0.2) Catalog `Tasks`:
+
+```yaml
+apiVersion: tekton.dev/v1beta1
+kind: PipelineRun
+metadata:
+  generateName: pipelinerun-
+spec:
+  pipelineSpec:
+    tasks:
+      - name: git-clone
+        taskRef:
+          name: git-clone
+      - name: go-build
+      # […]
+    finally:
+      - name: notify-commit-sha # executed only when commit sha is not the expected sha
+        when:
+          - input: $(tasks.git-clone.results.commit)
+            operator: notin
+            values: [$(params.expected-sha)]
+        taskRef:
+          name: github-add-comment
+      # […]
+  params:
+    - name: expected-sha
+      value: 54dd3984affab47f3018852e61a1a6f9946ecfa
+```
+
+If the `WhenExpressions` in a `Finally Task` use `Results` from a skipped or failed non-finally `Tasks`, then the
+`Finally Task` would also be skipped and be included in the list of `Skipped Tasks` in the `Status`, [similarly to when using
+`Results` in other parts of the `Finally Task`](#consuming-task-execution-results-in-finally).
+
+#### `WhenExpressions` using `Execution Status` of `PipelineTask` in `Finally Tasks`
+
+`WhenExpressions` in `Finally Tasks` can utilize [`Execution Status` of `PipelineTasks`](#using-execution-status-of-pipelinetask), 
+as as demonstrated using [`golang-build`](https://github.com/tektoncd/catalog/tree/master/task/golang-build/0.1) and
+[`send-to-channel-slack`](https://github.com/tektoncd/catalog/tree/master/task/send-to-channel-slack/0.1) Catalog `Tasks`:
+
+```yaml
+apiVersion: tekton.dev/v1beta1
+kind: PipelineRun
+metadata:
+  generateName: pipelinerun-
+spec:
+  pipelineSpec:
+    tasks:
+      - name: golang-build
+        taskRef:
+          name: golang-build
+      # […]
+    finally:
+      - name: notify-build-failure # executed only when build task fails
+        when:
+          - input: $(tasks.golang-build.status)
+            operator: in
+            values: ["Failed"]
+        taskRef:
+          name: send-to-slack-channel
+      # […]
+```
+
+For an end-to-end example, see [PipelineRun with WhenExpressions](../examples/v1beta1/pipelineruns/pipelinerun-with-when-expressions.yaml).
 
 ### Known Limitations
 

diff --git a/examples/v1beta1/pipelineruns/pipelinerun-with-when-expressions.yaml b/examples/v1beta1/pipelineruns/pipelinerun-with-when-expressions.yaml
@@ -100,7 +100,47 @@ spec:
               image: ubuntu
               script: exit 1
     finally:
-      - name: do-something-finally
+      - name: finally-task-should-be-skipped-1 # when expression using execution status, evaluates to false
+        when:
+          - input: "$(tasks.echo-file-exists.status)"
+            operator: in
+            values: ["Failure"]
+        taskSpec:
+          steps:
+            - name: echo
+              image: ubuntu
+              script: exit 1
+      - name: finally-task-should-be-skipped-2 # when expression using task result, evaluates to false
+        when:
+          - input: "$(tasks.check-file.results.exists)"
+            operator: in
+            values: ["missing"]
+        taskSpec:
+          steps:
+            - name: echo
+              image: ubuntu
+              script: exit 1
+      - name: finally-task-should-be-skipped-3 # when expression using parameter, evaluates to false
+        when:
+          - input: "$(params.path)"
+            operator: notin
+            values: ["README.md"]
+        taskSpec:
+          steps:
+            - name: echo
+              image: ubuntu
+              script: exit 1
+      - name: finally-task-should-be-executed # when expression using execution status, param and results
+        when:
+          - input: "$(tasks.echo-file-exists.status)"
+            operator: in
+            values: ["Succeeded"]
+          - input: "$(tasks.check-file.results.exists)"
+            operator: in
+            values: ["yes"]
+          - input: "$(params.path)"
+            operator: in
+            values: ["README.md"]
         taskSpec:
           steps:
             - name: echo

diff --git a/pkg/apis/pipeline/v1beta1/pipeline_validation.go b/pkg/apis/pipeline/v1beta1/pipeline_validation.go
@@ -70,7 +70,7 @@ func (ps *PipelineSpec) Validate(ctx context.Context) (errs *apis.FieldError) {
 	errs = errs.Also(validatePipelineResults(ps.Results))
 	errs = errs.Also(validateTasksAndFinallySection(ps))
 	errs = errs.Also(validateFinalTasks(ps.Tasks, ps.Finally))
-	errs = errs.Also(validateWhenExpressions(ps.Tasks))
+	errs = errs.Also(validateWhenExpressions(ps.Tasks, ps.Finally))
 	return errs
 }
 
@@ -329,22 +329,32 @@ func validateExecutionStatusVariablesInFinally(tasks []PipelineTask, finally []P
 	ptNames := PipelineTaskList(tasks).Names()
 	for idx, t := range finally {
 		for _, param := range t.Params {
-			// retrieve a list of substitution expression from a param
-			if ps, ok := GetVarSubstitutionExpressionsForParam(param); ok {
-				// validate tasks.pipelineTask.status if this expression is not a result reference
-				if !LooksLikeContainsResultRefs(ps) {
-					for _, p := range ps {
-						// check if it contains context variable accessing execution status - $(tasks.taskname.status)
-						if containsExecutionStatusRef(p) {
-							// strip tasks. and .status from tasks.taskname.status to further verify task name
-							pt := strings.TrimSuffix(strings.TrimPrefix(p, "tasks."), ".status")
-							// report an error if the task name does not exist in the list of dag tasks
-							if !ptNames.Has(pt) {
-								errs = errs.Also(apis.ErrInvalidValue(fmt.Sprintf("pipeline task %s is not defined in the pipeline", pt),
-									"value").ViaFieldKey("params", param.Name).ViaFieldIndex("finally", idx))
-							}
-						}
-					}
+			if expressions, ok := GetVarSubstitutionExpressionsForParam(param); ok {
+				errs = errs.Also(validateExecutionStatusVariablesExpressions(expressions, ptNames, "value").ViaFieldKey(
+					"params", param.Name).ViaFieldIndex("finally", idx))
+			}
+		}
+		for i, we := range t.WhenExpressions {
+			if expressions, ok := we.GetVarSubstitutionExpressions(); ok {
+				errs = errs.Also(validateExecutionStatusVariablesExpressions(expressions, ptNames, "").ViaFieldIndex(
+					"when", i).ViaFieldIndex("finally", idx))
+			}
+		}
+	}
+	return errs
+}
+
+func validateExecutionStatusVariablesExpressions(expressions []string, ptNames sets.String, fieldPath string) (errs *apis.FieldError) {
+	// validate tasks.pipelineTask.status if this expression is not a result reference
+	if !LooksLikeContainsResultRefs(expressions) {
+		for _, expression := range expressions {
+			// check if it contains context variable accessing execution status - $(tasks.taskname.status)
+			if containsExecutionStatusRef(expression) {
+				// strip tasks. and .status from tasks.taskname.status to further verify task name
+				pt := strings.TrimSuffix(strings.TrimPrefix(expression, "tasks."), ".status")
+				// report an error if the task name does not exist in the list of dag tasks
+				if !ptNames.Has(pt) {
+					errs = errs.Also(apis.ErrInvalidValue(fmt.Sprintf("pipeline task %s is not defined in the pipeline", pt), fieldPath))
 				}
 			}
 		}
@@ -429,9 +439,6 @@ func validateFinalTasks(tasks []PipelineTask, finalTasks []PipelineTask) *apis.F
 		if len(f.Conditions) != 0 {
 			return apis.ErrInvalidValue(fmt.Sprintf("no conditions allowed under spec.finally, final task %s has conditions specified", f.Name), "").ViaFieldIndex("finally", idx)
 		}
-		if len(f.WhenExpressions) != 0 {
-			return apis.ErrInvalidValue(fmt.Sprintf("no when expressions allowed under spec.finally, final task %s has when expressions specified", f.Name), "").ViaFieldIndex("finally", idx)
-		}
 	}
 
 	ts := PipelineTaskList(tasks).Names()
@@ -487,11 +494,14 @@ func validateTasksInputFrom(tasks []PipelineTask) (errs *apis.FieldError) {
 	return errs
 }
 
-func validateWhenExpressions(tasks []PipelineTask) (errs *apis.FieldError) {
+func validateWhenExpressions(tasks []PipelineTask, finalTasks []PipelineTask) (errs *apis.FieldError) {
 	for i, t := range tasks {
 		errs = errs.Also(validateOneOfWhenExpressionsOrConditions(t).ViaFieldIndex("tasks", i))
 		errs = errs.Also(t.WhenExpressions.validate().ViaFieldIndex("tasks", i))
 	}
+	for i, t := range finalTasks {
+		errs = errs.Also(t.WhenExpressions.validate().ViaFieldIndex("finally", i))
+	}
 	return errs
 }