Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

WIP - implementing finally at the pipeline level 🤞🏼💥😱 #2437

Closed
wants to merge 1 commit into from

Conversation

pritidesai
Copy link
Member

@pritidesai pritidesai commented Apr 18, 2020

Changes

We can now specify a list of tasks that needs to be executed just before
pipeline exits (either after finishing all non-final tasks successfully or after
a single failure)

Most common final tasks could be (1) report test results, (2) cleanup cluster resources, etc

apiVersion: tekton.dev/v1beta1
kind: Pipeline
metadata:
  name: pipeline-with-final-tasks
spec:
  tasks:
    - name: pre-work
      taskRef:
        Name: some-pre-work
    - name: unit-test
      taskRef:
        Name: run-unit-test
      runAfter:
        - pre-work
    - name: integration-test
      taskRef:
        Name: run-integration-test
      runAfter:
        - unit-test
  finally:
    - name: cleanup-test
      taskRef:
        Name: cleanup-cluster
    - name: report-results
      taskRef:
        Name: report-test-results

Design doc: https://docs.google.com/document/d/1lxpYQHppiWOxsn4arqbwAFDo4T0-LCqpNa6p-TJdHrw/edit#

Part of #1684
Fixes #2446

Depends on: #2504

Submitter Checklist

These are the criteria that every PR should meet, please check them off as you
review them:

See the contribution guide for more details.

Double check this list of stuff that's easy to miss:

Reviewer Notes

If API changes are included, additive changes must be approved by at least two OWNERS and backwards incompatible changes must be approved by more than 50% of the OWNERS, and they must first be added in a backwards compatible way.

Release Notes

Users can now specify Tasks within a Pipeline that will always execute, even if Tasks fail, via the new `finally` clause

@tekton-robot tekton-robot added the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label Apr 18, 2020
@googlebot googlebot added the cla: yes Trying to make the CLA bot happy with ppl from different companies work on one commit label Apr 18, 2020
@tekton-robot
Copy link
Collaborator

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by:
To complete the pull request process, please assign vdemeester
You can assign the PR to them by writing /assign @vdemeester in a comment when ready.

The full list of commands accepted by this bot can be found here.

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@tekton-robot tekton-robot added the size/XXL Denotes a PR that changes 1000+ lines, ignoring generated files. label Apr 18, 2020
@tekton-robot
Copy link
Collaborator

The following is the coverage report on pkg/.
Say /test pull-tekton-pipeline-go-coverage to re-run this coverage report

File Old Coverage New Coverage Delta
pkg/apis/pipeline/v1alpha1/pipeline_conversion.go 96.3% 64.3% -32.0
pkg/apis/pipeline/v1alpha1/pipeline_validation.go 96.9% 90.1% -6.8
test/builder/pipeline.go 83.0% 77.6% -5.4

@bobcatfish
Copy link
Collaborator

Hey @pritidesai ! Very excited to see this moving along!! :D :D

I'm going to update the PR description to include the release notes, design doc and related issue.

Is it possible to include the docs on how to use this in the PR? This makes reviewing the PR waaaay easier, otherwise one has to read the code to understand how this is intended to work

@linux-foundation-easycla
Copy link

linux-foundation-easycla bot commented Apr 21, 2020

CLA Check
The committers are authorized under a signed CLA.

@tekton-robot tekton-robot added size/XL Denotes a PR that changes 500-999 lines, ignoring generated files. and removed size/XXL Denotes a PR that changes 1000+ lines, ignoring generated files. labels Apr 21, 2020
@tekton-robot tekton-robot added size/XXL Denotes a PR that changes 1000+ lines, ignoring generated files. and removed size/XL Denotes a PR that changes 500-999 lines, ignoring generated files. labels Apr 21, 2020
@pritidesai pritidesai force-pushed the finally-4-16 branch 5 times, most recently from 06d4d8c to d6602c6 Compare April 21, 2020 21:33
@tekton-robot
Copy link
Collaborator

The following is the coverage report on pkg/.
Say /test pull-tekton-pipeline-go-coverage to re-run this coverage report

File Old Coverage New Coverage Delta
pkg/apis/pipeline/v1alpha1/pipeline_conversion.go 96.3% 93.8% -2.5
pkg/apis/pipeline/v1alpha1/pipeline_validation.go 96.9% 90.0% -6.9
pkg/apis/pipeline/v1beta1/pipeline_defaults.go 33.3% 28.6% -4.8
pkg/apis/pipeline/v1beta1/pipeline_validation.go 92.2% 89.8% -2.4

@tekton-robot
Copy link
Collaborator

The following is the coverage report on pkg/.
Say /test pull-tekton-pipeline-go-coverage to re-run this coverage report

File Old Coverage New Coverage Delta
pkg/apis/pipeline/v1alpha1/pipeline_conversion.go 96.3% 93.8% -2.5
pkg/apis/pipeline/v1alpha1/pipeline_validation.go 96.9% 90.0% -6.9
pkg/apis/pipeline/v1beta1/pipeline_defaults.go 33.3% 28.6% -4.8
pkg/apis/pipeline/v1beta1/pipeline_validation.go 92.2% 89.8% -2.4

@tekton-robot
Copy link
Collaborator

The following is the coverage report on pkg/.
Say /test pull-tekton-pipeline-go-coverage to re-run this coverage report

File Old Coverage New Coverage Delta
pkg/apis/pipeline/v1alpha1/pipeline_conversion.go 96.3% 93.8% -2.5
pkg/apis/pipeline/v1alpha1/pipeline_validation.go 96.9% 90.0% -6.9
pkg/apis/pipeline/v1beta1/pipeline_defaults.go 33.3% 28.6% -4.8
pkg/apis/pipeline/v1beta1/pipeline_validation.go 92.2% 89.8% -2.4

@tekton-robot
Copy link
Collaborator

The following is the coverage report on pkg/.
Say /test pull-tekton-pipeline-go-coverage to re-run this coverage report

File Old Coverage New Coverage Delta
pkg/apis/pipeline/v1alpha1/pipeline_conversion.go 96.4% 93.9% -2.5
pkg/apis/pipeline/v1alpha1/pipeline_validation.go 96.9% 90.0% -6.9
pkg/apis/pipeline/v1beta1/pipeline_defaults.go 33.3% 28.6% -4.8
pkg/apis/pipeline/v1beta1/pipeline_validation.go 92.2% 89.8% -2.4

@tekton-robot
Copy link
Collaborator

The following is the coverage report on the affected files.
Say /test pull-tekton-pipeline-go-coverage to re-run this coverage report

File Old Coverage New Coverage Delta
pkg/apis/pipeline/v1alpha1/pipeline_conversion.go 96.4% 95.5% -1.0
pkg/apis/pipeline/v1alpha1/pipeline_validation.go 96.9% 95.4% -1.5
pkg/apis/pipeline/v1beta1/pipeline_defaults.go 33.3% 28.6% -4.8
pkg/apis/pipeline/v1beta1/pipeline_validation.go 91.7% 91.0% -0.6
pkg/reconciler/pipelinerun/pipelinerun.go 72.4% 65.2% -7.2
pkg/reconciler/pipelinerun/resources/pipelinerunresolution.go 90.2% 89.5% -0.8
pkg/reconciler/pipelinerun/resources/resultrefresolution.go 75.6% 75.9% 0.3
test/builder/pipeline.go 83.6% 80.4% -3.2

@pritidesai
Copy link
Member Author

/test pull-tekton-pipeline-build-tests

@pritidesai pritidesai changed the title wip - implementing finally at the pipeline level 🤞🏼 implementing finally at the pipeline level 🤞🏼 Apr 29, 2020
@tekton-robot tekton-robot removed the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label Apr 29, 2020
@pritidesai
Copy link
Member Author

this is ready for review now, thanks 🙏

@imjasonh
Copy link
Member

/hold

There's still some ongoing discussion in the design doc that I'd like to see resolved before we merge this.

@tekton-robot tekton-robot added the do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. label Apr 29, 2020
Copy link
Member

@vdemeester vdemeester left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Few comments 👼

docs/pipelines.md Show resolved Hide resolved
docs/pipelines.md Outdated Show resolved Hide resolved
docs/pipelines.md Outdated Show resolved Hide resolved
docs/pipelines.md Outdated Show resolved Hide resolved
docs/pipelines.md Outdated Show resolved Hide resolved
@pritidesai pritidesai changed the title implementing finally at the pipeline level 🤞🏼 implementing finally at the pipeline level 🤞🏼💥😱 May 1, 2020
@tekton-robot
Copy link
Collaborator

The following is the coverage report on the affected files.
Say /test pull-tekton-pipeline-go-coverage to re-run this coverage report

File Old Coverage New Coverage Delta
pkg/apis/pipeline/v1alpha1/pipeline_conversion.go 96.4% 95.5% -1.0
pkg/apis/pipeline/v1alpha1/pipeline_validation.go 96.9% 95.4% -1.5
pkg/apis/pipeline/v1beta1/pipeline_defaults.go 33.3% 28.6% -4.8
pkg/apis/pipeline/v1beta1/pipeline_validation.go 91.7% 91.0% -0.6
pkg/reconciler/pipelinerun/pipelinerun.go 72.4% 65.2% -7.2
pkg/reconciler/pipelinerun/resources/pipelinerunresolution.go 90.2% 89.6% -0.6
pkg/reconciler/pipelinerun/resources/resultrefresolution.go 75.6% 75.9% 0.3
test/builder/pipeline.go 83.6% 80.4% -3.2

We can now specify a list of tasks needs to be executed just before
pipeline exits (either after finishing all non-final tasks successfully or after
a single failure)

Most useful for tasks such as report test results, cleanup cluster resources, etc

```
apiVersion: tekton.dev/v1beta1
kind: Pipeline
metadata:
  name: pipeline-with-final-tasks
spec:
  tasks:
    - name: pre-work
      taskRef:
        Name: some-pre-work
    - name: unit-test
      taskRef:
        Name: run-unit-test
      runAfter:
        - pre-work
    - name: integration-test
      taskRef:
        Name: run-integration-test
      runAfter:
        - unit-test
  finally:
    - name: cleanup-test
      taskRef:
        Name: cleanup-cluster
    - name: report-results
      taskRef:
        Name: report-test-results
```
@tekton-robot
Copy link
Collaborator

The following is the coverage report on the affected files.
Say /test pull-tekton-pipeline-go-coverage to re-run this coverage report

File Old Coverage New Coverage Delta
pkg/apis/pipeline/v1alpha1/pipeline_conversion.go 96.4% 95.5% -1.0
pkg/apis/pipeline/v1alpha1/pipeline_validation.go 96.9% 95.4% -1.5
pkg/apis/pipeline/v1beta1/pipeline_defaults.go 33.3% 28.6% -4.8
pkg/apis/pipeline/v1beta1/pipeline_validation.go 91.7% 91.0% -0.6
pkg/reconciler/pipelinerun/pipelinerun.go 72.4% 65.2% -7.2
pkg/reconciler/pipelinerun/resources/pipelinerunresolution.go 90.2% 89.6% -0.7
pkg/reconciler/pipelinerun/resources/resultrefresolution.go 75.6% 75.9% 0.3
test/builder/pipeline.go 83.6% 80.4% -3.2

@pritidesai
Copy link
Member Author

Few comments 👼

thanks @vdemeester I have addressed all of your comments, let me know if any more changes needed 🙏

Copy link
Collaborator

@bobcatfish bobcatfish left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is looking great! I think we're all almost totally agreed about the feature itself so most of my comments are about the implementation.

Main themes:

  • I'm not sure about allowing "from" in finally tasks, feels a bit confusing to me when we don't allow results (e.g. imagine a git resource that is "from" a task that builds - if the build task executes, the data on disc for the resource may be different than if it doesnt)
  • I think its worth trying to avoid duplicating the validation logic as much as possible, since it leads to some nasty bugs, for example we ran into that with results (Fix 3 bugs with results #2471) - we're still duplicating some logic there so it might not be completely avoidable, but if we can avoid it its worth the effort
  • If we get the structure of PipelineRunState right, i bet we can avoid some of the duplicated code in the reconciler
  • Better to add functions in separate packages than add more functions to the reconciler
  • Is it possible for the logic that determines what runs next to become aware of finally? If the logic that returns the next tasks to run can handle Finally tasks, i bet we need a lot less special casing for finally

It looks like the coverage generally went down, and we want it to generally go up or stay the same so it might be worth digging into that a bit once we're more sure about the details of the implementation:

image

docs/pipelines.md Show resolved Hide resolved
docs/pipelines.md Show resolved Hide resolved
docs/pipelines.md Show resolved Hide resolved
| one or more `PipelineTask` skipped and rest successful | all final tasks successful | `true` | `Completed` |
| one or more `PipelineTask` skipped and rest successful | one or more failure of final tasks | `false` | `Failed` |
| single failure of `PipelineTask` | all final tasks successful | `false` | `failed` |
| single failure of `PipelineTask` | one or more failure of final tasks | `false` | `failed` |
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nice! 🎉 thanks for making this explicit :D

pkg/reconciler/pipelinerun/pipelinerun.go Show resolved Hide resolved
pkg/reconciler/pipelinerun/pipelinerun.go Show resolved Hide resolved
pkg/reconciler/pipelinerun/pipelinerun.go Show resolved Hide resolved
pkg/reconciler/pipelinerun/pipelinerun.go Show resolved Hide resolved
pkg/reconciler/pipelinerun/pipelinerun.go Show resolved Hide resolved
@bobcatfish
Copy link
Collaborator

I was just also thinking about ways to make incremental progress on this - I wonder if it would work to have a PR to add the types and their validation before adding the functionality.

@pritidesai
Copy link
Member Author

pritidesai commented May 20, 2020

I have split this PR into two separate wip PRs:

(1) adding finally type and its validation - PR #2650
(2) implementing this new feature - PR #2661

will close this PR once all the comments from this PR are addressed in new PRs.

@ghost
Copy link

ghost commented May 21, 2020

/kind feature

@tekton-robot tekton-robot added the kind/feature Categorizes issue or PR as related to a new feature. label May 21, 2020
@pritidesai pritidesai changed the title implementing finally at the pipeline level 🤞🏼💥😱 hold - implementing finally at the pipeline level 🤞🏼💥😱 Jun 1, 2020
@pritidesai pritidesai changed the title hold - implementing finally at the pipeline level 🤞🏼💥😱 WIP - implementing finally at the pipeline level 🤞🏼💥😱 Jun 1, 2020
@tekton-robot tekton-robot added the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label Jun 1, 2020
@pritidesai
Copy link
Member Author

I have addressed all the comments from this PR. Please review the two (new) separate PRs:

Closing this PR 👋

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
cla: yes Trying to make the CLA bot happy with ppl from different companies work on one commit do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. kind/feature Categorizes issue or PR as related to a new feature. size/XXL Denotes a PR that changes 1000+ lines, ignoring generated files.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

"Finally" tasks for Pipeline
7 participants