Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

an aggregate status of tasks in finally #3817

Merged
merged 1 commit into from
May 6, 2021

Conversation

pritidesai
Copy link
Member

@pritidesai pritidesai commented Mar 9, 2021

Changes

Implementing TEP-0049, it is now possible to access an aggregate execution status of all the tasks using $(tasks.status). This context variable is only available in a finally task.

/kind feature

Partially Closes #1020
Closes #3806
Implements TEP #0049

Submitter Checklist

These are the criteria that every PR should meet, please check them off as you
review them:

  • Includes tests (if functionality changed/added)
  • Includes docs (if user facing)
  • Commit messages follow commit message best practices
  • Release notes block has been filled in or deleted (only if no user facing changes)

See the contribution guide for more details.

Double check this list of stuff that's easy to miss:

Reviewer Notes

If API changes are included, additive changes must be approved by at least two OWNERS and backwards incompatible changes must be approved by more than 50% of the OWNERS, and they must first be added in a backwards compatible way.

Release Notes

Introducing a variable $(tasks.status) to access aggregate execution status of tasks in finally.

@tekton-robot tekton-robot added release-note Denotes a PR that will be considered when it comes time to generate release notes. kind/feature Categorizes issue or PR as related to a new feature. labels Mar 9, 2021
@tekton-robot tekton-robot requested review from afrittoli and dibyom March 9, 2021 06:33
@tekton-robot tekton-robot added the size/L Denotes a PR that changes 100-499 lines, ignoring generated files. label Mar 9, 2021
@tekton-robot
Copy link
Collaborator

The following is the coverage report on the affected files.
Say /test pull-tekton-pipeline-go-coverage to re-run this coverage report

File Old Coverage New Coverage Delta
pkg/apis/pipeline/v1beta1/pipeline_validation.go 99.6% 99.6% 0.0
pkg/reconciler/pipelinerun/resources/pipelinerunstate.go 69.8% 71.8% 2.0

@tekton-robot
Copy link
Collaborator

The following is the coverage report on the affected files.
Say /test pull-tekton-pipeline-go-coverage to re-run this coverage report

File Old Coverage New Coverage Delta
pkg/apis/pipeline/v1beta1/pipeline_validation.go 99.6% 99.6% 0.0
pkg/reconciler/pipelinerun/resources/pipelinerunstate.go 69.8% 71.8% 2.0

@jerop jerop self-assigned this Mar 9, 2021
@tekton-robot
Copy link
Collaborator

The following is the coverage report on the affected files.
Say /test pull-tekton-pipeline-go-coverage to re-run this coverage report

File Old Coverage New Coverage Delta
pkg/apis/pipeline/v1beta1/pipeline_validation.go 99.6% 99.6% 0.0
pkg/reconciler/pipelinerun/resources/pipelinerunstate.go 69.8% 71.7% 1.8

@tekton-robot
Copy link
Collaborator

The following is the coverage report on the affected files.
Say /test pull-tekton-pipeline-go-coverage to re-run this coverage report

File Old Coverage New Coverage Delta
pkg/apis/pipeline/v1beta1/pipeline_validation.go 99.6% 99.6% 0.0
pkg/reconciler/pipelinerun/resources/pipelinerunstate.go 69.8% 71.7% 1.8

@pritidesai pritidesai added this to the Pipelines 0.23 milestone Mar 9, 2021
Base automatically changed from master to main March 11, 2021 15:29
@tekton-robot
Copy link
Collaborator

The following is the coverage report on the affected files.
Say /test pull-tekton-pipeline-go-coverage to re-run this coverage report

File Old Coverage New Coverage Delta
pkg/apis/pipeline/v1beta1/pipeline_validation.go 99.6% 99.6% 0.0
pkg/reconciler/pipelinerun/resources/pipelinerunstate.go 69.8% 71.7% 1.8

docs/pipelines.md Outdated Show resolved Hide resolved
@tekton-robot
Copy link
Collaborator

The following is the coverage report on the affected files.
Say /test pull-tekton-pipeline-go-coverage to re-run this coverage report

File Old Coverage New Coverage Delta
pkg/reconciler/pipelinerun/resources/pipelinerunstate.go 69.8% 71.7% 1.8

@tekton-robot tekton-robot added the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Mar 31, 2021
@tekton-robot tekton-robot removed the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Apr 1, 2021
@tekton-robot
Copy link
Collaborator

The following is the coverage report on the affected files.
Say /test pull-tekton-pipeline-go-coverage to re-run this coverage report

File Old Coverage New Coverage Delta
pkg/reconciler/pipelinerun/resources/pipelinerunstate.go 69.8% 71.7% 1.8

@tekton-robot
Copy link
Collaborator

The following is the coverage report on the affected files.
Say /test pull-tekton-pipeline-go-coverage to re-run this coverage report

File Old Coverage New Coverage Delta
pkg/reconciler/pipelinerun/resources/pipelinerunstate.go 69.8% 71.7% 1.8

@pritidesai
Copy link
Member Author

pritidesai commented Apr 2, 2021

The TEP is merged now 🎉 in proposed state, have to create one more PR to mark it implementable. I might not get to it until 0.23 early next week. If If not, please feel free to push it to next release. 🙏

docs/pipelines.md Outdated Show resolved Hide resolved
docs/pipelines.md Outdated Show resolved Hide resolved
docs/variables.md Outdated Show resolved Hide resolved
@@ -412,6 +412,28 @@ func (facts *PipelineRunFacts) GetPipelineTaskStatus() map[string]string {
tStatus[PipelineTaskStatusPrefix+t.PipelineTask.Name+PipelineTaskStatusSuffix] = s
}
}
// initialize aggregate status of all dag tasks to None
aggregateStatus := PipelineTaskStateNone
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe we should skip adding this to the map completely if checkDAGTasksDone is false? I don't see a way to end up in a None state except where Pipelines has incorrectly decided to run finally before tasks is finished, which would be a bug, I think?

Copy link
Member Author

@pritidesai pritidesai Apr 27, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yup, there is no way to end up in a None state from finally. This is added mainly for the unit testing. Any better alternative please? 🙏

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah, ok. The only other option I guess would be not unit testing this if we don't think it ever happens. But I don't feel strongly either way so I think this is good!

@ghost ghost modified the milestones: Pipelines 0.23, Pipelines 0.24 Apr 5, 2021
@ghost
Copy link

ghost commented Apr 5, 2021

The TEP is merged now 🎉 in proposed state, have to create one more PR to mark it implementable. I might not get to it until 0.23 early next week. If If not, please feel free to push it to next release. 🙏

OK cool, moved this one to milestone 0.24, thanks for updating the PR with this context, very helpful!

@tekton-robot
Copy link
Collaborator

The following is the coverage report on the affected files.
Say /test pull-tekton-pipeline-go-coverage to re-run this coverage report

File Old Coverage New Coverage Delta
pkg/reconciler/pipelinerun/resources/pipelinerunstate.go 69.8% 71.7% 1.8

@ghost
Copy link

ghost commented Apr 29, 2021

/test tekton-pipeline-unit-tests

@ghost
Copy link

ghost commented Apr 29, 2021

/lgtm

@tekton-robot tekton-robot assigned ghost Apr 29, 2021
@tekton-robot tekton-robot added the lgtm Indicates that a PR is ready to be merged. label Apr 29, 2021
docs/pipelines.md Outdated Show resolved Hide resolved
@tekton-robot tekton-robot removed the lgtm Indicates that a PR is ready to be merged. label May 3, 2021
@tekton-robot
Copy link
Collaborator

The following is the coverage report on the affected files.
Say /test pull-tekton-pipeline-go-coverage to re-run this coverage report

File Old Coverage New Coverage Delta
pkg/reconciler/pipelinerun/resources/pipelinerunstate.go 69.8% 71.7% 1.8

@pritidesai
Copy link
Member Author

/test pull-tekton-pipeline-integration-tests

@ghost
Copy link

ghost commented May 4, 2021

/lgtm

@tekton-robot tekton-robot added the lgtm Indicates that a PR is ready to be merged. label May 4, 2021
Copy link
Member

@jerop jerop left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

looks great, just a small nit

thanks @pritidesai!

Implementing TEP-0049, it is now possible to access aggregate execution
status of all tasks using `$(tasks.status)`. This context variable is
only available in a finally task.
@tekton-robot tekton-robot removed the lgtm Indicates that a PR is ready to be merged. label May 5, 2021
@tekton-robot
Copy link
Collaborator

The following is the coverage report on the affected files.
Say /test pull-tekton-pipeline-go-coverage to re-run this coverage report

File Old Coverage New Coverage Delta
pkg/reconciler/pipelinerun/resources/pipelinerunstate.go 69.8% 71.7% 1.8

Copy link
Member

@jerop jerop left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🎉

thanks @pritidesai

@tekton-robot
Copy link
Collaborator

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: jerop

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@tekton-robot tekton-robot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label May 5, 2021
@ghost
Copy link

ghost commented May 6, 2021

/lgtm

@tekton-robot tekton-robot added the lgtm Indicates that a PR is ready to be merged. label May 6, 2021
@tekton-robot tekton-robot merged commit d3cc94e into tektoncd:main May 6, 2021
Copy link
Member

@afrittoli afrittoli left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Half-way through review - so far so good, thank you!!

docs/pipelines.md Outdated Show resolved Hide resolved
| `Completed` | all `tasks` completed successfully including one or more skipped tasks |
| `None` | no aggregate execution status available (i.e. none of the above), one or more `tasks` could be pending/running/cancelled/timedout |

For an end-to-end example, see [`$(tasks.status)` usage in a `Pipeline`](../examples/v1beta1/pipelineruns/pipelinerun-task-execution-status.yaml).
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍

@@ -24,7 +24,7 @@ For instructions on using variable substitutions see the relevant section of [th
| `context.pipelineRun.uid` | The uid of the `PipelineRun` that this `Pipeline` is running in. |
| `context.pipeline.name` | The name of this `Pipeline` . |
| `tasks.<pipelineTaskName>.status` | The execution status of the specified `pipelineTask`, only available in `finally` tasks. |

| `tasks.status` | An aggregate status of all `tasks`, only available in `finally` tasks. |
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

NIT: might be worth saying that this is an aggregated status of all tasks not in finally - just for 100% clarity :)

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

thanks @afrittoli have created a PR to fix this #3919

if !LooksLikeContainsResultRefs(ps) {
for _, p := range ps {
// check if it contains context variable accessing execution status - $(tasks.taskname.status)
// or an aggregate status - $(tasks.status)
if containsExecutionStatusRef(p) {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice, the same function catches the new variable too :)

jerop added a commit to jerop/community that referenced this pull request Jun 3, 2021
jerop added a commit to jerop/community that referenced this pull request Jun 3, 2021
jerop added a commit to jerop/community that referenced this pull request Jun 3, 2021
tekton-robot pushed a commit to tektoncd/community that referenced this pull request Jun 3, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
approved Indicates a PR has been approved by an approver from all required OWNERS files. kind/feature Categorizes issue or PR as related to a new feature. lgtm Indicates that a PR is ready to be merged. release-note Denotes a PR that will be considered when it comes time to generate release notes. size/L Denotes a PR that changes 100-499 lines, ignoring generated files.
Projects
None yet
5 participants