[TEP-0075] Support Object Results substitution #5083

Yongxuanzhang · 2022-07-05T21:31:22Z

Changes

This is part of work in TEP-0075.
This commit provides the support to apply object results replacements.
Previous this commit we support emitting object results so users can
write object results to task level, but we cannot pass object results from
one task to another within one pipeline. This commit adds the support for this.

Note that this commit does not support object and array results for Matrix

/kind feature

Submitter Checklist

As the author of this PR, please check off the items in this checklist:

Docs included if any changes are user facing
Tests included if any functionality added or changed
Follows the commit message standard
Meets the Tekton contributor standards (including
functionality, content, code)
Release notes block below has been filled in
(if there are no user facing changes, use release note "NONE")

Release Notes

Support object results substitution as an alpha feature.

A task can specify a type to produce array result, such as:
results:
  - name: object-results
     type: object
     description: The object results
     properties:
        foo: {
          type: string
        }
        hello: {
          type: string
        }

And the task script can populate result in an object form with:

echo -n "{\"foo\":\"bar\",\"hello\":\"world\"}" | tee $(results.object-results.path)

and we can refer to the object results in param like:
  params:
    - name: whole-object
      value: "$(tasks.task1.results.object-results[*])"
    - name: object-element
      value: "$(tasks.task1.results.object-results.hello)"

This feature is part of the TEP-0075.

tekton-robot · 2022-07-05T21:31:23Z

Skipping CI for Draft Pull Request.
If you want CI signal for your change, please convert it to an actual PR.
You can still manually trigger a test run with /test all

pkg/reconciler/taskrun/validate_resources.go

tekton-robot · 2022-07-05T23:54:12Z

The following is the coverage report on the affected files.
Say /test pull-tekton-pipeline-go-coverage to re-run this coverage report

File	Old Coverage	New Coverage	Delta
pkg/reconciler/pipelinerun/resources/apply.go	99.2%	99.2%	0.0
pkg/reconciler/pipelinerun/resources/resultrefresolution.go	94.6%	95.1%	0.5
pkg/reconciler/taskrun/validate_resources.go	96.1%	95.5%	-0.6

Yongxuanzhang · 2022-07-06T14:11:31Z

/assign @ywluogg

ywluogg · 2022-07-06T17:55:42Z

#4723

tekton-robot · 2022-07-07T14:33:06Z

The following is the coverage report on the affected files.
Say /test pull-tekton-pipeline-go-coverage to re-run this coverage report

File	Old Coverage	New Coverage	Delta
pkg/reconciler/pipelinerun/resources/apply.go	99.2%	99.2%	0.0
pkg/reconciler/pipelinerun/resources/resultrefresolution.go	94.9%	96.3%	1.4

ywluogg · 2022-07-07T15:00:00Z

pkg/reconciler/pipelinerun/resources/apply.go

-			pipelineTask.Params = replaceParamValues(pipelineTask.Params, stringReplacements, arrayReplacements, nil)
-			pipelineTask.Matrix = replaceParamValues(pipelineTask.Matrix, stringReplacements, arrayReplacements, nil)
+			pipelineTask.Params = replaceParamValues(pipelineTask.Params, stringReplacements, arrayReplacements, objectReplacements)
+			pipelineTask.Matrix = replaceParamValues(pipelineTask.Matrix, stringReplacements, arrayReplacements, objectReplacements)


I think we don't want matrix to support nested values right now. This would essentially mean matrix can be [[array, object], [etc.]], is it?

oh thanks! I will remove it from Matrix, and I also don't add tests for Matrix, better leave it,

tekton-robot · 2022-07-07T18:51:57Z

The following is the coverage report on the affected files.
Say /test pull-tekton-pipeline-go-coverage to re-run this coverage report

File	Old Coverage	New Coverage	Delta
pkg/reconciler/pipelinerun/resources/apply.go	99.2%	99.2%	0.0
pkg/reconciler/pipelinerun/resources/resultrefresolution.go	94.9%	95.4%	0.5

ywluogg · 2022-07-07T19:10:25Z

/lgtm

Yongxuanzhang · 2022-07-07T20:17:57Z

/retest

examples/v1beta1/pipelineruns/alpha/pipeline-object-results.yaml

dibyom · 2022-07-07T20:17:56Z

pkg/reconciler/pipelinerun/resources/apply.go

 	for _, resolvedPipelineRunTask := range targets {
 		if resolvedPipelineRunTask.PipelineTask != nil {
 			pipelineTask := resolvedPipelineRunTask.PipelineTask.DeepCopy()
-			pipelineTask.Params = replaceParamValues(pipelineTask.Params, stringReplacements, arrayReplacements, nil)
-			pipelineTask.Matrix = replaceParamValues(pipelineTask.Matrix, stringReplacements, arrayReplacements, nil)


why are we removing this?

I discussed with @ywluogg in the thread above (unresolved it now), she suggests we don't support array and object for Matrix now. The array is added in previous PR by mistake. It is out of scope of TEP 75

got it, thanks!

This is part of work in TEP-0075. This commit provides the support to apply object results replacements. Previous this commit we support emitting object results so users can write object results to task level, but we cannot pass object results from one task to another within one pipeline. This commit adds the support for this.

tekton-robot · 2022-07-07T20:49:43Z

The following is the coverage report on the affected files.
Say /test pull-tekton-pipeline-go-coverage to re-run this coverage report

File	Old Coverage	New Coverage	Delta
pkg/reconciler/pipelinerun/resources/apply.go	99.2%	99.2%	0.0
pkg/reconciler/pipelinerun/resources/resultrefresolution.go	94.9%	96.3%	1.4

ywluogg · 2022-07-07T20:52:59Z

/lgtm

dibyom · 2022-07-07T20:58:27Z

/approve

tekton-robot · 2022-07-07T20:58:40Z

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: dibyom

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

~~OWNERS~~ [dibyom]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

Yongxuanzhang · 2022-07-07T21:26:06Z

/retest

Yongxuanzhang · 2022-07-08T13:47:21Z

/retest

tekton-robot requested review from afrittoli and lbernick July 5, 2022 21:31

Yongxuanzhang marked this pull request as ready for review July 5, 2022 23:46

tekton-robot removed the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label Jul 5, 2022

tekton-robot requested review from abayer and pritidesai July 5, 2022 23:46

Yongxuanzhang commented Jul 5, 2022

View reviewed changes

pkg/reconciler/taskrun/validate_resources.go Outdated Show resolved Hide resolved

tekton-robot assigned ywluogg Jul 6, 2022

tekton-robot added the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Jul 7, 2022

Yongxuanzhang force-pushed the object-results branch from 7861b38 to 7ad27e3 Compare July 7, 2022 14:27

tekton-robot removed the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Jul 7, 2022

ywluogg reviewed Jul 7, 2022

View reviewed changes

Yongxuanzhang force-pushed the object-results branch from 7ad27e3 to b66c26d Compare July 7, 2022 18:46

Yongxuanzhang requested a review from ywluogg July 7, 2022 18:47

tekton-robot added the lgtm Indicates that a PR is ready to be merged. label Jul 7, 2022

dibyom reviewed Jul 7, 2022

View reviewed changes

Yongxuanzhang force-pushed the object-results branch from b66c26d to 441d819 Compare July 7, 2022 20:45

tekton-robot removed the lgtm Indicates that a PR is ready to be merged. label Jul 7, 2022

tekton-robot added the lgtm Indicates that a PR is ready to be merged. label Jul 7, 2022

tekton-robot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Jul 7, 2022

tekton-robot merged commit dc9ea6c into tektoncd:main Jul 8, 2022

Yongxuanzhang mentioned this pull request Sep 27, 2022

Mark tep75&76 implemented tektoncd/community#833

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[TEP-0075] Support Object Results substitution #5083

[TEP-0075] Support Object Results substitution #5083

Yongxuanzhang commented Jul 5, 2022 •

edited

Loading

tekton-robot commented Jul 5, 2022

tekton-robot commented Jul 5, 2022

Yongxuanzhang commented Jul 6, 2022

ywluogg commented Jul 6, 2022

tekton-robot commented Jul 7, 2022

ywluogg Jul 7, 2022

Yongxuanzhang Jul 7, 2022

Yongxuanzhang Jul 7, 2022

tekton-robot commented Jul 7, 2022

ywluogg commented Jul 7, 2022

Yongxuanzhang commented Jul 7, 2022

dibyom Jul 7, 2022

Yongxuanzhang Jul 7, 2022

dibyom Jul 7, 2022

tekton-robot commented Jul 7, 2022

ywluogg commented Jul 7, 2022

dibyom commented Jul 7, 2022

tekton-robot commented Jul 7, 2022

Yongxuanzhang commented Jul 7, 2022

Yongxuanzhang commented Jul 8, 2022

[TEP-0075] Support Object Results substitution #5083

[TEP-0075] Support Object Results substitution #5083

Conversation

Yongxuanzhang commented Jul 5, 2022 • edited Loading

Changes

Submitter Checklist

Release Notes

tekton-robot commented Jul 5, 2022

tekton-robot commented Jul 5, 2022

Yongxuanzhang commented Jul 6, 2022

ywluogg commented Jul 6, 2022

tekton-robot commented Jul 7, 2022

ywluogg Jul 7, 2022

Choose a reason for hiding this comment

Yongxuanzhang Jul 7, 2022

Choose a reason for hiding this comment

Yongxuanzhang Jul 7, 2022

Choose a reason for hiding this comment

tekton-robot commented Jul 7, 2022

ywluogg commented Jul 7, 2022

Yongxuanzhang commented Jul 7, 2022

dibyom Jul 7, 2022

Choose a reason for hiding this comment

Yongxuanzhang Jul 7, 2022

Choose a reason for hiding this comment

dibyom Jul 7, 2022

Choose a reason for hiding this comment

tekton-robot commented Jul 7, 2022

ywluogg commented Jul 7, 2022

dibyom commented Jul 7, 2022

tekton-robot commented Jul 7, 2022

Yongxuanzhang commented Jul 7, 2022

Yongxuanzhang commented Jul 8, 2022

Yongxuanzhang commented Jul 5, 2022 •

edited

Loading