Add TektonListener to accept events and trigger pipeline/task runs

[iancoffey]

Changes

Proposal: https://docs.google.com/document/d/1tfG6dkARP2xko-5trDV3D62DJZiXQBZgLtLH7GyUopM/edit#

This is a mostly-functioning WIP of the proposal linked above. It adds a new type, PipelineListener and CRD which allow for a workflow to accept events and trigger pipeline runs end-to-end from any source. This implementation is meant to allow the project to accept any event type, with initial support for CloudEvents, and the ability to add event handling as needed for other types/sources.

Related

This concept started off in the build project, there is some good background there as well.

knative/build#577
knative/build#591

TODO

• Add tests
• investigate ftest situation
• end-to-end checksuite event success

Submitter Checklist

These are the criteria that every PR should meet, please check them off as you
review them:

• Includes tests (if functionality changed/added)
• Includes docs (if user facing)
• Commit messages follow commit message best practices

See the contribution guide
for more details.

Release Notes

Adds a new CRD and type `PipelineListener` to allow for PipelineRuns to be triggered automatically from events.

[tekton-robot]

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by: iancoffey
To fully approve this pull request, please assign additional approvers.
We suggest the following additional approver: imjasonh

If they are not already assigned, you can assign the PR to them by writing /assign @imjasonh in a comment when ready.

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

• OWNERS

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[dlorenc]

A few high level comments, thanks for sending this out!

[dlorenc]

Is there a way to take data out of the event payload and thread that through into the pipeline via creation of a new resource, or similar?

[dlorenc]

Here's a strawperson I put together on how binding eventdata into a resource might work: https://gist.github.com/dlorenc/bdd3dc952cd9b9cb10b837224b031214

Any thoughts?

[iancoffey]

I find this very interesting! Admittedly though, I am still processing how the example EventBinding type fits into the puzzle, inre: this PR.

Would that kind be used to spawn a listener behind the scenes based on the binding type? Or am I way off :)

I would love to chat about this more, feel free to ping on tekton slack as well.

[dlorenc]

What about storing info like the image used in a configmap or a flag somewhere?

[iancoffey]

That sounds like a better plan, Ill make that change.

[bobcatfish]

/ok-to-test

[khrm]

@iancoffey

1. Doesn't this implementation create a stateful set for every repo?

2. Shouldn't this pr be raised wrt https://github.com/tektoncd/experimental/tree/master/webhooks-extension ?

3. Great idea of using runSpec to define pipelineRun . I have done an implementation which uses location of pipelinerun.yaml to define it.

[iancoffey]

@khrm hello, some answers inline:

Doesn't this implementation create a stateful set for every repo?

It creates a statefulset for every listener. Since it creates a listener for any run spec provided - there could also be multiple Listeners for one repo. Thinking of use cases - there may be different events that need to be handled, or you may build different parts of your monorepo based on specific events. I think the implementation allows for almost any combo of repo/branch/pipeline but to be fair I have only tested 1:1:1.

Shouldn't this pr be raised wrt https://github.com/tektoncd/experimental/tree/master/webhooks-extension ?

Yes, its possible. My goal for next Tuesdays Tekton WG meet is to understand where this should be added to the experimental repo or if this can land right here once feedback is addressed.

Great idea of using runSpec to define pipelineRun . I have done an implementation which uses location of pipelinerun.yaml to define it.

Cool! Thanks. There are many updates coming via the proposal document that will likely change much of this but that part of the spec will likely stay and grow.

[bobcatfish]

@iancoffey can we close this PR now that we've got https://github.com/tektoncd/experimental/tree/master/tekton-listener ? :D

[iancoffey]

yes and ty!