Task Steps with Knative Service definition

[brindasanth]

Team,

I see Task -> Steps have containers, This will create new container for each request.

How do I point to existing Knative services, send request and wait for response completion ?
By this I can reuse the available services, avoid latency ( Starting up and Destroying of containers).

This will open up lot of use cases achieved with tektoncd.

[dibyom]

I see Task -> Steps have containers, This will create new container for each request.

This is by design since for most CI/CD use cases (building/testing) the latency of spinning up a new Pod is little compared to to the time it takes to run the build/test.

How do I point to existing Knative services, send request and wait for response completion ?

If you have an existing Knative service, you should be able to hit it directly....not sure how Tekton is involved here. Could you describe your use case in more detail?

[dibyom]

/kind question

[brindasanth]

We have Software Defined Network (SDN) automation process, which has sequenced of tasks, i.e mix and match of using Serverless functions and already hosted Knative/fixed scaled services. We are trying to reuse Tektoncd Pipeline runner / Argo workflow for that. For me TekonCD is getting popular, cloud native and it achieves the purpose we need. but always launching new containers for specific scenerio is not affordable with respect to latency in network automations.

[brindasanth]

Adding to the point, It will simplify pipeline and task definitions with only input parameters and service endpoint definitions, no need to configure other things such as mounts etc. Because that can be handled in service container spec definitions one time.

[vincent-pli]

@brindasanth
So, do you expect the existing KSVC as a step of your Pipeline or just want to invoke and get replay from your step to the KSVC?

[brindasanth]

I just want to invoke KSCV with defined input parameters on that task and wait for response, so that it can kick on next tasks in pipeline with that task output.

Example flow:
[Knative Service Task] -> [Build task] ->[Knative Service Task] ->[Server less function Task] -> [END]

[vincent-pli]

@brindasanth
I guess we need add cloudevent send/receive stuff to https://github.com/tektoncd/catalog

[ghost]

There has been a fair amount of abstract discussion about removing Pods / Containers backing Tasks See #215 for the earliest suggestion of this idea (that I can find).

@imjasonh has definitely had thoughts around this and made moves to reduce Tekton's reliance on Pods / Kubernetes-isms as well: #1689

On the other hand we've had requests to actually increase the "kubernetes-ness" too: #2327

Essentially I think there are ideas around this but very little actual concrete work. Tasks are backed by Pods and Steps are Containers and that's just how it works right now. Would love to see more concrete design, proposal and implementation around this area though.

[brindasanth]

Knative Container Source already uses the sink concept. for one way communication. I think, we shall have sink in task definitions as below. Some time injectable custom task implementation will solve the issue.

apiVersion: sources.knative.dev/v1alpha2
kind: ContainerSource
metadata:
  name: test-heartbeats
  sink:
    ref:
      apiVersion: serving.knative.dev/v1
      kind: Service
      name: event-display
      url : /xxx/xxx

would be:

apiVersion: tekton.dev/v1beta1
kind: Task
metadata:
  name: example-task-name
spec:
  params:
    - input: pathToDockerFile
      type: string
      description: The path to the dockerfile to build
      default: /workspace/workspace/Dockerfile
    - output: pathToDockerFile
      type: string
      description: The path to the dockerfile to build
      default: /workspace/workspace/Dockerfile
  steps:
    ref:
      apiVersion: serving.knative.dev/v1
      kind: Service
      name: event-display
      url : /xxx/xxx

[vincent-pli]

@brindasanth
What you need is step which could send cloudevent, I guess this is what you want: https://github.com/vincent-pli/cloudevent-client-tekton-step

[brindasanth]

@vincent-pli Thanks a lot for this implementation, I am looking for similar solution.
I have some questions in this implementation.

1. Does it initialize new container every time for cloud send event ? ( I am looking for solution, which doesn't initialize new container, so that we will get better performance, please refer the comments above), Step should't have image definitions, only Input, Output, and Sink Service definition. The main purpose is we will use Knative AutoScaling services to process the task.
2. Is Sink ->ref->url is populated in getSinkUrl() method ?
3. Will task wait for Service response and process it ?

[vincent-pli]

@brindasanth
For your questions:

1. Yes, it create new container, since it works as step of tekton, based current implements it will create a container.
2. Yes.
3. Yes, it will write response.data (event.data) to /tekton/results/response

[brindasanth]

@vincent-pli
Thanks for the clarifications.
Is it possible to have one more special or custom task, from framework perspective, which invokes services without creating new containers every time ?

[vincent-pli]

@brindasanth
I'm afraid you cannot do it with tekton, its stratagem it to leverage functions as container.

[tekton-robot]

Issues go stale after 90d of inactivity.
Mark the issue as fresh with /remove-lifecycle stale.
Stale issues rot after an additional 30d of inactivity and eventually close.
If this issue is safe to close now please do so with /close.

/lifecycle stale

Send feedback to tektoncd/plumbing.

[tekton-robot]

Stale issues rot after 30d of inactivity.
Mark the issue as fresh with /remove-lifecycle rotten.
Rotten issues close after an additional 30d of inactivity.
If this issue is safe to close now please do so with /close.

/lifecycle rotten

Send feedback to tektoncd/plumbing.

[tekton-robot]

Rotten issues close after 30d of inactivity.
Reopen the issue with /reopen.
Mark the issue as fresh with /remove-lifecycle rotten.

/close

Send feedback to tektoncd/plumbing.

[tekton-robot]

@tekton-robot: Closing this issue.

In response to this:

Rotten issues close after 30d of inactivity.
Reopen the issue with /reopen.
Mark the issue as fresh with /remove-lifecycle rotten.

/close

Send feedback to tektoncd/plumbing.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.