Support custom Pod scheduler

[kfox1111]

It would be interesting if volcano (http://volcano.sh/) would be supported to enable tasks to be queued and more fairly scheduled in a shared environment

[imjasonh]

Very interesting! Today Tekton just creates Pods and expects Kubernetes to schedule them as fairly as possible. In theory one could plug in whatever Pod scheduler you wanted to schedule jobs more fairly, if that's something you were concerned about.

Since Kubernetes is generally optimized for long-running serving jobs, I wouldn't be surprised if there was an opportunity to improve the scheduling algorithm, for instance to take into account a TaskRun's timeout, since you can know when a TaskRun is guaranteed to finish.

If there's some way to take advantage of Volcano's scheduler it would be great. I wouldn't want to take a hard dependency on it though unless there was sufficient benefit and users were comfortable with that dependency (WRT performance and security, for instance)

[imjasonh]

/kind feature

[kfox1111]

Volcano uses a fork of kube-batch at the moment. I think for kube-batch, it looks like it supports scheduling pods directly via the 'kube-batch' scheduler:
https://github.com/kubernetes-sigs/kube-batch/blob/master/doc/usage/tutorial.md
While they don't have a bare pod example, they do have a job example in the tutorial.

If that's the case, it may be as simple as making which scheduler to use configurable in tekton.

[imjasonh]

AFAIK all it would take to support custom schedulers in Tekton is to add schedulerName to TaskRunSpec's PodTemplate type, and pass it in when creating a Pod for a TaskRun (here).

This should be fairly straightforward to implement, if you're interested.

[waveywaves]

@imjasonh I would like to work on this issue :)

[waveywaves]

/assign