Synchronization (Lock)

[milenvk]

Feature request

The ability to coordinate and synchronize access to a shared resource between concurrently running tasks and pipelines.

Use case

Two or more tasks that run in parallel (either in the same pipeline run or on concurrently running pipeline runs) need to modify a shared resource. For example two task runs, executed in parallel, need to bump up a build number stored in a git repository. Both tasks need to fetch the current version number, increment it by 1, and then push the new version number. To avoid conflicts, the task runs try to acquire a cluster-level lock with a unique name, (say called "version-update"). The task run that manages to acquire the lock first, holds it until it finishes its work, and then releases it. The task run that failed to acquire the lock waits for the lock to be released, and then it acquires it, and does its work. No conflicts.

Comments

This could be manually implemented as part of each task (by say using leader election https://pkg.go.dev/k8s.io/client-go/tools/leaderelection), however the user needs to account for releasing the lock in case a task run failed or timed out.
Having a lock, or general synchronization as part of the Task CRD (see reference below), complete with a lock lifecycle management, would make things easier for the user.

References

For an implementation of a similar feature please see Argo Workflows Synchronization: https://argoproj.github.io/argo-workflows/synchronization/

[jerop]

Related to #2828 -- cc @wlynch

[tekton-robot]

Issues go stale after 90d of inactivity.
Mark the issue as fresh with /remove-lifecycle stale with a justification.
Stale issues rot after an additional 30d of inactivity and eventually close.
If this issue is safe to close now please do so with /close with a justification.
If this issue should be exempted, mark the issue as frozen with /lifecycle frozen with a justification.

/lifecycle stale

Send feedback to tektoncd/plumbing.

[jerop]

/lifecycle frozen