Split /tekton/run directories into separate volumes.

[wlynch]

Changes

This changes splits each /tekton/run volume into separate volumes
so that steps can only mutate their own runtime information. This
prevents steps from unexpectedly interfering with other step execution.

To do this, we repurpose the /tekton/run/# path into a
step-specific directory. Since this was previously used by the
entrypoint for the post/wait files, we now use /tekton/run/#/out as
the post/wait filepath instead.

This does not change behavior of the directory, it enforces expected
behavior of steps.

/tekton/run is considered an internal implementation detail and is not
covered by the API compatibility policy, so it is safe to make changes
to the behavior of these files/paths.

This does not stop user execution from writing the step's own /tekton/run/#
folder. This needs more discussion/design - additional changes (if
needed) will be made in another commit.

This change is only focused on /tekton/run to reduce PR complexity. We will
likely want to make a similar change to /tekton/steps in another commit.
We may also look to consolidate all per-step volumes into a single
source (i.e. creds-init does something similar as well).

AFAICT, Ephemeral Volumes (i.e. EmptyDir) are exempt from Node Volume
limits (https://kubernetes.io/docs/concepts/storage/storage-limits/) -
spot checked this with a TaskRun with 100+ steps on both kind and GKE.

Part of #4227

/kind cleanup

Performance

Not very rigorous/scientific, but I ran 50 TaskRuns against this commit and v0.28.1 on a kind cluster, each with 100+ steps of:

    - image: ubuntu
      script: echo hello

Version	v0.29.1	This PR
Avg	0:01:50	0:01:59
Median	0:01:50	0:01:58
Min	0:01:46	0:01:53
Max	0:02:04	0:02:28

Submitter Checklist

As the author of this PR, please check off the items in this checklist:

• Docs included if any changes are user facing
• Tests included if any functionality added or changed
• Follows the commit message standard
• Meets the Tekton contributor standards (including
  functionality, content, code)
• Release notes block below has been filled in or deleted (only if no user facing changes)

Release Notes

NONE

Internal only changes.

[tekton-robot]

The following is the coverage report on the affected files.
Say /test pull-tekton-pipeline-go-coverage to re-run this coverage report

File	Old Coverage	New Coverage	Delta
pkg/pod/entrypoint.go	87.0%	86.8%	-0.2
pkg/pod/pod.go	90.7%	91.2%	0.5

[ghost]

Would it be worthwhile to add either an e2e test or an example yaml that exercises the RO behaviour of the volumes?

E.g. something like:

kind: TaskRun
apiVersion: tekton.dev/v1beta1
spec:
  taskSpec:
    steps:
    - image: # image with a shell
      script: |
        set +e # dont fail the script on error
        printf "hello world" > /tekton/steps/run/1/foo.txt
        if [ $? -eq 0 ] ; then
          echo "able to write to run directory of non-current step"
          exit 1
        fi
      - image: # this step exists only to have the run/1 volumemount in the first step
        script: echo "step 2"

[wlynch]

Would it be worthwhile to add either an e2e test or an example yaml that exercises the RO behaviour of the volumes?

E.g. something like:

kind: TaskRun
apiVersion: tekton.dev/v1beta1
spec:
  taskSpec:
    steps:
    - image: # image with a shell
      script: |
        set +e # dont fail the script on error
        printf "hello world" > /tekton/steps/run/1/foo.txt
        if [ $? -eq 0 ] ; then
          echo "able to write to run directory of non-current step"
          exit 1
        fi
      - image: # this step exists only to have the run/1 volumemount in the first step
        script: echo "step 2"

Done. (I wish there was a separate test-only directory, but added to examples for now)

[tekton-robot]

The following is the coverage report on the affected files.
Say /test pull-tekton-pipeline-go-coverage to re-run this coverage report

File	Old Coverage	New Coverage	Delta
pkg/pod/entrypoint.go	87.0%	86.8%	-0.2
pkg/pod/pod.go	87.8%	88.5%	0.7

[tekton-robot]

The following is the coverage report on the affected files.
Say /test pull-tekton-pipeline-go-coverage to re-run this coverage report

File	Old Coverage	New Coverage	Delta
pkg/pod/entrypoint.go	87.0%	86.8%	-0.2
pkg/pod/pod.go	87.8%	88.5%	0.7

[ghost]

Cheers!

/lgtm

[wlynch]

/test pull-tekton-pipeline-alpha-integration-tests pull-tekton-pipeline-integration-tests

[wlynch]

/test pull-tekton-pipeline-integration-tests

[vdemeester]

/retest

[tekton-robot]

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: vdemeester

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

• OWNERS [vdemeester]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment