Skip to content

Commit

Permalink
Mount entrypoint volume as read-only.
Browse files Browse the repository at this point in the history
This change makes the entrypoint binary read-only by separating the
/tekton/tools directory:

- /tekton/bin - Mounted as RW by the place-tools init container, and RO
for all user steps. This directory will hold Tekton provided binaries (i.e.
entrypoint).

- /tekton/run - Named after Linux's /run directory
(https://en.wikipedia.org/wiki/Filesystem_Hierarchy_Standard). This
directory will hold Tekton runtime data (i.e. step post/wait files).

This is being done as an extra layer of security to prevent any tampering
of Tekton provided tools. This is similar in spirit to
89a6233 (making the scripts directory
read-only).

/tekton/tools was considered an internal directory, so this change is not bound to
API compatibility/deprecation policies. This change should have no
affect on the user API surface.

This change does not try to address any issues with the shared post/wait
file volume - this will be handled in another change.
  • Loading branch information
wlynch authored and tekton-robot committed Sep 18, 2021
1 parent 69edb2a commit 0fcdd39
Show file tree
Hide file tree
Showing 11 changed files with 325 additions and 276 deletions.
33 changes: 17 additions & 16 deletions cmd/entrypoint/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -29,14 +29,14 @@ Any extra positional arguments are passed to the original entrypoint command.
## Example

The following example of usage for `entrypoint` waits for
`/tekton/tools/3` file to exist and executes the command `bash` with args
`echo` and `hello`, then writes the file `/tekton/tools/4`, or
`/tekton/tools/4.err` in case the command fails.
`/tekton/run/3` file to exist and executes the command `bash` with args
`echo` and `hello`, then writes the file `/tekton/run/4`, or
`/tekton/run/4.err` in case the command fails.

```shell
entrypoint \
-wait_file /tekton/tools/3 \
-post_file /tekton/tools/4 \
-wait_file /tekton/run/3 \
-post_file /tekton/run/4 \
-entrypoint bash -- \
echo hello
```
Expand Down Expand Up @@ -64,22 +64,22 @@ to contain contents before proceeding.
The following example of usage for `entrypoint` waits for
`/tekton/downward/ready` file to exist and contain actual contents
(`-wait_file_contents`), and executes the command `bash` with args
`echo` and `hello`, then writes the file `/tekton/tools/1`, or
`/tekton/tools/1.err` in case the command fails.
`echo` and `hello`, then writes the file `/tekton/run/1`, or
`/tekton/run/1.err` in case the command fails.

```shell
entrypoint \
-wait_file /tekton/downward/ready \
-wait_file_contents \
-post_file /tekton/tools/1 \
-post_file /tekton/run/1 \
-entrypoint bash -- \
echo hello
```

## `cp` Mode

In order to make the `entrypoint` binary available to the user's steps, it gets
copied to a Volume that's shared with all the steps' containers. This is done
copied to a Volume that's shared with all the steps' containers as read-only. This is done
in an `initContainer` pre-step, that runs before steps start.

To reduce external dependencies, the `entrypoint` binary actually copies
Expand All @@ -95,22 +95,23 @@ initContainers:
args:
- cp
- /ko-app/entrypoint # <-- path to the entrypoint binary inside the image
- /tekton/tools/entrypoint
- /tekton/bin/entrypoint
volumeMounts:
- name: tekton-internal-tools
mountPath: /tekton/tools
- name: tekton-internal-bin
mountPath: /tekton/bin
containers:
- image: user-image
command:
- /tekton/tools/entrypoint
- /tekton/bin/entrypoint
... args to entrypoint ...
volumeMounts:
- name: tekton-internal-tools
mountPath: /tekton/tools
- name: tekton-internal-bin
mountPath: /tekton/bin
readonly: true
volumes:
- name: tekton-internal-tools
- name: tekton-internal-bin
volumeSource:
emptyDir: {}
```
12 changes: 6 additions & 6 deletions docs/debug.md
Original file line number Diff line number Diff line change
Expand Up @@ -47,7 +47,7 @@ and error. The subsequent steps are skipped in this case as well, marking the Ta

#### Halting a Step on failure

The failed step writes `<step-no>.err` to `/tekton/tools` and stops running completely. To be able to debug a step we would
The failed step writes `<step-no>.err` to `/tekton/run` and stops running completely. To be able to debug a step we would
need it to continue running (not exit), not skip the next steps and signal health of the step. By disabling step skipping,
stopping write of the `<step-no>.err` file and waiting on a signal by the user to disable the halt, we would be simulating a
"breakpoint".
Expand All @@ -58,7 +58,7 @@ environment using a CLI or an IDE.
#### Exiting breakpoint

To exit a step which has been paused upon failure, the step would wait on a file similar to `<step-no>.breakpointexit` which
would unpause and exit the step container. eg: Step 0 fails and is paused. Writing `0.breakpointexit` in `/tekton/tools`
would unpause and exit the step container. eg: Step 0 fails and is paused. Writing `0.breakpointexit` in `/tekton/run`
would unpause and exit the step container.

## Debug Environment
Expand All @@ -75,8 +75,8 @@ to reflect step number. eg: Step 0 will have `/tekton/debug/info/0`, Step 1 will

### Debug Scripts

`/tekton/debug/scripts/debug-continue` : Mark the step as completed with success by writing to `/tekton/tools`. eg: User wants to exit
breakpoint for failed step 0. Running this script would create `/tekton/tools/0` and `/tekton/tools/0.breakpointexit`.
`/tekton/debug/scripts/debug-continue` : Mark the step as completed with success by writing to `/tekton/run`. eg: User wants to exit
breakpoint for failed step 0. Running this script would create `/tekton/run/0` and `/tekton/run/0.breakpointexit`.

`/tekton/debug/scripts/debug-fail-continue` : Mark the step as completed with failure by writing to `/tekton/tools`. eg: User wants to exit
breakpoint for failed step 0. Running this script would create `/tekton/tools/0.err` and `/tekton/tools/0.breakpointexit`.
`/tekton/debug/scripts/debug-fail-continue` : Mark the step as completed with failure by writing to `/tekton/run`. eg: User wants to exit
breakpoint for failed step 0. Running this script would create `/tekton/run/0.err` and `/tekton/run/0.breakpointexit`.
14 changes: 9 additions & 5 deletions docs/developers/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -137,6 +137,8 @@ with the binary and file(s) is mounted.
If the image is a private registry, the service account should include an
[ImagePullSecret](https://kubernetes.io/docs/tasks/configure-pod-container/configure-service-account/#add-imagepullsecrets-to-a-service-account)

For more details, see [entrypoint/README.md](../../cmd/entrypoint/README.md).

## Reserved directories

### /workspace
Expand All @@ -152,6 +154,8 @@ Here is an example of a directory layout for a simple Task with 2 script steps:

```
/tekton
|-- bin
`-- entrypoint
|-- creds
|-- downward
| |-- ..2021_09_16_18_31_06.270542700
Expand All @@ -160,6 +164,8 @@ Here is an example of a directory layout for a simple Task with 2 script steps:
| `-- ready -> ..data/ready
|-- home
|-- results
|-- run
`-- 0
|-- scripts
| |-- script-0-t4jd8
| `-- script-1-4pjwp
Expand All @@ -169,23 +175,21 @@ Here is an example of a directory layout for a simple Task with 2 script steps:
| |-- step-foo
| `-- step-unnamed-0
| `-- exitCode
|-- termination
`-- tools
|-- 0
`-- entrypoint
`-- termination
```

| Path | Description |
| ------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| /tekton | Directory used for Tekton specific functionality |
| /tekton/bin | Tekton provided binaries / tools |
| /tekton/creds | Location of Tekton mounted secrets. See [Authentication at Run Time](../auth.md) for more details. |
| /tekton/downward | Location of data mounted via the [Downward API](https://kubernetes.io/docs/tasks/inject-data-application/downward-api-volume-expose-pod-information/#the-downward-api). |
| /tekton/home | (deprecated - see https://github.com/tektoncd/pipeline/issues/2013) Default home directory for user containers. |
| /tekton/results | Where [results](#results) are written to (path available to `Task` authors via [`$(results.name.path)`](../variables.md)) |
| /tekton/run | Runtime variable data. [Used for coordinating step ordering](#entrypoint-rewriting-and-step-ordering). |
| /tekton/scripts | Contains user provided scripts specified in the TaskSpec. |
| /tekton/steps | Where the `step` exitCodes are written to (path available to `Task` authors via [`$(steps.<stepName>.exitCode.path)`](../variables.md#variables-available-in-a-task)) |
| /tekton/termination | where the eventual [termination log message](https://kubernetes.io/docs/tasks/debug-application-cluster/determine-reason-pod-failure/#writing-and-reading-a-termination-message) is written to [Sequencing step containers](#entrypoint-rewriting-and-step-ordering) |
| /tekton/tools | Contains tools like the [entrypoint binary](#entrypoint-rewriting-and-step-ordering), post_files for coordinating step starts |

The following directories are covered by the
[Tekton API Compatibility policy](../api_compatibility_policy.md), and can be
Expand Down
2 changes: 2 additions & 0 deletions internal/builder/v1beta1/container.go
Original file line number Diff line number Diff line change
Expand Up @@ -78,6 +78,8 @@ func VolumeMount(name, mountPath string, ops ...VolumeMountOp) ContainerOp {
}
}

var VolumeMountRO = func(vm *corev1.VolumeMount) { vm.ReadOnly = true }

// Resources adds ResourceRequirements to the Container (step).
func Resources(ops ...ResourceRequirementsOp) ContainerOp {
return func(c *corev1.Container) {
Expand Down
43 changes: 30 additions & 13 deletions pkg/pod/entrypoint.go
Original file line number Diff line number Diff line change
Expand Up @@ -36,9 +36,12 @@ import (
)

const (
toolsVolumeName = "tekton-internal-tools"
mountPoint = "/tekton/tools"
entrypointBinary = mountPoint + "/entrypoint"
binVolumeName = "tekton-internal-bin"
binDir = "/tekton/bin"
entrypointBinary = binDir + "/entrypoint"

runVolumeName = "tekton-internal-run"
runDir = "/tekton/run"

downwardVolumeName = "tekton-internal-downward"
downwardMountPoint = "/tekton/downward"
Expand All @@ -55,12 +58,26 @@ const (

var (
// TODO(#1605): Generate volumeMount names, to avoid collisions.
toolsMount = corev1.VolumeMount{
Name: toolsVolumeName,
MountPath: mountPoint,
binMount = corev1.VolumeMount{
Name: binVolumeName,
MountPath: binDir,
}
binROMount = corev1.VolumeMount{
Name: binVolumeName,
MountPath: binDir,
ReadOnly: true,
}
binVolume = corev1.Volume{
Name: binVolumeName,
VolumeSource: corev1.VolumeSource{EmptyDir: &corev1.EmptyDirVolumeSource{}},
}

runMount = corev1.VolumeMount{
Name: runVolumeName,
MountPath: runDir,
}
toolsVolume = corev1.Volume{
Name: toolsVolumeName,
runVolume = corev1.Volume{
Name: runVolumeName,
VolumeSource: corev1.VolumeSource{EmptyDir: &corev1.EmptyDirVolumeSource{}},
}

Expand Down Expand Up @@ -105,7 +122,7 @@ func orderContainers(entrypointImage string, commonExtraEntrypointArgs []string,
// Invoke the entrypoint binary in "cp mode" to copy itself
// into the correct location for later steps.
Command: []string{"/ko-app/entrypoint", "cp", "/ko-app/entrypoint", entrypointBinary},
VolumeMounts: []corev1.VolumeMount{toolsMount},
VolumeMounts: []corev1.VolumeMount{binMount},
}

if len(steps) == 0 {
Expand All @@ -122,16 +139,16 @@ func orderContainers(entrypointImage string, commonExtraEntrypointArgs []string,
"-wait_file", filepath.Join(downwardMountPoint, downwardMountReadyFile),
"-wait_file_content", // Wait for file contents, not just an empty file.
// Start next step.
"-post_file", filepath.Join(mountPoint, fmt.Sprintf("%d", i)),
"-post_file", filepath.Join(runDir, fmt.Sprintf("%d", i)),
"-termination_path", terminationPath,
"-step_metadata_dir", filepath.Join(pipeline.StepsDir, name),
"-step_metadata_dir_link", filepath.Join(pipeline.StepsDir, fmt.Sprintf("%d", i)),
}
default:
// All other steps wait for previous file, write next file.
argsForEntrypoint = []string{
"-wait_file", filepath.Join(mountPoint, fmt.Sprintf("%d", i-1)),
"-post_file", filepath.Join(mountPoint, fmt.Sprintf("%d", i)),
"-wait_file", filepath.Join(runDir, fmt.Sprintf("%d", i-1)),
"-post_file", filepath.Join(runDir, fmt.Sprintf("%d", i)),
"-termination_path", terminationPath,
"-step_metadata_dir", filepath.Join(pipeline.StepsDir, name),
"-step_metadata_dir_link", filepath.Join(pipeline.StepsDir, fmt.Sprintf("%d", i)),
Expand Down Expand Up @@ -174,7 +191,7 @@ func orderContainers(entrypointImage string, commonExtraEntrypointArgs []string,

steps[i].Command = []string{entrypointBinary}
steps[i].Args = argsForEntrypoint
steps[i].VolumeMounts = append(steps[i].VolumeMounts, toolsMount)
steps[i].VolumeMounts = append(steps[i].VolumeMounts, binROMount, runMount)
steps[i].TerminationMessagePath = terminationPath
}
// Mount the Downward volume into the first step container.
Expand Down
Loading

0 comments on commit 0fcdd39

Please sign in to comment.