diff --git a/docs/install.md b/docs/install.md index b831bca0f09..de5fbba9a7d 100644 --- a/docs/install.md +++ b/docs/install.md @@ -13,11 +13,13 @@ Use this page to add the component to an existing Kubernetes cluster. ``` 2. Grant cluster-admin permissions to the current user: + ```bash kubectl create clusterrolebinding cluster-admin-binding \ --clusterrole=cluster-admin \ --user=$(gcloud config get-value core/account) ``` + _See [Role-based access control](https://cloud.google.com/kubernetes-engine/docs/how-to/role-based-access-control#prerequisites_for_using_role-based_access_control) for more information_. @@ -32,9 +34,11 @@ To add the Tekton Pipelines component to an existing cluster: and its dependencies: ```bash - kubectl apply --filename https://storage.googleapis.com/knative-releases/build-pipeline/latest/release.yaml + kubectl apply --filename https://storage.googleapis.com/tekton-releases/latest/release.yaml ``` + _(Previous versions will be available at `previous/$VERSION_NUMBER`, e.g. https://storage.googleapis.com/tekton-releases/previous/0.2.0/release.yaml.)_ + 1. Run the [`kubectl get`](https://kubernetes.io/docs/reference/generated/kubectl/kubectl-commands#get) command to monitor the Tekton Pipelines components until all of the @@ -82,7 +86,7 @@ the persistent volume can fail. ## Custom Releases -The [release script](./../hack/release.md) can be used for creating a custom +The [relase Task](./../tekton/README.md) can be used for creating a custom release of Tekton Pipelines. This can be useful for advanced users that need to configure the container images built and used by the Pipelines components. diff --git a/tekton/README.md b/tekton/README.md index 2b089a91ac9..2f694e07b07 100644 --- a/tekton/README.md +++ b/tekton/README.md @@ -30,8 +30,8 @@ The `Tasks` which make up our release `Pipeline` are: ### Running -To run these `Pipelines` and `Tasks`, you must have Tekton Pipelines installed, -either via +To run these `Pipelines` and `Tasks`, you must have Tekton Pipelines installed +(in your own kubernetes cluster) either via [an official release](https://github.com/knative/build-pipeline/blob/master/docs/install.md) or [from `HEAD`](https://github.com/knative/build-pipeline/blob/master/DEVELOPMENT.md#install-pipeline). @@ -56,12 +56,87 @@ image registry instead. ``` - [`publish-run.yaml`](publish-run.yaml) - This example `TaskRun` and - `PipelineResources` demonstrate how to invoke `publish.yaml`: + `PipelineResources` demonstrate how to invoke `publish.yaml` (see [Creating a new release](#creating-a-new-release)) - ```bash - kubectl apply -f tekton/publish.yaml - kubectl apply -f tekton/publish-run.yaml - ``` +#### Creating a new release + +The `TaskRun` will use + +- The kubernetes service account [`release-right-meow`](account.yaml), which by + default has no associated secrets +- A secret called `release-secret` + +It needs to run with a service account in the target GCP project with +[`Storage Admin`](https://cloud.google.com/container-registry/docs/access-control) +access). + +To run the `publish-tekton-pipelines` `Task` and create a release: + +1. Pick the revision you want to release and replace the value of the `PipelineResouce` + [`tekton-pipelines` `revision`](publish-run.yaml#11), e.g.: + + ```yaml + - name: revision + value: 67efb48746a9d5d7d3b9b5c5cc210de7a47c6ebc # REPLACE with your own commit + ``` + +2. Change the value of the `PipelineRun` [`publish-run`'s `versionTag` parameter], e.g.: + + ```yaml + params: + - name: versionTag + value: 0.2.0 # REPLACE with the version you want to release + ``` + +3. To run against your own infrastructure (not needed for actual releases), also replace + the `imageRegistry` param: + + ```yaml + - name: imageRegistry + value: gcr.io/tekton-releases # REPLACE with your own registry + ``` + + And the `location` of the `tekton-bucket`: + + ```yaml + - name: location + value: gs://tekton-releases # REPLACE with your own bucket + ``` + +4. Setup the required credentials for the `release-right-meow` service acount, either: + + - For [the GCP service account + `release-right-meow@tekton-releases.iam.gserviceaccount.com`](#production-service-account) + which has the proper authorization to release the images and yamls in + [our `tekton-releases` GCP project](../infra/README.md#prow) + - For [your own GCP service account](https://cloud.google.com/iam/docs/creating-managing-service-accounts) + if running against your own infrastructure + + ```bash + KEY_FILE=release.json + GENERIC_SECRET=release-secret + ACCOUNT=release-right-meow + # Replace with your own service account if using your own infra + GCP_ACCOUNT="release-right-meow@tekton-releases.iam.gserviceaccount.com" + + # 1. Create a private key for the service account, which you can use + gcloud iam service-accounts keys create --iam-account $GCP_ACCOUNT $KEY_FILE + + # 2. Create kubernetes secret, which we will use via a service account and directly mounting + kubectl create secret generic $GENERIC_SECRET --from-file=./$KEY_FILE + + # 3. Add the docker secret to the service account + kubectl apply -f tekton/account.yaml + kubectl patch serviceaccount $ACCOUNT \ + -p "{\"secrets\": [{\"name\": \"$GENERIC_SECRET\"}]}" + ``` + +5. Run the `publish-tekton-pipelines` `Task`: + + ```bash + kubectl apply -f tekton/publish.yaml + kubectl apply -f tekton/publish-run.yaml + ``` ### Authentication @@ -70,6 +145,7 @@ Users executing the publish task must be able to: - Push to the image registry (production registry is `gcr.io/tekton-releases`) - Write to the GCS bucket (production bucket is `gs://tekton-releases`) +TODO: To be able to publish images via `kaniko` or `ko`, you must be able to push to your image registry. At the moment, the publish `Task` will try to use your default service account in the namespace where you create the `TaskRun`. If that @@ -90,6 +166,15 @@ For now, users who need access to our production registry (`gcr.io/tekton-releases`) and production GCS bucket (`gs://tekton-releases`) should ping @bobcatfish or @dlorenc to get added to the authorized users. +##### Production service account + +TODO(christiewilson, dlorenc): add a group which has access to this service account + +The GCP service account for creating release is +`release-right-meow@tekton-releases.iam.gserviceaccount.com`. This account has the role +[`Storage Admin`](https://cloud.google.com/container-registry/docs/access-control) +in order to be able to read and write buckets and images. + ## Supporting scripts Some supporting scripts have been written using Python 2.7: diff --git a/tekton/account.yaml b/tekton/account.yaml new file mode 100644 index 00000000000..720059ead8e --- /dev/null +++ b/tekton/account.yaml @@ -0,0 +1,4 @@ +apiVersion: v1 +kind: ServiceAccount +metadata: + name: release-right-meow \ No newline at end of file diff --git a/tekton/publish-run.yaml b/tekton/publish-run.yaml index ad14933c548..cc846e0e0c3 100644 --- a/tekton/publish-run.yaml +++ b/tekton/publish-run.yaml @@ -129,6 +129,7 @@ kind: TaskRun metadata: name: publish-run spec: + serviceAccount: release-right-meow taskRef: name: publish-tekton-pipelines trigger: diff --git a/tekton/publish.yaml b/tekton/publish.yaml index af5f52014ff..6939949d8c2 100644 --- a/tekton/publish.yaml +++ b/tekton/publish.yaml @@ -52,6 +52,13 @@ spec: - --destination=${inputs.params.imageRegistry}/${inputs.params.pathToProject}/${outputs.resources.builtBaseImage.url} - --context=/workspace/go/src/github.com/tektoncd/pipeline + volumeMounts: + - name: gcp-secret + mountPath: /secret + env: + - name: GOOGLE_APPLICATION_CREDENTIALS + value: /secret/release.json + - name: create-ko-yaml image: busybox command: @@ -63,7 +70,7 @@ spec: set -x cat < /workspace/go/src/github.com/tektoncd/pipeline/.ko.yaml - # By default `ko` will build images on top of `distroless` + # By default ko will build images on top of distroless baseImageOverrides: ${inputs.params.pathToProject}/${outputs.resources.builtCredsInitImage.url}: ${inputs.params.imageRegistry}/${inputs.params.pathToProject}/build-base:latest ${inputs.params.pathToProject}/${outputs.resources.builtGitInitImage.url}: ${inputs.params.imageRegistry}/${inputs.params.pathToProject}/build-base:latest @@ -90,6 +97,8 @@ spec: value: ${inputs.params.imageRegistry} - name: GOPATH value: /workspace/go + - name: CLOUDSDK_AUTH_CREDENTIAL_FILE_OVERRIDE + value: /secret/release.json command: - /bin/sh args: @@ -98,14 +107,20 @@ spec: set -e set -x - # TODO(#631) This is a hack to auth with the default creds, need a solution that supports service accounts + # Auth with CLOUDSDK_AUTH_CREDENTIAL_FILE_OVERRIDE gcloud auth configure-docker # ko requires this variable to be set in order to set image creation timestamps correctly https://github.com/google/go-containerregistry/pull/146 export SOURCE_DATE_EPOCH=`date +%s` + # Change to directory with our .ko.yaml + cd /workspace/go/src/github.com/tektoncd/pipeline + # Publish images and create release.yaml ko resolve --preserve-import-paths -f /workspace/go/src/github.com/tektoncd/pipeline/config/ > /workspace/bucket/latest/release.yaml + volumeMounts: + - name: gcp-secret + mountPath: /secret - name: copy-to-tagged-bucket image: busybox @@ -140,8 +155,13 @@ spec: ${inputs.params.imageRegistry}/${inputs.params.pathToProject}/${outputs.resources.builtControllerImage.url} ${inputs.params.imageRegistry}/${inputs.params.pathToProject}/${outputs.resources.builtWebhookImage.url} ) + # Parse the built images from the release.yaml generated by ko BUILT_IMAGES=( $(/workspace/go/src/github.com/tektoncd/pipeline/tekton/koparse/koparse.py --path /workspace/bucket/latest/release.yaml --base ${inputs.params.imageRegistry}/${inputs.params.pathToProject} --images ${IMAGES[@]}) ) + # Auth with account credentials + gcloud auth activate-service-account --key-file=/secret/release.json + + # Tag the images and put them in all the regions for IMAGE in "${BUILT_IMAGES[@]}" do for REGION in "${REGIONS[@]}" @@ -152,4 +172,12 @@ spec: gcloud -q container images add-tag ${IMAGE} ${REGION}.${IMAGE_WITHOUT_SHA}:$TAG done done - done \ No newline at end of file + done + volumeMounts: + - name: gcp-secret + mountPath: /secret + + volumes: + - name: gcp-secret + secret: + secretName: release-secret \ No newline at end of file