Migrate our Tekton CD folder away from PipelineResources

[ghost]

Feature request

The Tasks and Pipelines we currently have in our cd directory utilize Cluster PipelineResources because they provide a way to include cluster secrets that won't expose them in logs.

This issue is to capture work to move away from PipelineResources in the resources from that directory. The current best alternative to the Cluster PipelineResource might be https://github.com/tektoncd/catalog/blob/main/task/kubeconfig-creator/0.1/kubeconfig-creator.yaml although that currently accepts credentials via param which is definitely not a secure approach.

• Folder deployment: Migrate the folder template CD service to workspaces #1157
• Namespace cleanup Migrate the namespace cleanup template CD service to workspaces #1159
• Configmap deployment Migrate the configmap CD service to workspaces #1160
• Tekton release logs & cleanup unused release resources Cleanup the tekton/resources/release folder #1161
• Helm deployment Migrate the helm template CD service to workspaces #1163
• Tekton deployment / Install tekton release Migrate install Tekton release CD service to workspaces #1164

[tekton-robot]

Issues go stale after 90d of inactivity.
Mark the issue as fresh with /remove-lifecycle stale with a justification.
Stale issues rot after an additional 30d of inactivity and eventually close.
If this issue is safe to close now please do so with /close with a justification.
If this issue should be exempted, mark the issue as frozen with /lifecycle frozen with a justification.

/lifecycle stale

Send feedback to tektoncd/plumbing.

[afrittoli]

/lifecycle frozen This is something that need to do in light of the deprecation of pipeline resources

[vdemeester]

/lifecycle frozen

[afrittoli]

Current cluster resources in use:

$ tkn resource list -t cluster
NAME                           TYPE      DETAILS
dogfooding                     cluster   url: https://35.222.251.168
dogfooding-tekton-cd           cluster   url: https://35.222.251.168
dogfooding-tekton-ci-default   cluster   url: https://35.222.251.168
dogfooding-tekton-deployer     cluster   url: https://35.222.251.168
dogfooding-tektoncd-cleaner    cluster   url: https://35.222.251.168
dogfooding-tektonci-default    cluster   url: https://35.222.251.168
prow-cluster-config-bot        cluster   url: https://104.198.136.199
prow-github-admin-default      cluster   url: https://104.198.136.199
robocat-cadmin                 cluster   url: https://35.228.156.151
robocat-tekton-deployer        cluster   url: https://35.228.156.151

An alternative approach could be to generate a kubeconfig for each of these resources and store it in a secret on the cluster.
We can rewrite the CD template to accept a workspace as input and bind it to a different secret depending on the target cluster / service account.

If any of the clusters or service accounts is recreated, the secrets will have to be refreshed, which is true today as well.

[afrittoli]

Tekton resources to convert:

apiVersion: v1
kind: ServiceAccount
metadata:
  name: resource-converter
---
apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
  name: secret-creator
rules:
- apiGroups: [""]
  resources: ["secrets"]
  verbs: ["get", "list", "watch", "create", "patch", "update"]
- apiGroups: ["tekton.dev"]
  resources: ["PipelineResources"]
  verbs: ["get", "list"]
---
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
  name: resource-converter-secret-creator
subjects:
- kind: ServiceAccount
  name: resource-converter
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: Role
  name: secret-creator
---
apiVersion: tekton.dev/v1beta1
kind: Task
metadata:
  name: resource-to-secret
spec:
  params:
    - name: targetSecret
  resources:
    inputs:
      - name: sourceResource
        type: cluster
  stepTemplate:
    env:
      - name: TARGET_SECRET
        value: $(params.targetSecret)
      - name: KUBECONFIG_PATH
        value: /workspace/$(resources.inputs.sourceResource.name)
  steps:
    - name: create-secret
      image: gcr.io/tekton-releases/dogfooding/kubectl
      script: |
        #!/bin/sh
        set -ex

        kubectl create secret generic "${TARGET_SECRET}" \
          --from-file="${KUBECONFIG_PATH}/kubeconfig" \
          --type=kubeconfig

        kubectl label "secret/${TARGET_SECRET}" app=tekton.cd

[afrittoli]

Script to convert all resources:

#!/bin/sh 

tkn resource list -t cluster | awk '/cluster/{ print $1 }' | while read aa; do 
  tkn task start resource-to-secret -s resource-converter -i sourceResource=${aa} -p targetSecret=tektoncd-${aa}; 
done

Resulting secrets:

$ kubectl get secret -l app=tekton.cd
NAME                                    TYPE         DATA   AGE
tektoncd-dogfooding                     kubeconfig   1      18s
tektoncd-dogfooding-tekton-cd           kubeconfig   1      18s
tektoncd-dogfooding-tekton-ci-default   kubeconfig   1      15s
tektoncd-dogfooding-tektoncd-cleaner    kubeconfig   1      15s
tektoncd-dogfooding-tektonci-default    kubeconfig   1      11s
tektoncd-prow-cluster-config-bot        kubeconfig   1      13s
tektoncd-prow-github-admin-default      kubeconfig   1      11s
tektoncd-robocat-cadmin                 kubeconfig   1      9s
tektoncd-robocat-tekton-deployer        kubeconfig   1      8s