Better support for CI systems that provide many env vars

[bobcatfish]

Expected Behavior

Many CI systems expect to pass a plethora of environment variables to executing logic (in our case, to the executing Step containers).

For example, these are environment varaibles that Jenkins X provides, and these are environment variables that Prow provides. Each has ~12-15+ env vars to provide.

It should be possible for CI systems that want to integrate with Tekton to:

1. Pass all environment variables to executing Tasks + Steps
2. Be able to add new environment variables without needing to change all Pipelines + Tasks

But at the same time we need to uphold Tekton principles, specifically avoiding implied coupling between Step containers and the CI system.

For example, say the CI system provides an environment variable PROW_JOB_ID and container gcr.io/someregistry/write-results uses the environment variable PROW_JOB_ID. It should be clear to anyone using a step that uses gcr.io/someregistry/write-results that the environment variable PROW_JOB_ID must be provided.

Actual Behavior

Currently all environment variables that are provided to a step must be listed explicitly. Providing a set of ~15 environment variable automatically to a Pipeline + Tasks could easily end up looking like:

• All Pipelines take all 15 env vars as parameters - if any env vars are added, all Pipelines + PipelineRun templates must be updated
• All Tasks referred to by all Pipelines could end up taking all 15 env vars as parameters

Both of these lead to brittle Pipelines and Tasks that can't be reused, and need to be updated to stay in sync with the CI system.

Additional Info

• It could be that this is best solved by providing best practices and guidance to CI authors integrating with Tekton Pipelines and nothing actually needs to change
• One interesting way to look at this problem could be in the context of TriggerTemplates, i.e. how could a triggering system provide all the context necessary
• @imjasonh suggested the idea of an EnvVarPipelineResource which could maybe hold bundle of env vars, and there would be a way to easily link those Resources to executing images (maybe @imjasonh could provide more detail here)

[imjasonh]

@poy is working on migrating https://github.com/google/kf from Knative Build to Tekton (🎉 ) and ran into this speedbump. As a workaround they're going to create a ConfigMap that defines all the env vars for each TaskRun, then create both and tie them together with:

steps:
- image: ubuntu
  envFrom:
  - configMapRef:
      name: $(inputs.params.ENVS)

Thought it might be worth mentioning as a (kinda cruddy) workaround until we come up with something better.

[tekton-robot]

Rotten issues close after 30d of inactivity.
Reopen the issue with /reopen.
Mark the issue as fresh with /remove-lifecycle rotten.

/close

Send feedback to tektoncd/plumbing.

[tekton-robot]

Issues go stale after 90d of inactivity.
Mark the issue as fresh with /remove-lifecycle stale.
Stale issues rot after an additional 30d of inactivity and eventually close.
If this issue is safe to close now please do so with /close.

/lifecycle stale

Send feedback to tektoncd/plumbing.

[tekton-robot]

@tekton-robot: Closing this issue.

In response to this:

Rotten issues close after 30d of inactivity.
Reopen the issue with /reopen.
Mark the issue as fresh with /remove-lifecycle rotten.

/close

Send feedback to tektoncd/plumbing.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

[vdemeester]

/remove-lifecycle rotten
/remove-lifecycle stale
/reopen

[tekton-robot]

@vdemeester: Reopened this issue.

In response to this:

/remove-lifecycle rotten
/remove-lifecycle stale
/reopen

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

[coryrc]

I think people who want to use Tekton should make dependencies explicit by using parameters. Environment variables are globals and we stopped using globals in code; I say it shouldn't be encouraged.

I am writing a Task to parse each Prow environment variable and put each into a result. (It'll get it from Prow via an encoded string passed as a single param).

[tekton-robot]

Issues go stale after 90d of inactivity.
Mark the issue as fresh with /remove-lifecycle stale.
Stale issues rot after an additional 30d of inactivity and eventually close.
If this issue is safe to close now please do so with /close.

/lifecycle stale

Send feedback to tektoncd/plumbing.

[tekton-robot]

Stale issues rot after 30d of inactivity.
Mark the issue as fresh with /remove-lifecycle rotten with a justification.
Rotten issues close after an additional 30d of inactivity.
If this issue is safe to close now please do so with /close with a justification.
If this issue should be exempted, mark the issue as frozen with /lifecycle frozen with a justification.

/lifecycle rotten

Send feedback to tektoncd/plumbing.

[tekton-robot]

Rotten issues close after 30d of inactivity.
Reopen the issue with /reopen with a justification.
Mark the issue as fresh with /remove-lifecycle rotten with a justification.
If this issue should be exempted, mark the issue as frozen with /lifecycle frozen with a justification.

/close

Send feedback to tektoncd/plumbing.

[tekton-robot]

@tekton-robot: Closing this issue.

In response to this:

Rotten issues close after 30d of inactivity.
Reopen the issue with /reopen with a justification.
Mark the issue as fresh with /remove-lifecycle rotten with a justification.
If this issue should be exempted, mark the issue as frozen with /lifecycle frozen with a justification.

/close

Send feedback to tektoncd/plumbing.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.