[BUG] Custom environment variable injection does not work for complex workflow setup

[honnix]

Describe the bug

A highly appreciated feature enabling injecting custom environment variable was added to flyte 1.6.0 release, for example the related change in flyteadmin: flyteorg/flyteadmin#556

We tested this feature on one of our workflows, and it partially worked.

For a simple container task, this worked well and we could observed the custom environment variables were injected into the container. For a complex task, in our case a remote launch plan (for a workflow with multiple tasks) backed task, the custom environment variables were not injected into containers that were part of the remote launch plan.

Expected behavior

Custom environment variables should be injected to all containers launched for a workflow execution, regardless how the workflow is composed.

Additional context to reproduce

No response

Screenshots

No response

Are you sure this issue hasn't been raised already?

• Yes

Have you read the Code of Conduct?

• Yes