Proto-docs: Compile-time and Runtime PodTemplates (#3391)

* Proto-docs: client-side pod templates Signed-off-by: Eduardo Apolinario <[email protected]> * In between change Signed-off-by: eduardo apolinario <[email protected]> * Compile-time versus Runtime PodTemplates Signed-off-by: eduardo apolinario <[email protected]> * Change top-level heading to match https://www.sphinx-doc.org/en/master/usage/restructuredtext/basics.html#sections Signed-off-by: eduardo apolinario <[email protected]> * Explain the difference between the two types of Runtime PodTemplates Signed-off-by: eduardo apolinario <[email protected]> * Disambiguate the evaluation precedence of Runtime PodTemplates. Signed-off-by: Eduardo Apolinario <[email protected]> Signed-off-by: eduardo apolinario <[email protected]> --------- Signed-off-by: Eduardo Apolinario <[email protected]> Signed-off-by: eduardo apolinario <[email protected]> Co-authored-by: Eduardo Apolinario <[email protected]>
flyteorg · Jun 2, 2023 · 14171fd · 14171fd
1 parent baba9bd
commit 14171fd
Showing 1 changed file with 292 additions and 13 deletions.
diff --git a/rsts/deployment/configuration/general.rst b/rsts/deployment/configuration/general.rst
@@ -1,7 +1,8 @@
 .. _deployment-configuration-general:
 
+#################################
 Configuring Custom K8s Resources
-----------------------------------
+#################################
 
 ***************************
 Configurable Resource Types
@@ -278,7 +279,7 @@ Configuring K8s Pod
 ===================
 
 There are two approaches to applying the K8s Pod configuration. The **recommended**
-method is to use Flyte's default PodTemplate scheme. You can do this by creating
+method is to use Flyte's Compile-time and Runtime PodTemplate schemes. You can do this by creating
 K8s PodTemplate resource/s that serves as the base configuration for all the
 task Pods that Flyte initializes. This solution ensures completeness regarding
 support configuration options and maintainability as new features are added to K8s. 
@@ -290,7 +291,7 @@ The legacy technique is to set configuration options in Flyte's K8s plugin confi
     These two approaches can be used simultaneously, where the K8s plugin configuration will override the default PodTemplate values.
 
 *******************************
-Using Default K8s PodTemplates
+Using K8s PodTemplates
 *******************************
 
 `PodTemplate <https://kubernetes.io/docs/concepts/workloads/pods/#pod-templates>`__
@@ -304,10 +305,67 @@ of Flyte's task execution. It ensures complete control over Pod configuration,
 supporting all options available through the resource and ensuring maintainability
 in future versions.
 
-To initialize a default PodTemplate in Flyte:
+Starting with the Flyte 1.4 release, we now have 2 ways of defining `PodTemplate <https://kubernetes.io/docs/concepts/workloads/pods/#pod-templates>`__:
+1. Compile-time PodTemplate defined at the task level
+2. Runtime PodTemplates
+
+
+Compile-time PodTemplates
+=========================
+
+We can define a compile-time pod template, as part of the definition of a `Task <https://docs.flyte.org/projects/flytekit/en/latest/generated/flytekit.task.html#flytekit-task>`__, for example:
+
+.. code-block:: python
+
+    @task(
+        pod_template=PodTemplate(
+            primary_container_name="primary",
+            labels={"lKeyA": "lValA", "lKeyB": "lValB"},
+            annotations={"aKeyA": "aValA", "aKeyB": "aValB"},
+            pod_spec=V1PodSpec(
+                containers=[
+                    V1Container(
+                        name="primary",
+                        image="repo/placeholderImage:0.0.0",
+                        command="echo",
+                        args=["wow"],
+                        resources=V1ResourceRequirements(limits={"cpu": "999", "gpu": "999"}),
+                        env=[V1EnvVar(name="eKeyC", value="eValC"), V1EnvVar(name="eKeyD", value="eValD")],
+                    ),
+                ],
+                volumes=[V1Volume(name="volume")],
+                tolerations=[
+                    V1Toleration(
+                        key="num-gpus",
+                        operator="Equal",
+                        value=1,
+                        effect="NoSchedule",
+                    ),
+                ],
+            )
+        )
+    )
+    def t1() -> int:
+        ...
+
+Notice how in this example we are defining a new PodTemplate inline, which allows us to define a full
+`V1PodSpec <https://github.com/kubernetes-client/python/blob/master/kubernetes/docs/V1PodSpec.md>`__ and also define
+the name of the primary container, labels, and annotations.
+
+The term compile-time here refers to the fact that the pod template definition is part of the `TaskSpec <https://docs.flyte.org/projects/flyteidl/en/latest/protos/docs/admin/admin.html#ref-flyteidl-admin-taskclosure>`__.
+
+Runtime PodTemplates
+====================
+
+Runtime PodTemplates, as the name suggests, are applied during runtime, as part of building the resultant Pod. In terms of how
+they are applied, you have two choices: (1) you either elect one specific PodTemplate to be considered as default, or (2) you
+define a PodTemplate name and use that in the declaration of the task. Those two options are mutually exclusive, meaning that
+in the situation where a default PodTemplate is set and a PodTemplate name is present in the task definition, only the
+PodTemplate name will be used.
+
 
 Set the ``default-pod-template-name`` in FlytePropeller
-========================================================
+--------------------------------------------------------
 
 This `option <https://docs.flyte.org/en/latest/deployment/cluster_config/flytepropeller_config.html#default-pod-template-name-string>`__
 initializes a K8s informer internally to track system PodTemplate updates
@@ -330,14 +388,14 @@ An example configuration is:
         default-pod-template-name: <your_template_name>
  
 Create a PodTemplate resource
-=============================
+------------------------------
 
 Flyte recognizes PodTemplate definitions with the ``default-pod-template-name`` at two granularities. 
 
 1. A system-wide configuration can be created in the same namespace that
    FlytePropeller is running in (typically `flyte`). 
 2. PodTemplates can be applied from the same namespace that the Pod will be
-   created in. FlytePropeller always favours the PodTemplate with the more
+   created in. FlytePropeller always favors the PodTemplate with the more
    specific namespace. For example, a Pod created in the ``flytesnacks-development``
    namespace will first look for a PodTemplate from the ``flytesnacks-development``
    namespace. If that PodTemplate doesn't exist, it will look for a PodTemplate
@@ -364,6 +422,23 @@ during Pod construction. Similarly, each k8s container is required to have an
 set to anything. However, we recommend using a real image, for example
 ``docker.io/rwgrim/docker-noop``.
 
+Using ``pod_template_name`` in a Task
+--------------------------------------
+
+It's also possible to use PodTemplate in tasks by specifying ``pod_template_name`` in the task definition. For example:
+
+.. code-block:: python
+
+    @task(
+        pod_template_name="a_pod_template",
+    )
+    def t1() -> int:
+        ...
+
+In this example we're specifying that a previously created Runtime PodTemplate resource named ``a_pod_template`` is going to be applied.
+The only requirement is that this PodTemplate exists at the moment this task is about to be executed.
+
+
 *********************************
 Flyte's K8s Plugin Configuration
 *********************************
@@ -374,14 +449,28 @@ which are applied when constructing a Pod. Typically, these options map one-to-o
 with K8s Pod fields. This makes it difficult to maintain configuration options as K8s
 versions change and fields are added/deprecated.
 
-*******************
-Example PodTemplate
-*******************
+*********************************
+Evaluation Order in PodTemplates
+*********************************
+
+The following diagram shows the precedence in evaluation order between the different types of PodTemplates and K8s Plugin Configuration. The precedence is higher at the top and decreases as the height of the tree increases.
+
+.. mermaid::
+   :alt: Evaluation order of PodTemplates
+
+   graph BT
+     B["@task pod_template"] --> A["k8s plugin"]
+     C["runtime PodTemplate"] --> B
+     D["@task pod_template_name"] --> B
+
+
+To better understand how Flyte constructs task execution Pods based on Compile-time and Runtime PodTemplates,
+and K8s plugin configuration options, let's take a few examples.
 
-To better understand how Flyte constructs task execution Pods based on the default
-PodTemplate and K8s plugin configuration options, let's take an example. 
+Example 1: Runtime PodTemplate and K8s Plugin Configuration
+===========================================================
 
-If you have the default PodTemplate defined in the ``flyte`` namespace
+If you have a Runtime PodTemplate defined in the ``flyte`` namespace
 (where FlytePropeller instance is running), then it is applied to all Pods that
 Flyte creates, unless a **more specific** PodTemplate is defined in the namespace
 where you start the Pod.
@@ -457,3 +546,193 @@ Plugin configuration (that is, list appends and map overrides). Task-specific
 options are intentionally robust to provide fine-grained control over task
 execution in diverse use-cases. Therefore, exploration is beyond this scope
 and has therefore been omitted from this documentation.
+
+Example 2: A Runtime and Compile-time PodTemplates
+==================================================
+
+In this example we're going to have a Runtime PodTemplate and a Compile-time PodTemplate defined in a task.
+
+Let's say we have this Runtime PodTemplate defined in the same namespace as the one used to kick off an execution
+of the task. For example:
+
+.. code-block:: yaml
+
+    apiVersion: v1
+    kind: PodTemplate
+    metadata:
+      name: flyte-template
+      namespace: flytesnacks-development
+    template:
+      metadata:
+        annotations:
+          - annotation_1: initial-value
+          - bar: initial-value
+      spec:
+        containers:
+          - name: default
+            image: docker.io/rwgrim/docker-noop
+            terminationMessagePath: "/dev/foo"
+
+And the definition of the Compile-time PodTemplate in a task:
+
+.. code-block:: python
+
+    @task(
+        pod_template=PodTemplate(
+            primary_container_name="primary",
+            labels={
+              "label_1": "value-1",
+              "label_2": "value-2",
+            },
+            annotations={
+              "annotation_1": "value-1",
+              "annotation_2": "value-2",
+            },
+            pod_spec=V1PodSpec(
+                containers=[
+                    V1Container(
+                        name="primary",
+                        image="a.b.c/image:v1",
+                        command="cmd",
+                        args=[],
+                    ),
+                ],
+            )
+        )
+    )
+    def t1() -> int:
+        ...
+
+The resultant Pod is as follows:
+
+.. code-block:: yaml
+
+    apiVersion: v1
+    kind: Pod
+    metadata:
+      name: example-pod
+      namespace: flytesnacks-development
+      labels:
+        - label_1: value-1  # from Compile-time value
+        - label_2: value-2  # from Compile-time value
+      annotations:
+        - annotation_1: value-1  # value overridden by Compile-time PodTemplate
+        - annotation_2: value-2  # from Compile-time PodTemplate
+        - bar: initial-value  # from Runtime PodTemplate
+    spec:
+      containers:
+        - name: default
+          image: docker.io/rwgrim/docker-noop
+          terminationMessagePath: "/dev/foo"
+        - name: primary
+          image: a.b.c/image:v1
+          command: cmd
+          args: []
+          // remaining container configuration omitted
+
+Notice how options follow the same merging rules, i.e. lists append and maps override.
+
+
+Example 3: Runtime and Compile-time PodTemplates and K8s Plugin Configuration
+=============================================================================
+
+Now let's make a slightly more complicated example where now we have both Compile-time and Runtime PodTemplates being combined
+with K8s Configuration.
+
+Here's the definition of a Compile-time PodTemplate:
+
+.. code-block:: python
+
+    @task(
+        pod_template=PodTemplate(
+            primary_container_name="primary",
+            labels={
+              "label_1": "value-compile",
+              "label_2": "value-compile",
+            },
+            annotations={
+              "annotation_1": "value-compile",
+              "annotation_2": "value-compile",
+            },
+            pod_spec=V1PodSpec(
+                containers=[
+                    V1Container(
+                        name="primary",
+                        image="a.b.c/image:v1",
+                        command="cmd",
+                        args=[],
+                    ),
+                ],
+                host_network=True,
+            )
+        )
+    )
+    def t1() -> int:
+        ...
+
+
+And a Runtime PodTemplate:
+
+.. code-block:: yaml
+
+    apiVersion: v1
+    kind: PodTemplate
+    metadata:
+      name: flyte-template
+      namespace: flyte
+    template:
+      metadata:
+        labels:
+          - label_1: value-runtime
+          - label_2: value-runtime
+          - label_3: value-runtime
+        annotations:
+          - foo: value-runtime
+          - bar: value-runtime
+      spec:
+        containers:
+          - name: default
+            image: docker.io/rwgrim/docker-noop
+            terminationMessagePath: "/dev/foo"
+        hostNetwork: false
+
+And the following K8s Plugin Configuration:
+
+.. code-block:: yaml
+
+    plugins:
+       k8s:
+        default-labels:
+          - label_1: value-plugin
+        default-annotations:
+          - annotation_1: value-plugin
+          - baz: value-plugin
+
+The resultant pod for that task is as follows:
+
+.. code-block:: yaml
+
+    apiVersion: v1
+    kind: Pod
+    metadata:
+      name: example-pod
+      namespace: flytesnacks-development
+      labels:
+        - label_1: value-plugin
+        - label_2: value-compile
+      annotations:
+        - annotation_1: value-plugin
+        - annotation_2: value-compile
+        - foo: value-runtime
+        - bar: value-runtime
+        - baz: value-plugin
+    spec:
+      containers:
+        - name: default
+          image: docker.io/rwgrim/docker-noop
+          terminationMessagePath: "/dev/foo"
+        - name: primary
+          image: a.b.c/image:v1
+          command: cmd
+          args: []
+          // remaining container configuration omitted