Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Proto-docs: Compile-time and Runtime PodTemplates #3391

Merged
merged 7 commits into from
Jun 2, 2023
Merged

Proto-docs: Compile-time and Runtime PodTemplates #3391

Changes from all commits
Commits
Show all changes
7 commits
Select commit Hold shift + click to select a range
3ec7e3f
Proto-docs: client-side pod templates
eapolinario Mar 3, 2023
7d91076
In between change
eapolinario Apr 4, 2023
b6af172
Compile-time versus Runtime PodTemplates
eapolinario Jun 1, 2023
ef22016
Merge remote-tracking branch 'origin' into add-task-podtemplate-docs
eapolinario Jun 1, 2023
07f540a
Change top-level heading to match https://www.sphinx-doc.org/en/maste…
eapolinario Jun 1, 2023
1d8b03a
Explain the difference between the two types of Runtime PodTemplates
eapolinario Jun 2, 2023
6dc6a4f
Disambiguate the evaluation precedence of Runtime PodTemplates.
eapolinario Jun 2, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
305 changes: 292 additions & 13 deletions rsts/deployment/configuration/general.rst
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,7 +1,8 @@
.. _deployment-configuration-general:

#################################
Configuring Custom K8s Resources
----------------------------------
#################################

***************************
Configurable Resource Types
Expand Down Expand Up @@ -278,7 +279,7 @@ Configuring K8s Pod
===================

There are two approaches to applying the K8s Pod configuration. The **recommended**
method is to use Flyte's default PodTemplate scheme. You can do this by creating
method is to use Flyte's Compile-time and Runtime PodTemplate schemes. You can do this by creating
K8s PodTemplate resource/s that serves as the base configuration for all the
task Pods that Flyte initializes. This solution ensures completeness regarding
support configuration options and maintainability as new features are added to K8s.
Expand All @@ -290,7 +291,7 @@ The legacy technique is to set configuration options in Flyte's K8s plugin confi
These two approaches can be used simultaneously, where the K8s plugin configuration will override the default PodTemplate values.

*******************************
Using Default K8s PodTemplates
Using K8s PodTemplates
*******************************

`PodTemplate <https://kubernetes.io/docs/concepts/workloads/pods/#pod-templates>`__
Expand All @@ -304,10 +305,67 @@ of Flyte's task execution. It ensures complete control over Pod configuration,
supporting all options available through the resource and ensuring maintainability
in future versions.

To initialize a default PodTemplate in Flyte:
Starting with the Flyte 1.4 release, we now have 2 ways of defining `PodTemplate <https://kubernetes.io/docs/concepts/workloads/pods/#pod-templates>`__:
1. Compile-time PodTemplate defined at the task level
2. Runtime PodTemplates


Compile-time PodTemplates
=========================

We can define a compile-time pod template, as part of the definition of a `Task <https://docs.flyte.org/projects/flytekit/en/latest/generated/flytekit.task.html#flytekit-task>`__, for example:

.. code-block:: python

@task(
pod_template=PodTemplate(
primary_container_name="primary",
labels={"lKeyA": "lValA", "lKeyB": "lValB"},
annotations={"aKeyA": "aValA", "aKeyB": "aValB"},
pod_spec=V1PodSpec(
containers=[
V1Container(
name="primary",
image="repo/placeholderImage:0.0.0",
command="echo",
args=["wow"],
resources=V1ResourceRequirements(limits={"cpu": "999", "gpu": "999"}),
env=[V1EnvVar(name="eKeyC", value="eValC"), V1EnvVar(name="eKeyD", value="eValD")],
),
],
volumes=[V1Volume(name="volume")],
tolerations=[
V1Toleration(
key="num-gpus",
operator="Equal",
value=1,
effect="NoSchedule",
),
],
)
)
)
def t1() -> int:
...

Notice how in this example we are defining a new PodTemplate inline, which allows us to define a full
`V1PodSpec <https://github.com/kubernetes-client/python/blob/master/kubernetes/docs/V1PodSpec.md>`__ and also define
the name of the primary container, labels, and annotations.

The term compile-time here refers to the fact that the pod template definition is part of the `TaskSpec <https://docs.flyte.org/projects/flyteidl/en/latest/protos/docs/admin/admin.html#ref-flyteidl-admin-taskclosure>`__.

Runtime PodTemplates
====================

Runtime PodTemplates, as the name suggests, are applied during runtime, as part of building the resultant Pod. In terms of how
they are applied, you have two choices: (1) you either elect one specific PodTemplate to be considered as default, or (2) you
define a PodTemplate name and use that in the declaration of the task. Those two options are mutually exclusive, meaning that
in the situation where a default PodTemplate is set and a PodTemplate name is present in the task definition, only the
PodTemplate name will be used.


Set the ``default-pod-template-name`` in FlytePropeller
========================================================
--------------------------------------------------------
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The Set the default-pod-template-name in FlytePropeller above this line is only required when using a default PodTemplate and not if it is set in the task decorator. Does it make sense to make this distinction?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm going to add a little blurb explaining the difference between the default Runtime PodTemplate and a named one.


This `option <https://docs.flyte.org/en/latest/deployment/cluster_config/flytepropeller_config.html#default-pod-template-name-string>`__
initializes a K8s informer internally to track system PodTemplate updates
Expand All @@ -330,14 +388,14 @@ An example configuration is:
default-pod-template-name: <your_template_name>

Create a PodTemplate resource
=============================
------------------------------

Flyte recognizes PodTemplate definitions with the ``default-pod-template-name`` at two granularities.

1. A system-wide configuration can be created in the same namespace that
FlytePropeller is running in (typically `flyte`).
2. PodTemplates can be applied from the same namespace that the Pod will be
created in. FlytePropeller always favours the PodTemplate with the more
created in. FlytePropeller always favors the PodTemplate with the more
specific namespace. For example, a Pod created in the ``flytesnacks-development``
namespace will first look for a PodTemplate from the ``flytesnacks-development``
namespace. If that PodTemplate doesn't exist, it will look for a PodTemplate
Expand All @@ -364,6 +422,23 @@ during Pod construction. Similarly, each k8s container is required to have an
set to anything. However, we recommend using a real image, for example
``docker.io/rwgrim/docker-noop``.

Using ``pod_template_name`` in a Task
--------------------------------------

It's also possible to use PodTemplate in tasks by specifying ``pod_template_name`` in the task definition. For example:

.. code-block:: python

@task(
pod_template_name="a_pod_template",
)
def t1() -> int:
...

In this example we're specifying that a previously created Runtime PodTemplate resource named ``a_pod_template`` is going to be applied.
The only requirement is that this PodTemplate exists at the moment this task is about to be executed.


*********************************
Flyte's K8s Plugin Configuration
*********************************
Expand All @@ -374,14 +449,28 @@ which are applied when constructing a Pod. Typically, these options map one-to-o
with K8s Pod fields. This makes it difficult to maintain configuration options as K8s
versions change and fields are added/deprecated.

*******************
Example PodTemplate
*******************
*********************************
Evaluation Order in PodTemplates
*********************************

The following diagram shows the precedence in evaluation order between the different types of PodTemplates and K8s Plugin Configuration. The precedence is higher at the top and decreases as the height of the tree increases.

.. mermaid::
:alt: Evaluation order of PodTemplates

graph BT
B["@task pod_template"] --> A["k8s plugin"]
C["runtime PodTemplate"] --> B
D["@task pod_template_name"] --> B


To better understand how Flyte constructs task execution Pods based on Compile-time and Runtime PodTemplates,
and K8s plugin configuration options, let's take a few examples.

To better understand how Flyte constructs task execution Pods based on the default
PodTemplate and K8s plugin configuration options, let's take an example.
Example 1: Runtime PodTemplate and K8s Plugin Configuration
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think some of these examples may be redundant. Example 3 covers Example 1 and Example 2 right? OK to leave all as well.

Copy link
Contributor Author

@eapolinario eapolinario Jun 2, 2023 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, I just want to make it clear that we can mix and match (up to a point obviously, since one cannot use a default Runtime PodTemplate and a named one at the same time).

===========================================================

If you have the default PodTemplate defined in the ``flyte`` namespace
If you have a Runtime PodTemplate defined in the ``flyte`` namespace
(where FlytePropeller instance is running), then it is applied to all Pods that
Flyte creates, unless a **more specific** PodTemplate is defined in the namespace
where you start the Pod.
Expand Down Expand Up @@ -457,3 +546,193 @@ Plugin configuration (that is, list appends and map overrides). Task-specific
options are intentionally robust to provide fine-grained control over task
execution in diverse use-cases. Therefore, exploration is beyond this scope
and has therefore been omitted from this documentation.

Example 2: A Runtime and Compile-time PodTemplates
==================================================

In this example we're going to have a Runtime PodTemplate and a Compile-time PodTemplate defined in a task.

Let's say we have this Runtime PodTemplate defined in the same namespace as the one used to kick off an execution
of the task. For example:

.. code-block:: yaml

apiVersion: v1
kind: PodTemplate
metadata:
name: flyte-template
namespace: flytesnacks-development
template:
metadata:
annotations:
- annotation_1: initial-value
- bar: initial-value
spec:
containers:
- name: default
image: docker.io/rwgrim/docker-noop
terminationMessagePath: "/dev/foo"

And the definition of the Compile-time PodTemplate in a task:

.. code-block:: python

@task(
pod_template=PodTemplate(
primary_container_name="primary",
labels={
"label_1": "value-1",
"label_2": "value-2",
},
annotations={
"annotation_1": "value-1",
"annotation_2": "value-2",
},
pod_spec=V1PodSpec(
containers=[
V1Container(
name="primary",
image="a.b.c/image:v1",
command="cmd",
args=[],
),
],
)
)
)
def t1() -> int:
...

The resultant Pod is as follows:

.. code-block:: yaml

apiVersion: v1
kind: Pod
metadata:
name: example-pod
namespace: flytesnacks-development
labels:
- label_1: value-1 # from Compile-time value
- label_2: value-2 # from Compile-time value
annotations:
- annotation_1: value-1 # value overridden by Compile-time PodTemplate
- annotation_2: value-2 # from Compile-time PodTemplate
- bar: initial-value # from Runtime PodTemplate
spec:
containers:
- name: default
image: docker.io/rwgrim/docker-noop
terminationMessagePath: "/dev/foo"
- name: primary
image: a.b.c/image:v1
command: cmd
args: []
// remaining container configuration omitted

Notice how options follow the same merging rules, i.e. lists append and maps override.


Example 3: Runtime and Compile-time PodTemplates and K8s Plugin Configuration
=============================================================================

Now let's make a slightly more complicated example where now we have both Compile-time and Runtime PodTemplates being combined
with K8s Configuration.

Here's the definition of a Compile-time PodTemplate:

.. code-block:: python

@task(
pod_template=PodTemplate(
primary_container_name="primary",
labels={
"label_1": "value-compile",
"label_2": "value-compile",
},
annotations={
"annotation_1": "value-compile",
"annotation_2": "value-compile",
},
pod_spec=V1PodSpec(
containers=[
V1Container(
name="primary",
image="a.b.c/image:v1",
command="cmd",
args=[],
),
],
host_network=True,
)
)
)
def t1() -> int:
...


And a Runtime PodTemplate:

.. code-block:: yaml

apiVersion: v1
kind: PodTemplate
metadata:
name: flyte-template
namespace: flyte
template:
metadata:
labels:
- label_1: value-runtime
- label_2: value-runtime
- label_3: value-runtime
annotations:
- foo: value-runtime
- bar: value-runtime
spec:
containers:
- name: default
image: docker.io/rwgrim/docker-noop
terminationMessagePath: "/dev/foo"
hostNetwork: false

And the following K8s Plugin Configuration:

.. code-block:: yaml

plugins:
k8s:
default-labels:
- label_1: value-plugin
default-annotations:
- annotation_1: value-plugin
- baz: value-plugin

The resultant pod for that task is as follows:

.. code-block:: yaml

apiVersion: v1
kind: Pod
metadata:
name: example-pod
namespace: flytesnacks-development
labels:
- label_1: value-plugin
- label_2: value-compile
annotations:
- annotation_1: value-plugin
- annotation_2: value-compile
- foo: value-runtime
- bar: value-runtime
- baz: value-plugin
spec:
containers:
- name: default
image: docker.io/rwgrim/docker-noop
terminationMessagePath: "/dev/foo"
- name: primary
image: a.b.c/image:v1
command: cmd
args: []
// remaining container configuration omitted