Skip to content

Commit

Permalink
AWS EKS Illustration: Production single cluster mode (#799)
Browse files Browse the repository at this point in the history
Signed-off-by: Haytham Abuelfutuh <[email protected]>
  • Loading branch information
kumare3 authored and EngHabu committed Apr 30, 2021
1 parent 42dde79 commit f91c267
Show file tree
Hide file tree
Showing 2 changed files with 22 additions and 0 deletions.
1 change: 1 addition & 0 deletions rsts/howto/productionize/index.rst
Original file line number Diff line number Diff line change
Expand Up @@ -10,3 +10,4 @@ How Do I productionize my Flyte cluster
:name: howtoprovguidestoc

production
production_eks
21 changes: 21 additions & 0 deletions rsts/howto/productionize/production_eks.rst
Original file line number Diff line number Diff line change
@@ -0,0 +1,21 @@
.. _production-eks:

Using AWS EKS to host Flyte
------------------------------

Illustration
*************

.. note::

- Flyte needs a prefix in an AWS S3 bucket to store all its metadata. This is where the data about executions, workflows, tasks is stored
- this S3 bucket/prefix should be accessible to all FlytePropeller, FlyteAdmin, Datacatalog and running executions (user pods)
- FlyteAdmin can use any RDBMS database but we recommend Postgres. At scale we have used AWS Aurora
- Datacatalog also uses a postgres database similar to admin. They both could share the same physical instance, but prefer to have 2 logically separate databases
- If you want to use AWS IAM role for SeviceAccounts, then you have to manage the provisioning of the service account and providing it to Flyte at the time of execution
- For secrets, you can use Vault, Kube secrets etc, we are working on getting first class support for this

.. image:: https://raw.githubusercontent.com/flyteorg/flyte/static-resources/img/core/flyte_single_cluster_eks.png
:alt: Illustration of setting up Flyte Cluster in a single AWS EKS (or any K8s cluster on AWS)


0 comments on commit f91c267

Please sign in to comment.