From df31c7e39a333bbf1c49a8749de347b6fae0019e Mon Sep 17 00:00:00 2001 From: Ketan Umare Date: Thu, 4 Mar 2021 22:05:18 -0800 Subject: [PATCH] AWS EKS Illustration: Production single cluster mode Signed-off-by: Ketan Umare --- rsts/howto/productionize/index.rst | 1 + rsts/howto/productionize/production_eks.rst | 21 +++++++++++++++++++++ 2 files changed, 22 insertions(+) create mode 100644 rsts/howto/productionize/production_eks.rst diff --git a/rsts/howto/productionize/index.rst b/rsts/howto/productionize/index.rst index 4104251076..d08c4a7a2b 100644 --- a/rsts/howto/productionize/index.rst +++ b/rsts/howto/productionize/index.rst @@ -10,3 +10,4 @@ How Do I productionize my Flyte cluster :name: howtoprovguidestoc production + production_eks diff --git a/rsts/howto/productionize/production_eks.rst b/rsts/howto/productionize/production_eks.rst new file mode 100644 index 0000000000..56731f738c --- /dev/null +++ b/rsts/howto/productionize/production_eks.rst @@ -0,0 +1,21 @@ +.. _production-eks: + +Using AWS EKS to host Flyte +------------------------------ + +Illustration +************* + +.. note:: + + - Flyte needs a prefix in an AWS S3 bucket to store all its metadata. This is where the data about executions, workflows, tasks is stored + - this S3 bucket/prefix should be accessible to all FlytePropeller, FlyteAdmin, Datacatalog and running executions (user pods) + - FlyteAdmin can use any RDBMS database but we recommend Postgres. At scale we have used AWS Aurora + - Datacatalog also uses a postgres database similar to admin. They both could share the same physical instance, but prefer to have 2 logically separate databases + - If you want to use AWS IAM role for SeviceAccounts, then you have to manage the provisioning of the service account and providing it to Flyte at the time of execution + - For secrets, you can use Vault, Kube secrets etc, we are working on getting first class support for this + +.. image:: https://raw.githubusercontent.com/flyteorg/flyte/static-resources/img/core/flyte_single_cluster_eks.png + :alt: Illustration of setting up Flyte Cluster in a single AWS EKS (or any K8s cluster on AWS) + +