Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Milestone v0.11 #794

Merged
merged 7 commits into from
Mar 5, 2021
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
21 changes: 21 additions & 0 deletions CHANGELOG/CHANGELOG-v0.11.0.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,21 @@
# Flyte v0.11.0

## Flyte Platform
* New to flyte? https://start.flyte.org takes you through first run experience. (Thanks to @jeevb)
* [Grafana templates](https://docs.flyte.org/en/latest/howto/monitoring/index.html) for monitoring Flyte System and User Workflows.
* [Extend Flyte](https://docs.flyte.org/en/latest/plugins/index.html) docs.
* [FlyteIdl Docs](https://docs.flyte.org/projects/flyteidl/en/latest/) are published! You can learn about the core language that makes it all work.
* [Additional knob](https://github.com/flyteorg/flytepropeller/pull/219/files#diff-91657d6448dfbf87f4cecf126ad02bd668ea233edcf74e860ef4f54bdd4cb552R78) for fine tuning flyte propeller performance that speeds up executions drastically.
* OidC support for Google Idp (And other OidC compliant Idps)
* Various stabilization bugs.

## Flytekit
Since v0.16.0a2, the last flytekit milestone release, all effort has been towards stabilizing the new API. Please see the individual [releases](https://github.com/flyteorg/flytekit/releases) for detailed information. The highlights are

* Serialization/registration processes have been firmed up and utilities to ease that process introduced (not having to build a container to serialize for instance).
* Plugins structure revamped (eventually we'll move to a separate new repo entirely)
* User-facing imports have been organized into three top-level subpackages (`flytekit`, `flytekit.testing`, and `flytekit.extend`)
* Retries added to read-only Admin calls in client
* Lots of cleanup and additions to the [cookbook](https://flytecookbook.readthedocs.io/en/latest/) and documentation generally.
* Bug fixes.

14 changes: 7 additions & 7 deletions deployment/eks/flyte_generated.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -8607,7 +8607,7 @@ spec:
- --config
- /etc/flyte/config/*.yaml
- serve
image: ghcr.io/lyft/flyteadmin:v0.3.29
image: ghcr.io/flyteorg/flyteadmin:v0.3.37
imagePullPolicy: IfNotPresent
name: flyteadmin
ports:
Expand Down Expand Up @@ -8653,7 +8653,7 @@ spec:
- /etc/flyte/config/*.yaml
- migrate
- run
image: ghcr.io/lyft/flyteadmin:v0.3.29
image: ghcr.io/flyteorg/flyteadmin:v0.3.37
imagePullPolicy: IfNotPresent
name: run-migrations
volumeMounts:
Expand All @@ -8671,7 +8671,7 @@ spec:
- flytesnacks
- flytetester
- flyteexamples
image: ghcr.io/lyft/flyteadmin:v0.3.29
image: ghcr.io/flyteorg/flyteadmin:v0.3.37
imagePullPolicy: IfNotPresent
name: seed-projects
volumeMounts:
Expand All @@ -8686,7 +8686,7 @@ spec:
- /etc/flyte/config/*.yaml
- clusterresource
- sync
image: ghcr.io/lyft/flyteadmin:v0.3.29
image: ghcr.io/flyteorg/flyteadmin:v0.3.37
imagePullPolicy: IfNotPresent
name: sync-cluster-resources
volumeMounts:
Expand Down Expand Up @@ -8733,7 +8733,7 @@ spec:
- envFrom:
- configMapRef:
name: flyte-console-config
image: ghcr.io/lyft/flyteconsole:v0.19.1
image: ghcr.io/flyteorg/flyteconsole:v0.19.6
name: flyteconsole
ports:
- containerPort: 8080
Expand Down Expand Up @@ -8777,7 +8777,7 @@ spec:
valueFrom:
fieldRef:
fieldPath: metadata.name
image: ghcr.io/flyteorg/flytepropeller:v0.6.1
image: ghcr.io/flyteorg/flytepropeller:v0.6.2
imagePullPolicy: IfNotPresent
name: flytepropeller
ports:
Expand Down Expand Up @@ -8965,7 +8965,7 @@ spec:
- /etc/flyte/config/*.yaml
- clusterresource
- sync
image: ghcr.io/lyft/flyteadmin:v0.3.29
image: ghcr.io/flyteorg/flyteadmin:v0.3.37
imagePullPolicy: IfNotPresent
name: sync-cluster-resources
volumeMounts:
Expand Down
14 changes: 7 additions & 7 deletions deployment/gcp/flyte_generated.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -8639,7 +8639,7 @@ spec:
- --config
- /etc/flyte/config/*.yaml
- serve
image: ghcr.io/lyft/flyteadmin:v0.3.29
image: ghcr.io/flyteorg/flyteadmin:v0.3.37
imagePullPolicy: IfNotPresent
name: flyteadmin
ports:
Expand Down Expand Up @@ -8685,7 +8685,7 @@ spec:
- /etc/flyte/config/*.yaml
- migrate
- run
image: ghcr.io/lyft/flyteadmin:v0.3.29
image: ghcr.io/flyteorg/flyteadmin:v0.3.37
imagePullPolicy: IfNotPresent
name: run-migrations
volumeMounts:
Expand All @@ -8703,7 +8703,7 @@ spec:
- flytesnacks
- flytetester
- flyteexamples
image: ghcr.io/lyft/flyteadmin:v0.3.29
image: ghcr.io/flyteorg/flyteadmin:v0.3.37
imagePullPolicy: IfNotPresent
name: seed-projects
volumeMounts:
Expand All @@ -8718,7 +8718,7 @@ spec:
- /etc/flyte/config/*.yaml
- clusterresource
- sync
image: ghcr.io/lyft/flyteadmin:v0.3.29
image: ghcr.io/flyteorg/flyteadmin:v0.3.37
imagePullPolicy: IfNotPresent
name: sync-cluster-resources
volumeMounts:
Expand Down Expand Up @@ -8765,7 +8765,7 @@ spec:
- envFrom:
- configMapRef:
name: flyte-console-config
image: ghcr.io/lyft/flyteconsole:v0.19.1
image: ghcr.io/flyteorg/flyteconsole:v0.19.6
name: flyteconsole
ports:
- containerPort: 8080
Expand Down Expand Up @@ -8809,7 +8809,7 @@ spec:
valueFrom:
fieldRef:
fieldPath: metadata.name
image: ghcr.io/flyteorg/flytepropeller:v0.6.1
image: ghcr.io/flyteorg/flytepropeller:v0.6.2
imagePullPolicy: IfNotPresent
name: flytepropeller
ports:
Expand Down Expand Up @@ -8997,7 +8997,7 @@ spec:
- /etc/flyte/config/*.yaml
- clusterresource
- sync
image: ghcr.io/lyft/flyteadmin:v0.3.29
image: ghcr.io/flyteorg/flyteadmin:v0.3.37
imagePullPolicy: IfNotPresent
name: sync-cluster-resources
volumeMounts:
Expand Down
14 changes: 7 additions & 7 deletions deployment/sandbox/flyte_generated.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -2723,7 +2723,7 @@ spec:
- --config
- /etc/flyte/config/*.yaml
- serve
image: ghcr.io/lyft/flyteadmin:v0.3.29
image: ghcr.io/flyteorg/flyteadmin:v0.3.37
imagePullPolicy: IfNotPresent
name: flyteadmin
ports:
Expand Down Expand Up @@ -2775,7 +2775,7 @@ spec:
- /etc/flyte/config/*.yaml
- migrate
- run
image: ghcr.io/lyft/flyteadmin:v0.3.29
image: ghcr.io/flyteorg/flyteadmin:v0.3.37
imagePullPolicy: IfNotPresent
name: run-migrations
volumeMounts:
Expand All @@ -2792,7 +2792,7 @@ spec:
- seed-projects
- flytesnacks
- flyteexamples
image: ghcr.io/lyft/flyteadmin:v0.3.29
image: ghcr.io/flyteorg/flyteadmin:v0.3.37
imagePullPolicy: IfNotPresent
name: seed-projects
volumeMounts:
Expand All @@ -2807,7 +2807,7 @@ spec:
- /etc/flyte/config/*.yaml
- clusterresource
- sync
image: ghcr.io/lyft/flyteadmin:v0.3.29
image: ghcr.io/flyteorg/flyteadmin:v0.3.37
imagePullPolicy: IfNotPresent
name: sync-cluster-resources
volumeMounts:
Expand Down Expand Up @@ -2854,7 +2854,7 @@ spec:
- envFrom:
- configMapRef:
name: flyte-console-config
image: ghcr.io/lyft/flyteconsole:v0.19.1
image: ghcr.io/flyteorg/flyteconsole:v0.19.6
name: flyteconsole
ports:
- containerPort: 8080
Expand Down Expand Up @@ -2898,7 +2898,7 @@ spec:
valueFrom:
fieldRef:
fieldPath: metadata.name
image: ghcr.io/flyteorg/flytepropeller:v0.6.1
image: ghcr.io/flyteorg/flytepropeller:v0.6.2
imagePullPolicy: IfNotPresent
name: flytepropeller
ports:
Expand Down Expand Up @@ -3204,7 +3204,7 @@ spec:
- /etc/flyte/config/*.yaml
- clusterresource
- sync
image: ghcr.io/lyft/flyteadmin:v0.3.29
image: ghcr.io/flyteorg/flyteadmin:v0.3.37
imagePullPolicy: IfNotPresent
name: sync-cluster-resources
volumeMounts:
Expand Down
10 changes: 5 additions & 5 deletions deployment/test/flyte_generated.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -594,7 +594,7 @@ spec:
- --config
- /etc/flyte/config/*.yaml
- serve
image: ghcr.io/lyft/flyteadmin:v0.3.29
image: ghcr.io/flyteorg/flyteadmin:v0.3.37
imagePullPolicy: IfNotPresent
name: flyteadmin
ports:
Expand Down Expand Up @@ -646,7 +646,7 @@ spec:
- /etc/flyte/config/*.yaml
- migrate
- run
image: ghcr.io/lyft/flyteadmin:v0.3.29
image: ghcr.io/flyteorg/flyteadmin:v0.3.37
imagePullPolicy: IfNotPresent
name: run-migrations
volumeMounts:
Expand All @@ -663,7 +663,7 @@ spec:
- seed-projects
- flytetester
- flytesnacks
image: ghcr.io/lyft/flyteadmin:v0.3.29
image: ghcr.io/flyteorg/flyteadmin:v0.3.37
imagePullPolicy: IfNotPresent
name: seed-projects
volumeMounts:
Expand All @@ -678,7 +678,7 @@ spec:
- /etc/flyte/config/*.yaml
- clusterresource
- sync
image: ghcr.io/lyft/flyteadmin:v0.3.29
image: ghcr.io/flyteorg/flyteadmin:v0.3.37
imagePullPolicy: IfNotPresent
name: sync-cluster-resources
volumeMounts:
Expand Down Expand Up @@ -735,7 +735,7 @@ spec:
valueFrom:
fieldRef:
fieldPath: metadata.name
image: ghcr.io/flyteorg/flytepropeller:v0.6.1
image: ghcr.io/flyteorg/flytepropeller:v0.6.2
imagePullPolicy: IfNotPresent
name: flytepropeller
ports:
Expand Down
2 changes: 1 addition & 1 deletion end2end/tests/endtoend.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,7 @@ spec:
command:
- bash
- -c
image: ghcr.io/flyteorg/flytetools:85ab24ff1025327a02cbd570ab96bbdded132ce9
image: ghcr.io/flyteorg/flytetools:1fde462c72029d5f4d73ade8bff51af846598754
imagePullPolicy: IfNotPresent
name: flytetester
resources:
Expand Down
64 changes: 32 additions & 32 deletions kustomize/overlays/eks/kustomization.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -2,38 +2,38 @@ apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization

bases:
# All the resources that make up the deployment
- flyte
########
# Operators to be enabled
#############
# Dependencies to be configured
# In local we use minio, but in cloud environment use S3 / GCS / AFS / Oracle Blob store etc
# This is used for Resource pooling. On cloud you can use hosted redis (e.g. AWS elasticache)
# Contour is used to create ingress. On cloud service use the default provided ingress controllers or cloud LB's
# Add node ports for ease of use locally
- ../../base/operators/spark
- ../../base/operators/kfoperators/pytorch
# TODO Fix deployment of alb_ingress controller
#- ../../base/addons/alb_ingress
# Optional dependency
- ../../base/addons/redis
# All the resources that make up the deployment
- flyte
########
# Operators to be enabled
#############
# Dependencies to be configured
# In local we use minio, but in cloud environment use S3 / GCS / AFS / Oracle Blob store etc
# This is used for Resource pooling. On cloud you can use hosted redis (e.g. AWS elasticache)
# Contour is used to create ingress. On cloud service use the default provided ingress controllers or cloud LB's
# Add node ports for ease of use locally
- ../../base/operators/spark
- ../../base/operators/kfoperators/pytorch
# TODO Fix deployment of alb_ingress controller
#- ../../base/addons/alb_ingress
# Optional dependency
- ../../base/addons/redis

# Images that should be used
images:
# FlyteAdmin
- name: flyteadmin # match images with this name
newTag: v0.3.29 # override the tag
newName: ghcr.io/lyft/flyteadmin # override the name
# FlyteConsole
- name: flyteconsole # match images with this name
newTag: v0.19.1 # override the tag
newName: ghcr.io/lyft/flyteconsole # override the namep
# Flyte DataCatalog
- name: datacatalog # match images with this name
newTag: v0.2.130 # override the tag
newName: ghcr.io/lyft/datacatalog # override the name
# FlytePropeller
- name: flytepropeller # match images with this name
newTag: v0.6.1 # override the tag
newName: ghcr.io/flyteorg/flytepropeller # override the name
# FlyteAdmin
- name: flyteadmin # match images with this name
newTag: v0.3.37 # override the tag
newName: ghcr.io/flyteorg/flyteadmin # override the name
# FlyteConsole
- name: flyteconsole # match images with this name
newTag: v0.19.6 # override the tag
newName: ghcr.io/flyteorg/flyteconsole # override the namep
# Flyte DataCatalog
- name: datacatalog # match images with this name
newTag: v0.2.130 # override the tag
newName: ghcr.io/lyft/datacatalog # override the name
# FlytePropeller
- name: flytepropeller # match images with this name
newTag: v0.6.2 # override the tag
newName: ghcr.io/flyteorg/flytepropeller # override the name
Loading