Docs: Clarify how networking between data plane propeller and control plane data catalog can be configured in multi-cluster deployment #5345
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
… data catalog can be configured Signed-off-by: Fabio Grätz <[email protected]>
Codecov ReportAll modified and coverable lines are covered by tests ✅
Additional details and impacted files@@ Coverage Diff @@
## master #5345 +/- ##
==========================================
- Coverage 61.10% 61.10% -0.01%
==========================================
Files 794 794
Lines 51213 51213
==========================================
- Hits 31295 31294 -1
- Misses 17037 17038 +1
Partials 2881 2881
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Sentry. |
fg91
commented
May 9, 2024
| in the control plane cluster can for instance be made available to the ``flytepropeller`` services in the data plane | ||
| clusters with an internal load balancer service (see e.g. `GKE documentation <https://cloud.google.com/kubernetes-engine/docs/how-to/internal-load-balancing#create>`_ | ||
| or `AWS Load Balancer Controller <https://kubernetes-sigs.github.io/aws-load-balancer-controller/latest/guide/service/nlb/>`_). | ||
| if the clusters use the same VPC network. | ||
|
|
There was a problem hiding this comment.
Happy to modify the formulation.
There was a problem hiding this comment.
this looks good to me.
fg91
changed the title
neverett
reviewed
May 14, 2024
There was a problem hiding this comment.
Looks OK to me docs-wise -- @davidmirror-ops would also appreciate your input here
| not exposed via the ingress by default and does not have its own authentication mechanism. The ``catalog`` service | ||
| in the control plane cluster can for instance be made available to the ``flytepropeller`` services in the data plane | ||
| clusters with an internal load balancer service (see e.g. `GKE documentation <https://cloud.google.com/kubernetes-engine/docs/how-to/internal-load-balancing#create>`_ | ||
| or `AWS Load Balancer Controller <https://kubernetes-sigs.github.io/aws-load-balancer-controller/latest/guide/service/nlb/>`_). |
There was a problem hiding this comment.
Suggested change
| or `AWS Load Balancer Controller <https://kubernetes-sigs.github.io/aws-load-balancer-controller/latest/guide/service/nlb/>`_). | |
| or `AWS Load Balancer Controller <https://kubernetes-sigs.github.io/aws-load-balancer-controller/latest/guide/service/nlb/>`_) |
eapolinario
approved these changes
May 15, 2024
| in the control plane cluster can for instance be made available to the ``flytepropeller`` services in the data plane | ||
| clusters with an internal load balancer service (see e.g. `GKE documentation <https://cloud.google.com/kubernetes-engine/docs/how-to/internal-load-balancing#create>`_ | ||
| or `AWS Load Balancer Controller <https://kubernetes-sigs.github.io/aws-load-balancer-controller/latest/guide/service/nlb/>`_). | ||
| if the clusters use the same VPC network. | ||
|
|
There was a problem hiding this comment.
this looks good to me.
robert-ulbrich-mercedes-benz
pushed a commit
to robert-ulbrich-mercedes-benz/flyte
that referenced
this pull request
Jul 2, 2024
… data catalog can be configured (flyteorg#5345) Signed-off-by: Fabio Grätz <[email protected]> Co-authored-by: Fabio Grätz <[email protected]>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Why are the changes needed?
The multi-cluster deployment documentation is slightly ambiguous on how the networking between the data plane cluster flytepropeller service and control plane cluster datacatalog service works, suggesting that datacatalog was exposed via the ingress.
I clarify that the user would need to expose datacatalog themselves and that datacatalog does not have its own auth mechanism. I also suggest to use a VPC-internal load balancer service for this purpose instead.