Skip to content
This repository has been archived by the owner on Oct 9, 2023. It is now read-only.

feat: add support for inter-cluster communication for Ray plugin #321

Merged
merged 7 commits into from
Mar 20, 2023

Conversation

aybidi
Copy link
Contributor

@aybidi aybidi commented Feb 24, 2023

TL;DR

This PR addresses the concern "Supporting customized k8s client in various k8s plugins" but narrows it down to the ray plugin. In short, users can configure ray plugin to do inter-cluster communication, essentially submitting a custom resource to a specified k8s cluster instead of the one flytepropeller runs on.

Type

  • Bug Fix
  • Feature
  • Plugin

Are all requirements met?

  • Code completed
  • Smoke tested
  • Unit tests added
  • Code documentation added
  • Any pending items have an associated Issue

Complete description

How did you fix the bug, make the feature etc. Link to any design docs etc

Tracking Issue

flyteorg/flyte#2883

Follow-up issue

NA
OR
https://github.com/flyteorg/flyte/issues/

@welcome
Copy link

welcome bot commented Feb 24, 2023

Thank you for opening this pull request! 🙌

These tips will help get your PR across the finish line:

  • Most of the repos have a PR template; if not, fill it out to the best of your knowledge.
  • Sign off your commits (Reference: DCO Guide).

@codecov
Copy link

codecov bot commented Feb 27, 2023

Codecov Report

Merging #321 (a05851e) into master (5981c35) will increase coverage by 1.18%.
The diff coverage is 33.33%.

❗ Current head a05851e differs from pull request most recent head b528e3d. Consider uploading reports for the commit b528e3d to get more accurate results

@@            Coverage Diff             @@
##           master     #321      +/-   ##
==========================================
+ Coverage   62.90%   64.09%   +1.18%     
==========================================
  Files         146      146              
  Lines       12162     9873    -2289     
==========================================
- Hits         7651     6328    -1323     
+ Misses       3934     2967     -967     
- Partials      577      578       +1     
Flag Coverage Δ
unittests ?

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files Coverage Δ
...s/plugins/k8s/kfoperators/tensorflow/tensorflow.go 75.00% <ø> (-0.48%) ⬇️
go/tasks/plugins/k8s/ray/config.go 50.00% <ø> (ø)
go/tasks/plugins/k8s/ray/ray.go 77.01% <0.00%> (-3.45%) ⬇️
go/tasks/plugins/webapi/databricks/plugin.go 66.32% <0.00%> (+3.39%) ⬆️
go/tasks/plugins/webapi/snowflake/plugin.go 66.25% <0.00%> (+4.01%) ⬆️
go/tasks/pluginmachinery/flytek8s/pod_helper.go 71.38% <60.00%> (-3.90%) ⬇️

... and 125 files with indirect coverage changes

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

@aybidi aybidi force-pushed the ray-plugin-inter-cluster branch from 45cfdb9 to 32016aa Compare March 16, 2023 19:07
aybidi added 2 commits March 16, 2023 16:21
Signed-off-by: Abdullah Mobeen <[email protected]>
Signed-off-by: Abdullah Mobeen <[email protected]>
@aybidi aybidi marked this pull request as ready for review March 17, 2023 12:40
Co-authored-by: Kevin Su <[email protected]>
Signed-off-by: Abdullah Mobeen <[email protected]>
@hamersaw hamersaw merged commit 0a681cd into flyteorg:master Mar 20, 2023
@welcome
Copy link

welcome bot commented Mar 20, 2023

Congrats on merging your first pull request! 🎉

eapolinario pushed a commit that referenced this pull request Sep 6, 2023
* feat: add inter-cluster com to ray plugin

Signed-off-by: Abdullah Mobeen <[email protected]>

* test: add tests for remote cluster

Signed-off-by: Abdullah Mobeen <[email protected]>

* refactor: move auth to config

Signed-off-by: Abdullah Mobeen <[email protected]>

* fix description

Co-authored-by: Kevin Su <[email protected]>
Signed-off-by: Abdullah Mobeen <[email protected]>

* refactor: move clusterconfig and auth to pluginmachinery k8s

Signed-off-by: Abdullah Mobeen <[email protected]>

* refactor: move clusterconfig and auth to pluginmachinery k8s

Signed-off-by: Abdullah Mobeen <[email protected]>

* chore: remove commented out code

Signed-off-by: Abdullah Mobeen <[email protected]>

---------

Signed-off-by: Abdullah Mobeen <[email protected]>
Signed-off-by: Abdullah Mobeen <[email protected]>
Co-authored-by: Kevin Su <[email protected]>
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants