Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Plugin] Sklearn integration #2752

Closed
Tracked by #2917
cosmicBboy opened this issue Aug 9, 2022 · 4 comments · Fixed by flyteorg/flytekit#1244
Closed
Tracked by #2917

[Plugin] Sklearn integration #2752

cosmicBboy opened this issue Aug 9, 2022 · 4 comments · Fixed by flyteorg/flytekit#1244
Labels
good first issue Good for newcomers hacktoberfest plugins Plugins related labels (backend or frontend)

Comments

@cosmicBboy
Copy link
Contributor

cosmicBboy commented Aug 9, 2022

Flyte is an ML- and data- aware orchestrator, it would make sense to support the most popular ML frameworks as first-class citizen types in the Flyte type system.

For sklearn, there's really only one base class, which is the BaseEstimator.

We'll need to follow the conventions set in the flytekit.extras.pytorch module, by doing the following:

  1. the __init__.py file should handle the case where the user doesn't have sklearn and joblib installed.
  2. implement a TypeTransformer for the BaseEstimator class:
  3. add tests here: https://github.com/flyteorg/flytekit/tree/master/tests/flytekit/unit/extras. Test out:

Resources

Open Questions

  • joblib/sklearn will complain if the an estimator is serialized in some version x and deserialized in another version y, but this is just a user warning... how do we handle this?
@SmritiSatyanV
Copy link
Contributor

Is this something I can take up? @cosmicBboy

@kumare3
Copy link
Contributor

kumare3 commented Aug 13, 2022

@cosmicBboy can we add a more detailed description- I think this is a great good first issue for open source contributors?

@cosmicBboy
Copy link
Contributor Author

assigned @SmritiSatyanV to this, gonna fill out #2759 and #2753 for others to pick up.

@kumare3 as a side note, I think we should host OSS planning meetings (similar to UnionML) so that we can get better involvement.

@techytushar
Copy link

I can pick this up, I have already worked on adding TypeTransformer for Tensorflow Models here: flyteorg/flytekit#1241
this seems to be very similar to it

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
good first issue Good for newcomers hacktoberfest plugins Plugins related labels (backend or frontend)
Projects
None yet
Development

Successfully merging a pull request may close this issue.

5 participants