Sagemaker inference agent #2027

kumare3 · 2023-12-07T06:29:25Z

Tracking issue

Fixes flyteorg/flyte#4079

Why are the changes needed?

To deploy models and generate predictions using Sagemaker on Flyte seamlessly.

What changes were proposed in this pull request?

The Sagemaker agent lets users deploy models on Sagemaker, configure and invoke endpoints.

How was this patch tested?

Setup process

Screenshots

Check all the applicable boxes

I updated the documentation accordingly.
All new and existing tests passed.
All commits are signed-off.

Related PRs

Docs link

Signed-off-by: Ketan Umare <[email protected]>

codecov · 2023-12-07T06:37:35Z

Codecov Report

Attention: Patch coverage is 94.69697% with 21 lines in your changes are missing coverage. Please review.

Project coverage is 83.52%. Comparing base (3642ec6) to head (5a906a3).
Report is 1 commits behind head on master.

Files	Patch %	Lines
...tekitplugins/awssagemaker_inference/boto3_mixin.py	80.35%	11 Missing ⚠️
...er/flytekitplugins/awssagemaker_inference/agent.py	89.74%	4 Missing ⚠️
...ytekitplugins/awssagemaker_inference/boto3_task.py	92.59%	2 Missing ⚠️
...flytekitplugins/awssagemaker_inference/workflow.py	96.00%	2 Missing ⚠️
...tekitplugins/awssagemaker_inference/boto3_agent.py	97.29%	1 Missing ⚠️
...tions/flytekitplugins/great_expectations/schema.py	75.00%	1 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##           master    #2027      +/-   ##
==========================================
- Coverage   83.76%   83.52%   -0.24%     
==========================================
  Files         329      324       -5     
  Lines       25002    24578     -424     
  Branches     3708     3496     -212     
==========================================
- Hits        20943    20529     -414     
+ Misses       3432     3420      -12     
- Partials      627      629       +2

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

Signed-off-by: Ketan Umare <[email protected]>

Signed-off-by: Samhita Alla <[email protected]>

Future-Outlier · 2024-01-04T09:25:44Z

If this PR needs my help, feel free to ping me, thanks a lot!

Signed-off-by: Samhita Alla <[email protected]>

plugins/flytekit-aws-sagemaker/flytekitplugins/awssagemaker/workflow.py

Signed-off-by: Samhita Alla <[email protected]>

* [wip] Sagemaker serving agent Signed-off-by: Ketan Umare <[email protected]> * added sync agent example Signed-off-by: Ketan Umare <[email protected]> * initial version Signed-off-by: Samhita Alla <[email protected]> * add deployment workflow Signed-off-by: Samhita Alla <[email protected]> * add a workflow to delete sagemaker deployment Signed-off-by: Samhita Alla <[email protected]> * clean up Signed-off-by: Samhita Alla <[email protected]> * modify dict update logic, create sagemaker deployment tasks Signed-off-by: Samhita Alla <[email protected]> * nit Signed-off-by: Samhita Alla <[email protected]> * dockerfile and setup changes Signed-off-by: Samhita Alla <[email protected]> * update Signed-off-by: Samhita Alla <[email protected]> * update Signed-off-by: Samhita Alla <[email protected]> * pin aioboto3 version Signed-off-by: Samhita Alla <[email protected]> * remove boto3 directory Signed-off-by: Samhita Alla <[email protected]> * update imports Signed-off-by: Samhita Alla <[email protected]> * boto3 update code and add tests Signed-off-by: Samhita Alla <[email protected]> * remove output type Signed-off-by: Samhita Alla <[email protected]> * add await Signed-off-by: Samhita Alla <[email protected]> * remove sync Signed-off-by: Samhita Alla <[email protected]> * modify imports Signed-off-by: Samhita Alla <[email protected]> * modify container logic Signed-off-by: Samhita Alla <[email protected]> * modify output key Signed-off-by: Samhita Alla <[email protected]> * add default container image Signed-off-by: Samhita Alla <[email protected]> * remove struct Signed-off-by: Samhita Alla <[email protected]> * add region Signed-off-by: Samhita Alla <[email protected]> * add output to gettaskresponse Signed-off-by: Samhita Alla <[email protected]> * convert to dict to str Signed-off-by: Samhita Alla <[email protected]> * revert Signed-off-by: Samhita Alla <[email protected]> * remove timeout and add creds to boto3 calls Signed-off-by: Samhita Alla <[email protected]> * add to_flyte_idl Signed-off-by: Samhita Alla <[email protected]> * subclass fix Signed-off-by: Samhita Alla <[email protected]> * invoke endpoint async Signed-off-by: Samhita Alla <[email protected]> * remove output type Signed-off-by: Samhita Alla <[email protected]> * modify create sagemaker deployment code Signed-off-by: Samhita Alla <[email protected]> * dict loop Signed-off-by: Samhita Alla <[email protected]> * add wf output Signed-off-by: Samhita Alla <[email protected]> * set lhs to an empty string for pythoninstancetask & modify param name in create deployment task Signed-off-by: Samhita Alla <[email protected]> * update tracker and delete deployment workflow Signed-off-by: Samhita Alla <[email protected]> * instance to instancetask Signed-off-by: Samhita Alla <[email protected]> * add tests Signed-off-by: Samhita Alla <[email protected]> * ruff isort Signed-off-by: Samhita Alla <[email protected]> * ruff isort Signed-off-by: Samhita Alla <[email protected]> * isort Signed-off-by: Samhita Alla <[email protected]> * add test Signed-off-by: Samhita Alla <[email protected]> * pin greatexpectations version Signed-off-by: Samhita Alla <[email protected]> * update secret name Signed-off-by: Samhita Alla <[email protected]> * add typing dict and update tracker Signed-off-by: Samhita Alla <[email protected]> * modify tracker test Signed-off-by: Samhita Alla <[email protected]> * add sync agent Signed-off-by: Samhita Alla <[email protected]> * add name Signed-off-by: Samhita Alla <[email protected]> * add syncagentexecutormixin Signed-off-by: Samhita Alla <[email protected]> * modify sync output Signed-off-by: Samhita Alla <[email protected]> * metadata to resource_meta Signed-off-by: Samhita Alla <[email protected]> * remote conversion to flyte idl Signed-off-by: Samhita Alla <[email protected]> * add output type Signed-off-by: Samhita Alla <[email protected]> * floats to ints Signed-off-by: Samhita Alla <[email protected]> * in place modification Signed-off-by: Samhita Alla <[email protected]> * chain tasks Signed-off-by: Samhita Alla <[email protected]> * great expectations revert Signed-off-by: Samhita Alla <[email protected]> * optimize float to int code Signed-off-by: Samhita Alla <[email protected]> * snake case Signed-off-by: Samhita Alla <[email protected]> * modify plugin name Signed-off-by: Samhita Alla <[email protected]> * modify plugin name Signed-off-by: Samhita Alla <[email protected]> * modify tracker tests Signed-off-by: Samhita Alla <[email protected]> * fix tests, revert tracker changes and remove pythoninstancetask Signed-off-by: Samhita Alla <[email protected]> * tracker changes revert Signed-off-by: Samhita Alla <[email protected]> * dict to Dict Signed-off-by: Samhita Alla <[email protected]> * make images optional Signed-off-by: Samhita Alla <[email protected]> * add image_name Signed-off-by: Samhita Alla <[email protected]> * debug Signed-off-by: Samhita Alla <[email protected]> * debug Signed-off-by: Samhita Alla <[email protected]> * debug Signed-off-by: Samhita Alla <[email protected]> * debug Signed-off-by: Samhita Alla <[email protected]> * debug Signed-off-by: Samhita Alla <[email protected]> * debug Signed-off-by: Samhita Alla <[email protected]> * debug Signed-off-by: Samhita Alla <[email protected]> * debug Signed-off-by: Samhita Alla <[email protected]> * debug Signed-off-by: Samhita Alla <[email protected]> * debug Signed-off-by: Samhita Alla <[email protected]> * debug Signed-off-by: Samhita Alla <[email protected]> * debug Signed-off-by: Samhita Alla <[email protected]> * debug Signed-off-by: Samhita Alla <[email protected]> * dict to Dict Signed-off-by: Samhita Alla <[email protected]> * state to phase Signed-off-by: Samhita Alla <[email protected]> * add encode mode to secrets.get Signed-off-by: Samhita Alla <[email protected]> * revert: add encode mode to secrets.get Signed-off-by: Samhita Alla <[email protected]> * add decode Signed-off-by: Samhita Alla <[email protected]> * revert decode Signed-off-by: Samhita Alla <[email protected]> * ergonomic improvements; change plugin name Signed-off-by: Samhita Alla <[email protected]> * change plugin name Signed-off-by: Samhita Alla <[email protected]> * nit Signed-off-by: Samhita Alla <[email protected]> * image check Signed-off-by: Samhita Alla <[email protected]> * add api docs Signed-off-by: Samhita Alla <[email protected]> * add api docs Signed-off-by: Samhita Alla <[email protected]> * incorporate Kevin's suggestions Signed-off-by: Samhita Alla <[email protected]> * handle scenario when the same input is present in the wf already Signed-off-by: Samhita Alla <[email protected]> * add support for region to be a user-provided input at run-time Signed-off-by: Samhita Alla <[email protected]> * modify workflow code to accommodate providing regions at runtime Signed-off-by: Samhita Alla <[email protected]> * code optimization and add region support to workflows Signed-off-by: Samhita Alla <[email protected]> * nit Signed-off-by: Samhita Alla <[email protected]> * fixed an input_types bug Signed-off-by: Samhita Alla <[email protected]> * fix images bug Signed-off-by: Samhita Alla <[email protected]> * replace endpoint_name with config Signed-off-by: Samhita Alla <[email protected]> * input_region default Signed-off-by: Samhita Alla <[email protected]> * add inputs Signed-off-by: Samhita Alla <[email protected]> * replace bug Signed-off-by: Samhita Alla <[email protected]> * add return type Signed-off-by: Samhita Alla <[email protected]> * kwtypes fix bug Signed-off-by: Samhita Alla <[email protected]> * add cache_ignore_input_vars to task template in tests Signed-off-by: Samhita Alla <[email protected]> * fix test Signed-off-by: Samhita Alla <[email protected]> * fix test Signed-off-by: Samhita Alla <[email protected]> * add test case for boto3 call method Signed-off-by: Samhita Alla <[email protected]> * lint Signed-off-by: Samhita Alla <[email protected]> * lint Signed-off-by: Samhita Alla <[email protected]> * lint Signed-off-by: Samhita Alla <[email protected]> * lint Signed-off-by: Samhita Alla <[email protected]> * lint Signed-off-by: Samhita Alla <[email protected]> * ruff version Signed-off-by: Samhita Alla <[email protected]> * ruff version Signed-off-by: Samhita Alla <[email protected]> * nit Signed-off-by: Samhita Alla <[email protected]> * docstring update Signed-off-by: Samhita Alla <[email protected]> --------- Signed-off-by: Ketan Umare <[email protected]> Signed-off-by: Samhita Alla <[email protected]> Co-authored-by: Samhita Alla <[email protected]> Signed-off-by: Jan Fiedler <[email protected]>

[wip] Sagemaker serving agent

f025b8c

Signed-off-by: Ketan Umare <[email protected]>

kumare3 requested review from wild-endeavor, eapolinario, pingsutw and cosmicBboy as code owners December 7, 2023 06:29

kumare3 marked this pull request as draft December 7, 2023 06:29

added sync agent example

5db022a

Signed-off-by: Ketan Umare <[email protected]>

pingsutw assigned kumare3 Dec 13, 2023

initial version

deed189

Signed-off-by: Samhita Alla <[email protected]>

samhita-alla self-assigned this Dec 28, 2023

samhita-alla added 5 commits December 29, 2023 19:53

add deployment workflow

56277a6

Signed-off-by: Samhita Alla <[email protected]>

add a workflow to delete sagemaker deployment

53b9fce

Signed-off-by: Samhita Alla <[email protected]>

clean up

c64da88

Signed-off-by: Samhita Alla <[email protected]>

modify dict update logic, create sagemaker deployment tasks

e2bbe08

Signed-off-by: Samhita Alla <[email protected]>

nit

6e24e75

Signed-off-by: Samhita Alla <[email protected]>

dockerfile and setup changes

1fbdf51

Signed-off-by: Samhita Alla <[email protected]>

kumare3 commented Jan 25, 2024

View reviewed changes

plugins/flytekit-aws-sagemaker/flytekitplugins/awssagemaker/workflow.py Outdated Show resolved Hide resolved

samhita-alla added 11 commits January 26, 2024 22:29

update

e5392d4

Signed-off-by: Samhita Alla <[email protected]>

update

2cfba8a

Signed-off-by: Samhita Alla <[email protected]>

pin aioboto3 version

93eb222

Signed-off-by: Samhita Alla <[email protected]>

remove boto3 directory

34afdaf

Signed-off-by: Samhita Alla <[email protected]>

update imports

979379a

Signed-off-by: Samhita Alla <[email protected]>

boto3 update code and add tests

1760ffe

Signed-off-by: Samhita Alla <[email protected]>

remove output type

78a2069

Signed-off-by: Samhita Alla <[email protected]>

add await

dab6780

Signed-off-by: Samhita Alla <[email protected]>

remove sync

36850c3

Signed-off-by: Samhita Alla <[email protected]>

modify imports

47f250e

Signed-off-by: Samhita Alla <[email protected]>

modify container logic

e601af8

Signed-off-by: Samhita Alla <[email protected]>

samhita-alla added 22 commits March 20, 2024 21:45

code optimization and add region support to workflows

6b95417

Signed-off-by: Samhita Alla <[email protected]>

nit

9ef7513

Signed-off-by: Samhita Alla <[email protected]>

fixed an input_types bug

8f1a2e3

Signed-off-by: Samhita Alla <[email protected]>

fix images bug

346c894

Signed-off-by: Samhita Alla <[email protected]>

replace endpoint_name with config

ef053ca

Signed-off-by: Samhita Alla <[email protected]>

input_region default

04fca76

Signed-off-by: Samhita Alla <[email protected]>

add inputs

03f4507

Signed-off-by: Samhita Alla <[email protected]>

replace bug

9df6cf5

Signed-off-by: Samhita Alla <[email protected]>

add return type

79a1d68

Signed-off-by: Samhita Alla <[email protected]>

kwtypes fix bug

5864d96

Signed-off-by: Samhita Alla <[email protected]>

add cache_ignore_input_vars to task template in tests

6ed332c

Signed-off-by: Samhita Alla <[email protected]>

fix test

b22e8ff

Signed-off-by: Samhita Alla <[email protected]>

fix test

85a9c08

Signed-off-by: Samhita Alla <[email protected]>

Merge remote-tracking branch 'origin/master' into sagemaker-agent

210516e

add test case for boto3 call method

fda1dd9

Signed-off-by: Samhita Alla <[email protected]>

lint

191fca3

Signed-off-by: Samhita Alla <[email protected]>

lint

0975917

Signed-off-by: Samhita Alla <[email protected]>

lint

36f4315

Signed-off-by: Samhita Alla <[email protected]>

lint

bef6a4c

Signed-off-by: Samhita Alla <[email protected]>

lint

2fecd37

Signed-off-by: Samhita Alla <[email protected]>

ruff version

a833ff1

Signed-off-by: Samhita Alla <[email protected]>

ruff version

c7bb5ac

Signed-off-by: Samhita Alla <[email protected]>

pingsutw previously approved these changes Mar 21, 2024

View reviewed changes

dosubot bot added the lgtm This PR has been approved by maintainer label Mar 21, 2024

nit

b86b4c2

Signed-off-by: Samhita Alla <[email protected]>

samhita-alla dismissed pingsutw’s stale review via b86b4c2 March 22, 2024 06:30

docstring update

5a906a3

Signed-off-by: Samhita Alla <[email protected]>

pingsutw approved these changes Mar 23, 2024

View reviewed changes

samhita-alla merged commit 224b1c9 into master Mar 24, 2024
47 of 48 checks passed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Sagemaker inference agent #2027

Sagemaker inference agent #2027

kumare3 commented Dec 7, 2023 •

edited by samhita-alla

Loading

codecov bot commented Dec 7, 2023 •

edited

Loading

Future-Outlier commented Jan 4, 2024

Sagemaker inference agent #2027

Sagemaker inference agent #2027

Conversation

kumare3 commented Dec 7, 2023 • edited by samhita-alla Loading

Tracking issue

Why are the changes needed?

What changes were proposed in this pull request?

How was this patch tested?

Setup process

Screenshots

Check all the applicable boxes

Related PRs

Docs link

codecov bot commented Dec 7, 2023 • edited Loading

Codecov Report

Future-Outlier commented Jan 4, 2024

kumare3 commented Dec 7, 2023 •

edited by samhita-alla

Loading

codecov bot commented Dec 7, 2023 •

edited

Loading