-
Notifications
You must be signed in to change notification settings - Fork 300
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
feat: add TypeTransformer for Tensorflow Model #1241
Conversation
4ec3f78
to
42d3362
Compare
Signed-off-by: Tushar Mittal <[email protected]>
42d3362
to
1eeddfb
Compare
Signed-off-by: Tushar Mittal <[email protected]>
Adding |
Signed-off-by: Tushar Mittal <[email protected]>
…-type-transformer Signed-off-by: Tushar Mittal <[email protected]>
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Could we move these files to flytekit/extras/tensorflow/model
?
…-type-transformer Signed-off-by: Tushar Mittal <[email protected]>
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Two tests are failing. ModuleNotFoundError: No module named 'tensorflow'
You have to run make dev-requirements.txt
to update the requirements in dev-requirements.txt
, and then GA will be able to install TensorFlow.
Signed-off-by: Tushar Mittal <[email protected]>
raise ValueError(f"Transformer {self} cannot reverse {literal_type}") | ||
|
||
|
||
class TensorflowModelTransformer(TensorflowTypeTransformer[tf.keras.Model]): |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@techytushar, tf.Module
is the parent class for tf.keras.Model
and tf.keras.layers.Layer
. Can you use that here? Also, can you merge this class into the parent class cause I don't think we need to use Generic in the first place if there's only one override?
@pingsutw, what do you think about having a |
@task | ||
def get_model_layers(model: tf.keras.Model) -> List[tf.keras.layers.Layer]: | ||
return model.layers |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This test shouldn't use your type transformer; it should just pickle. So it doesn't make sense to have unless you add a type transformer for tf.Module
.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Also, I'm unsure if List[...] will work. tf.keras.layers.Layer
should, however, work after you add tf.Module
.
yes. make sense. I prefer |
Signed-off-by: Tushar Mittal <[email protected]>
Codecov Report
@@ Coverage Diff @@
## master #1241 +/- ##
==========================================
- Coverage 68.80% 68.70% -0.10%
==========================================
Files 286 292 +6
Lines 26363 26460 +97
Branches 2492 2124 -368
==========================================
+ Hits 18138 18180 +42
- Misses 7740 7801 +61
+ Partials 485 479 -6
Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here. |
Signed-off-by: Tushar Mittal <[email protected]>
@techytushar, let me know once the changes are in. |
Signed-off-by: Tushar Mittal <[email protected]>
@samhita-alla I have pushed the changes. Note: I could not use the generic Struct type for storing the layer config because in protobuf format the More info here: https://stackoverflow.com/questions/51818125/how-to-use-ints-in-a-protobuf-struct |
Signed-off-by: Tushar Mittal <[email protected]>
010413b
to
b3229cd
Compare
e95e7a1
to
8debe9f
Compare
Signed-off-by: Tushar Mittal <[email protected]>
8debe9f
to
cb2cf77
Compare
…-type-transformer Signed-off-by: Tushar Mittal <[email protected]>
@samhita-alla since its the last day of Hacktoberfest and all comments are resolved, please can you merge this PR or add label so that it can count towards my contribution |
@techytushar, I'll merge the PR later as I need to push some changes. |
fixes flyteorg/flyte#2570. We're going to need to move these changes into the tensorflow plugin @eapolinario |
Great, I'll make sure to incorporate those there. |
TL;DR
This PR adds support for:
Note: I was not able to run the
pip-compile
commands to generate the newrequirements
files, it failed onprotobuf
version errorType
Are all requirements met?
Complete description
How did you fix the bug, make the feature etc. Link to any design docs etc
Tracking Issue
flyteorg/flyte#2570
Follow-up issue
NA