-
Notifications
You must be signed in to change notification settings - Fork 301
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Single Hive query nodes #1
Conversation
This PR deprecates the QueryCollection field in the QuboleHiveJob custom plugin object. Please see the SDK PR for more information: flyteorg/flytekit#1
@matthewphsmith - had to roll back the propeller change due to an issue haytham found in the updated plugins but the fix for it should be deployed tomorrow. After which this change can go out. Could you take another look please? Thank you. |
ptal @matthewphsmith |
wait ptal doesn't do anything anymore. |
flytekit/__init__.py
Outdated
@@ -1,4 +1,5 @@ | |||
from __future__ import absolute_import | |||
import flytekit.plugins | |||
|
|||
__version__ = '0.1.9' | |||
__version__ = '0.1.10' |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
We should make 0.2.0 so we mark this as a feature change.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I say make 0.2.0 for cleanliness.
Single Hive query nodes
Signed-off-by: Sean Lin <[email protected]>
This PR deprecates the QueryCollection field in the QuboleHiveJob custom plugin object. Please see the SDK PR for more information: flyteorg/flytekit#1
This PR deprecates the QueryCollection field in the QuboleHiveJob custom plugin object. Please see the SDK PR for more information: flyteorg/flytekit#1 Signed-off-by: Eduardo Apolinario <[email protected]>
Hive queries were originally bundled into one large dynamic node. This was a flawed design as it made the execution side of things far more complex. This now splits out each individual query into different nodes. See the corresponding change in the IDL flyteorg/flyteidl#3.