-
Notifications
You must be signed in to change notification settings - Fork 1.3k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Support for !=
predicate in pruning predicates
#420
Comments
@alamb Can I try to solve this issue ? If I understand it correctly, for the non equal predicate the expression should be pruned if the literal value does not fall between the min and max values, Am I right ? |
Thanks @jgoday ! That would be great For |
apache#544)" This reverts commit 2568323.
Is your feature request related to a problem or challenge? Please describe what you are trying to do.
While evaluating queries against data stored in containers / multiple files, it is helpful to prune entire files using statistics (see #363 for more details). DataFusion has this logic already for
==
,<
,<=
,>
and>=
operators in the pruning predicate.However, as @NGA-TRAN noticed, there is no support for
!=
at the moment.https://github.com/apache/arrow-datafusion/blob/14f1eebef068a9e65f556ed74d2b6d98376c97f4/datafusion/src/physical_plan/parquet.rs#L683
Describe the solution you'd like
Add support and tests for
!=
in predicate pruning logicThe text was updated successfully, but these errors were encountered: