Feature Request: Predicate Pushdown

[rajasekarv]

It will be hugely beneficial to have more abstract and more functional predicate pushdown filters while reading row groups and while reading individual records.

Ref: https://github.com/apache/parquet-mr/blob/master/parquet-column/src/main/java/org/apache/parquet/filter2/predicate/FilterPredicate.java

[jorgecarleitao]

Quite a coincidence!

We already support filter pushdown at row group level and page level.

The page level filter still requires reading the page header atm.

We added support to read bloom filters yesterday, and there is a PR to read column indexes and page indexes.

I am working on allowing to use these indexes to be used on our page iterators, as this requires piping things around.

Happy to hear ideas on how the API could look like for an end user.

[jorgecarleitao]

With #102 landed, all types of filter push-down in parquet's spec are fully supported in main. Closing as done.

Thanks again @rajasekarv for your interest.