CHANGELOG.md

Changelog

v0.5.3 (2021-09-14)

Full Changelog

New features:

• Added support to read and write extension types to and from parquet #396 (jorgecarleitao)

Fixed bugs:

• Fixed error writing dictionary extension to IPC #397 (jorgecarleitao)
• Fixed error in extending MutableBitmap #393 (jorgecarleitao)

Enhancements:

• Added support to read dict-encoded required primitive types from parquet #402 (Dandandan)
• Added Array::with_validity #399 (ritchie46)

Testing updates:

• Fix testing of SIMD #394 (jorgecarleitao)

v0.5.1 (2021-09-09)

Full Changelog

Documentation updates:

• [0.5] The docs io module has no submodules #390
• Made docs be compiled with feature full #391 (jorgecarleitao)

v0.5.0 (2021-09-07)

Full Changelog

Breaking changes:

• Added Extension to DataType #361
• MonthDayNano added to enum IntervalUnit #360
• Make io::parquet::write::write_* return size of file in bytes #354
• Renamed bitmap::utils::null_count to bitmap::utils::count_zeros #342
• Made GroupFilter optional in parquet'sRecordReader and added method to set it. #386 (jorgecarleitao)
• Removed PartialOrd and Ord of all enums in datatypes #379 (jorgecarleitao)
• Made cargo features not default #369 (jorgecarleitao)
• Prepare APIs for extension types #357 (jorgecarleitao)

New features:

• Added support for async parquet write #372 (GrandChaman)
• Add support to extension types in FFI #363 (jorgecarleitao)
• Added support for field's metadata via FFI #362 (jorgecarleitao)
• Added support for Extension (logical) type #359 (jorgecarleitao)
• Added support for compute to BinaryArray #346 (zhyass)
• Added support for reading binary from CSV #337 (jorgecarleitao)
• Added support for MONTH_DAY_NANO interval type #268 (jorgecarleitao)

Fixed bugs:

• Parquet read skips a few rows at the end of the page #373
• parquet_read fails when a column has too many rows with string values #366
• parquet_read panics with index_out_of_bounds #351
• Fixed error in MutableBitmap::push_unchecked #384 (jorgecarleitao)
• Fixed display of timestamp with tz. #375 (jorgecarleitao)

Enhancements:

• Added extend_*values to MutablePrimitiveArray #383 (ritchie46)
• Improved performance of writing to CSV (20-25%) #382 (jorgecarleitao)
• Bumped lexical-core #378 (jorgecarleitao)
• Fixed casting of utf8 <> Timestamp with and without timezone #376 (jorgecarleitao)
• Added Send+Sync to MutableBuffer #368 (jorgecarleitao)
• Improved performance of unary _not_ for aligned bitmaps (3x) #365 (jorgecarleitao)
• Reduced dependencies within num #353 (jorgecarleitao)
• Bumped to parquet2 v0.4 #352 (jorgecarleitao)
• Bumped tonic and prost in flight #344 (PsiACE)
• Improved null count calculation (5x) #343 (jorgecarleitao)
• Improved perf of deserializing integers from json (30%) #340 (jorgecarleitao)
• Simplified code of json schema inference #339 (jorgecarleitao)

Documentation updates:

• Moved guide examples to examples/ #387 (jorgecarleitao)
• Added more docs. #358 (jorgecarleitao)
• Improved API docs. #355 (jorgecarleitao)

Testing updates:

• Moved tests to tests/ #389 (jorgecarleitao)
• Moved compute tests to tests/ #388 (jorgecarleitao)
• Added more tests. #380 (jorgecarleitao)
• Pinned nightly in SIMD tests #364 (jorgecarleitao)
• Improved benches for take #348 (jorgecarleitao)
• Made IPC integration tests run tests that are not run by arrow-rs #278 (jorgecarleitao)

v0.4.0 (2021-08-24)

Full Changelog

Breaking changes:

• Change dictionary iterator of values from Arrays of one element to Scalars #335
• Align FFI API with arrow's C++ API #328
• Make *_compare_scalar not return Result #316
• Make io::print, get_value_display and get_display not return Result #286
• Add MetadataVersion to IPC interfaces #282
• Change DataType::Union to enable round trips in IPC #281
• Removed clone requirement in StructArray -> RecordBatch #307 (jorgecarleitao)
• Fixed error in reading a non-finished IPC stream. #302 (jorgecarleitao)
• Generalized ZipIterator to accept a BitmapIter #296 (jorgecarleitao)

New features:

• Added API to FFI Field #321 (jorgecarleitao)
• Added compare_scalar #317 (jorgecarleitao)
• Add UnionArray #283 (jorgecarleitao)

Fixed bugs:

• SliceIterator of last bytes is not correct #292
• Fixed error in displaying dictionaries with nulls in values #334 (jorgecarleitao)
• Fixed error in dict equality #333 (jorgecarleitao)
• Fixed small inconsistencies between compute::cast and compute::can_cast #295 (jorgecarleitao)
• Removed order implementation for days_ms / Interval(DayTime) #285 (jorgecarleitao)

Enhancements:

• Added support for remaining non-nested datatypes #336 (jorgecarleitao)
• Made multiversion and lexical-core optional #324 (jorgecarleitao)
• Improved performance of utf8 comparison (1.7x-4x) #322 (jorgecarleitao)
• Improved performance of boolean comparison (5x-14x) #318 (jorgecarleitao)
• Added trait TryPush #314 (jorgecarleitao)
• Added cast date32 -> i64 and date64 -> i32 #308 (ritchie46)
• Improved performance of comparison with SIMD feature flag (2x-3.5x) #305 (jorgecarleitao)
• Added support to read json to BinaryArray #304 (jorgecarleitao)
• Improved MutableFixedSizeBinaryArray #303 (jorgecarleitao)
• Improved MutablePrimitiveArray and MutableUtf8Array #299 (jorgecarleitao)
• Improved MutableBooleanArray #297 (jorgecarleitao)
• Improved performance of concatenating non-aligned validities (15x) #291 (jorgecarleitao)
• Added support for timestamps with tz and interval to io::print::write #287 (jorgecarleitao)
• Improved debug repr of buffers and bitmaps. #284 (jorgecarleitao)
• Cleaned up internals of json integration #280 (jorgecarleitao)
• Removed serde_derive dependency #279 (jorgecarleitao)
• Simplified IPC code. #277 (jorgecarleitao)
• Reduced dependencies from confi-table and enabled wasm on io_print feature. #276 (jorgecarleitao)
• Improve performance of rem_scalar/div_scalar for integer types (4x-10x) #275 (ritchie46)

Documentation updates:

• Cleaned examples and docs from old API. #330 (jorgecarleitao)
• Improved documentation #306 (jorgecarleitao)

Testing updates:

• Improved naming of testing workflows #315 (jorgecarleitao)
• Added tests to scalar API #300 (jorgecarleitao)
• Made CSV and JSON tests not use files. #290 (jorgecarleitao)
• Moved tests to integration tests #289 (jorgecarleitao)

Closed issues:

• Make parquet_read_record support async #331
• Panic due to SIMD comparison #312
• Bitmap::mutable line 155 may Panic/segfault #309
• IPC's StreamReader may abort due to excessive memory by overflowing a usized variable #301
• Improve performance of rem_scalar/div_scalar for integer types (4x-10x) #259

v0.3.0 (2021-08-11)

Full Changelog

Breaking changes:

• Renamed sum to sum_primitive #273
• Moved trait Index from array::Index to types::Index #272
• Added optional projection to IPC FileReader #271
• Added optional page_filter to parquet's RecordReader and get_page_iterator #270
• Renamed parquets' CompressionCodec to Compression #269

New features:

• Added support for FFI of dictionary-encoded arrays #267 (jorgecarleitao)
• Added support for projection pushdown on IPC files #264 (jorgecarleitao)
• Added support to read parquet asynchronously #260 (jorgecarleitao)
• Added support to filter parquet pages. #256 (jorgecarleitao)
• Added wrapping_cast to cast kernels #254 (sundy-li)
• Added support to parquet IO on wasm32 #239 (jorgecarleitao)
• Added support to round-trip dictionary arrays on parquet #232 (jorgecarleitao)
• Added Scalar API #56 (jorgecarleitao)

Fixed bugs:

• Fixed error in computing remainder of chunk iterator #262 (jorgecarleitao)
• Fixed error in slicing bitmap. #250 (jorgecarleitao)

Enhancements:

• Improve the performance in cast kernel using AsPrimitive trait in generic dispatch #252
• Poor performance in sort::sort_to_indices with limit option in arrow2 #245
• Support loading Feather v2 (IPC) files with more than 1 million tables #231
• Migrated to parquet2 v0.3 #265 (jorgecarleitao)
• Added more tests to cast and min/max #253 (jorgecarleitao)
• Prettytable is unmaintained. Change to comfy-table #251 (PsiACE)
• Added IndexRange to remove checks in hot loops #247 (jorgecarleitao)
• Make merge_sort_slices MergeSortSlices public #243 (sundy-li)

Documentation updates:

• Added example and guide section on compute #242 (jorgecarleitao)

Closed issues:

• Allow projection pushdown to IPC files #261
• Add support to write dictionary-encoded pages #211
• Make IpcWriteOptions easier to find. #120

v0.2.0 (2021-07-30)

Full Changelog

Breaking changes:

• Simplified new signature of growable API #238 (jorgecarleitao)
• Add support to merge sort with a limit #222 (sundy-li)
• Generalized sort to accept indices other than i32. #220 (jorgecarleitao)
• Added support for limited sort #218 (jorgecarleitao)

New features:

• Merge sort support limit option #221
• Introduce limit option to sort #215
• Added support for take of interval of days_ms #219 (jorgecarleitao)
• Added FFI for remaining types #213 (jorgecarleitao)

Fixed bugs:

• Filter operation on sliced utf8 arrays are incorrect #233
• Fixed error in slicing bitmap. #237 (jorgecarleitao)
• Fixed nested FFI. #212 (jorgecarleitao)

Enhancements:

• Avoid materialization of indices in filter_record_batch for single arrays #234
• Add integration tests for writing to parquet #80
• Short-circuited boolean evaluation in GrowableList #228 (ritchie46)
• Add extra inlining to speed up take #226 (Dandandan)
• Removed un-needed unsafe #225 (jorgecarleitao)

Documentation updates:

• Add documentation to guide #96
• Add git submodule command to correct the test doc #223 (sundy-li)
• Added badges to README #216 (sundy-li)
• Clarified differences with arrow crate #210 (alamb)
• Clarified differences with arrow crate #209 (alamb)

* This Changelog was automatically generated by github_changelog_generator