Skip to content
This repository has been archived by the owner on Feb 18, 2024. It is now read-only.

Parquet reading cannot handle negative values of type Decimal #676

Closed
mdrach opened this issue Dec 13, 2021 · 0 comments · Fixed by #679 or #684
Closed

Parquet reading cannot handle negative values of type Decimal #676

mdrach opened this issue Dec 13, 2021 · 0 comments · Fixed by #679 or #684

Comments

@mdrach
Copy link
Contributor

mdrach commented Dec 13, 2021

When I read a parquet file that contains a negative value of type Decimal, underflow occurs.

How to repro:

Change int64 input here
https://github.com/jorgecarleitao/arrow2/blob/main/parquet_integration/write_parquet.py#L52
from int64 = [0, 1, 2, 3, 4, 5, 6, 7, 8, 9] -> int64 = [-1, 0, 1, 2, 3, 4, 5, 6, 7, 8]

Output:

thread 'io::parquet::read::v1_decimal_18_required' panicked at 'assertion failed: `(left == right)`
  left: `PrimitiveArray { data_type: Decimal(18, 0), values: [-1, 0, 1, 2, 3, 4, 5, 6, 7, 8], validity: None, offset: 0 }`,
 right: `PrimitiveArray { data_type: Decimal(18, 0), values: [18446744073709551615, 0, 1, 2, 3, 4, 5, 6, 7, 8], validity: None, offset: 0 }`', tests/it/io/parquet/read.rs:48:5

I'm working on a fix.

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
1 participant