Support Map type from parquet

[cjermain]

The Map type is reasonably common in parquet files, so being able to deserialize them would be of significant value. At first glance it looks like arrow2::array::MapArray already provides the Arrow support for this. A simple parquet example can be generated with the following code:

import pandas as pd
import pyarrow as pa
import pyarrow.parquet as pq

df = pd.DataFrame({
        'col1': pd.Series([
            [('key', 'foo'), ('value', 'bar')],
            [('key', 'foo'), ('value', 'biz')],
        ]),
        'col2': pd.Series(['fiz', 'buz'])
    }
)

udt = pa.map_(pa.string(), pa.string())
schema = pa.schema([pa.field('col1', udt), pa.field('col2', pa.string())])

table = pa.Table.from_pandas(df, schema)
pq.write_table(table, 'test.parquet')

Loading using arrow2 fails on the match statement in the parquet deserialization:

arrow2/src/io/parquet/read/deserialize/mod.rs

Line 320 in 267cac0

_ => todo!(),

[jorgecarleitao]

Agree - we should support this.

The steps I usually perform to implement this are:

• Add cases in parquet_integration/write_parquet.py
• Add the corresponding data in tests/it/io/parquet/mod.rs
• Add tests in tests/it/io/parquet/read.rs
• Implement

in no particular order

I can try to take a stab at this in 2 weeks or so - I am focused on writing nested parquet atm

[jorgecarleitao]

Addressed in #1045 :)

[cjermain]

Awesome, thanks!