You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Nested JSON fields are unfortunately common in some parquet and JSON files. While polars provides a json_path_match function, it can only return a string in the current implementation. This makes it of limited value when working with nested JSON fields of various types within a Utf8Chunked array. I did a POC locally showing that correct typing can be achieved with a few tweaks to json_path_match, and a modification of the deserialization approach in arrow2. I put up a PR on arrow2 (jorgecarleitao/arrow2#989) to address the underlying support for JSON parsing from a arrow2::array::Utf8Array, and will follow up with a PR that uses that support to output a Series with proper type.
When this is added, I think the API for json_path_match should change as follows:
Nested JSON fields are unfortunately common in some parquet and JSON files. While
polars
provides ajson_path_match
function, it can only return a string in the current implementation. This makes it of limited value when working with nested JSON fields of various types within a Utf8Chunked array. I did a POC locally showing that correct typing can be achieved with a few tweaks tojson_path_match
, and a modification of the deserialization approach inarrow2
. I put up a PR onarrow2
(jorgecarleitao/arrow2#989) to address the underlying support for JSON parsing from aarrow2::array::Utf8Array
, and will follow up with a PR that uses that support to output aSeries
with proper type.When this is added, I think the API for
json_path_match
should change as follows:The text was updated successfully, but these errors were encountered: