-
-
Notifications
You must be signed in to change notification settings - Fork 2.1k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
read_json & read_ndjson do not support scientific notation with + symbol #5687
Comments
StijnKas
changed the title
read_json & read_ndjson do not support scientific notation
read_json & read_ndjson do not support scientific notation with + symbol
Nov 30, 2022
I can take a look at this one. I expect it is an upstream issue with simd-json |
The root cause is actually from the infer schema which uses json-deserializer. |
once arrow2 is updated with the latest json-deserializer version, this should be resolved. See jorgecarleitao/arrow2#1321 |
Closed by #5781 |
StijnKas
added a commit
to pegasystems/pega-datascientist-tools
that referenced
this issue
Dec 12, 2022
…ng (#63) With pola-rs/polars#5687 closed, I've updated all import logic for pdstools to use polars over pyarrow. Since we make heavy use of ndjson, this issue was a blocker. All imports should now be much faster. Additionally, I've added some basic logging to the file import logic. * Fill SnapshotTime with nan if missing * Changed all import logic to Polars from pyarrow * Updated ValueFinder to use correct keyword for polars instead of pyarrow Co-authored-by: Stijn Kas <[email protected]>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Polars version checks
I have checked that this issue has not already been reported.
I have confirmed this bug exists on the latest version of Polars.
Issue description
It seems read_json and read_ndjson do not currently support scientific notation with + symbol. Notation with the - symbol is supported.
Reproducible example
Expected behavior
I would expect it to support the plus symbol while reading, seeing as it's supported by the official json spec https://www.json.org/json-en.html.
Installed versions
The text was updated successfully, but these errors were encountered: