Skip to content
This repository has been archived by the owner on Feb 18, 2024. It is now read-only.

Unescaped " in value produces invalid json when encoded #811

Closed
SimonSchneider opened this issue Feb 5, 2022 · 0 comments · Fixed by #812
Closed

Unescaped " in value produces invalid json when encoded #811

SimonSchneider opened this issue Feb 5, 2022 · 0 comments · Fixed by #812
Labels
bug Something isn't working no-changelog Issues whose changes are covered by a PR and thus should not be shown in the changelog

Comments

@SimonSchneider
Copy link
Contributor

SimonSchneider commented Feb 5, 2022

coming here from polars:
reference issue: pola-rs/polars#2548

When encoding any value containing " to json will produce invalid json. The " is not escaped correctly. An example is available here pola-rs/polars#2548 (comment) (It comes from polars which I hope is ok, but if needed I can try making a new one for only the arrow2 part)

Some digging lead me here (" is an ascii byte)

fn utf8_serialize(value: &str, buf: &mut Vec<u8>) {

#[inline]
fn utf8_serialize(value: &str, buf: &mut Vec<u8>) {
    if value.as_bytes().is_ascii() {
        buf.reserve(value.len() + 2);
        buf.push(b'"');
        buf.extend_from_slice(value.as_bytes());
        buf.push(b'"');
    } else {
        // it may contain reserved keywords: perform roundtrip for
        // todo: avoid this roundtrip over serde_json
        serde_json::to_writer(buf, &Value::String(value.to_string())).unwrap();
    }
}
@jorgecarleitao jorgecarleitao added bug Something isn't working no-changelog Issues whose changes are covered by a PR and thus should not be shown in the changelog labels Mar 6, 2022
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
bug Something isn't working no-changelog Issues whose changes are covered by a PR and thus should not be shown in the changelog
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants