CSV writing: distinguish between missing and empty strings

[ritchie46]

Currently the csv writer encodes missing data and empty strings equally. Because a csv field may be quoted, we could use that as the empty string value.

2 empty strings

"",""\n

2 missing data

,\n

Current behavior:

If we encode a single column, we seem to quote both missing and emtpy:

f = io.BytesIO()
pl.DataFrame({
    "s1": ["", None],
}).to_csv(f)

f.seek(0)
print(f.read())

b's1\n""\n""\n'

However if we encode multiple columns, we change our behavior and we don't encode any.

f = io.BytesIO()
pl.DataFrame({
    "s1": ["", None],
    "s2": ["", None],
}).to_csv(f)

f.seek(0)
print(f.read())

b's1,s2\n,\n,\n'

I think we should also be consistent no matter how many columns we encode.

[ritchie46]

This turns out to be harder than I thought in current setup. We dispatch the &[u8] to the csv crate, which handles escaping. It has no notion of a missing string vs and empty string.

One thing that could help this is doing the csv encoding only in the string serializers. This gives us more control and will likely also be faster as we don't encode any other data types. At the moment the whole row is encoded, which in case of no string data is a wasted scan.

When writing the whole row, we can just slam a delimiter between the serialized fields and end with a new line.