Time32/Time64 deserialization is not yet implemented

[christophe-petitjean]

Environment: WASM

When reading a csv file where one of the columns is typed as Time32, I got an error:

xxx_bg.js?t=1675332350159:464 panicked at 'called `Result::unwrap()` on an `Err` value: NotYetImplemented("Deserializing type \"Time32(Millisecond)\" is not implemented")', src/csv.rs:50:91

Stack:

Error
    at http://localhost:5173/wasm/bin/xxx_bg.js?t=1675332350159:471:17
    at logError (http://localhost:5173/wasm/bin/xxx_bg.js?t=1675332350159:223:18)
    at __wbg_new_abda76e883ba8a5f (http://localhost:5173/wasm/bin/xxx_bg.js?t=1675332350159:470:55)
    at console_error_panic_hook::Error::new::h35b6a75325e86198 (http://localhost:5173/wasm/bin/xxx_bg.wasm:wasm-function[5890]:0x1f8374)
    at console_error_panic_hook::hook_impl::h823791faa8b4b2b4 (http://localhost:5173/wasm/bin/xxx_bg.wasm:wasm-function[1161]:0x138ac2)
    at console_error_panic_hook::hook::h4ee5a0e6f6e3907a (http://localhost:5173/wasm/bin/xxx_bg.wasm:wasm-function[6574]:0x2017b8)
    at core::ops::function::Fn::call::h78a30b0e74b14f95 (http://localhost:5173/wasm/bin/xxx_bg.wasm:wasm-function[5676]:0x1f4e62)
    at std::panicking::rust_panic_with_hook::hb42d415afcc11f2f (http://localhost:5173/wasm/bin/xxx_bg.wasm:wasm-function[2516]:0x198272)
    at std::panicking::begin_panic_handler::{{closure}}::h57dd84c078404aa2 (http://localhost:5173/wasm/bin/xxx_bg.wasm:wasm-function[3244]:0x1b8aa4)
    at std::sys_common::backtrace::__rust_end_short_backtrace::hef94dcc71d3b37f5 (http://localhost:5173/wasm/bin/xxx_bg.wasm:wasm-function[6783]:0x2041af)

It seems that deserializing of Time32 and Time64 is not implemented in io/csv/read_util.rs/deserialize_column function

[jorgecarleitao]

agree - would you like to address it, or should I do it?

[christophe-petitjean]

I'd definitely like to try but I never did any contribution to an open source project and I'm almost a newbie in Rust, so I'd need some guidance :)

[jorgecarleitao]

@christophe-petitjean we are all here to learn :)

From our perspective, reading any format is composed by two main parts: type inference (what type is column X) and deserialization (map bytes to the type).

By the error, you passed a specific type to the deserialization, but the deserializer does not know how to map the bytes to that type. This is where we need to work on for this issue.

The entry point of this is here: https://github.com/jorgecarleitao/arrow2/blob/main/src/io/csv/read_utils.rs#L148

We need essentially a new branch on that function for time, and write the function itself.

From here, we can start to think about the deserialization of the times themselves. One approach here is to use chrono and map XX:YY to an integer representing time via chromo, very similar to how we do it in deserialize_datetime, and then map it to seconds/milliseconds/etc according to the logical type.

Does this make sense? Let me know if you need more guidance or help.

[christophe-petitjean]

Thanks for your help ! Yes it makes sense, I'll work on it as soon as I'm back from vacation (in a week)

[AudriusButkevicius]

#1425 is merged, should be closed?

[christophe-petitjean]

Yes I think so. Can I let you close it? I don't know if there is some process to follow

[AudriusButkevicius]

I don't have the perms, was merely pointing out that I think this os fixed.