Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

polars-cli complain that you're using scan_csv to read a compressed file even when using read_csv #60

Open
2 tasks done
apcamargo opened this issue Mar 7, 2024 · 1 comment
Labels
bug Something isn't working

Comments

@apcamargo
Copy link

Checks

  • I have checked that this issue has not already been reported.

  • I have confirmed this bug exists on the latest version of the Polars CLI.

Reproducible example

〉select * FROM read_csv('data.tsv.gz');
Error: cannot scan compressed csv; use `read_csv` for compressed data


### Issue description

`polars-cli` complains that it can't scan compressed files, and suggests using `read_csv`. My understanding here is that`polars-cli` uses `scan_csv` behind the scenes even though the command we use is called `read_csv`. In this case, maybe the error message should be adjusted to make sense in the context of `polars-cli`?

### Expected behavior

The function should either (1) be able to read compressed files (which would require using `read_csv` as a backend), or (2) have an error message saying that `polars-cli` can't read compressed files (or suggesting an alternative way of reading those)

### Installed version

0.7.0
@apcamargo apcamargo added the bug Something isn't working label Mar 7, 2024
@apcamargo
Copy link
Author

I've been looking at table_functions.rs and noticed that read_csv there actually uses a LazyCsvReader. This got me thinking, because if you're familiar with the Python API, read_csv there doesn't give you a lazy frame. So, it kind of feels like the name read_csv might be a bit misleading. Wouldn't scan_csv be a more fitting name? Although, I'm sure there's a reason for these naming choices.

Also, I'm curious if there's a way to use the eager API with these table functions. What's the use case for the table functions, aside from their use in polars-cli? Is there any documentation available that goes into more detail about this?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

No branches or pull requests

1 participant