Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Read string as Utf8View in sqllogictest #13408

Open
Tracked by #13504
jayzhan211 opened this issue Nov 14, 2024 · 4 comments
Open
Tracked by #13504

Read string as Utf8View in sqllogictest #13408

jayzhan211 opened this issue Nov 14, 2024 · 4 comments
Labels
enhancement New feature or request

Comments

@jayzhan211
Copy link
Contributor

jayzhan211 commented Nov 14, 2024

Is your feature request related to a problem or challenge?

This is the simplest way to create a string column

statement ok
create table t(a varchar) as values ('1'), ('2');

query T
select arrow_typeof(a) from t;
----
Utf8
Utf8

statement ok
drop table t

I'm finding a simple way to create Utf8View column. We can achieve this with casting (See datafusion/sqllogictest/test_files/string/string_view.slt) but I think we can have a more simpler way to do this.

Describe the solution you'd like

I have two idea but not sure which one is better

Approach 1

Set the configuration with use_string_view = true, and we will read the column (varchar) as string view type instead of string type.

Something like

set datafusion.xxx.use_string_view = true

statement ok
create table t(a varchar) as values ('1'), ('2');

query T
select arrow_typeof(a) from t;
----
Utf8View
Utf8View

statement ok
drop table t

This avoid adding new syntax.

Approach 2

introduce string_view syntax like varchar_view so we know we want to read it as string view type

Something like

statement ok
create table t(a varchar_view) as values ('1'), ('2');

query T
select arrow_typeof(a) from t;
----
Utf8View
Utf8View

statement ok
drop table t

This adds complexity of understanding of string view type for people from Postgres.

Describe alternatives you've considered

No response

Additional context

No response

@jayzhan211 jayzhan211 added the enhancement New feature or request label Nov 14, 2024
@jayzhan211 jayzhan211 changed the title Configure string view as default type in sqllogictest Read string as Utf8View in sqllogictest Nov 14, 2024
@jayzhan211
Copy link
Contributor Author

Now I would prefer the first approach since string view type is more to a "physical type" concept, add this to parser doesn't fit very well.

@Omega359
Copy link
Contributor

The first approach 100%

@Weijun-H
Copy link
Member

vote for 1st approach

@findepi
Copy link
Member

findepi commented Nov 16, 2024

statement ok
create table t(a varchar) as values ('1'), ('2');

query T
select arrow_typeof(a) from t;

Utf8View
Utf8View

if the table t is created internally using Utf8View, then i agree that SLT should return that on query output.
if the table t is created internally using Utf8, SLT should continue to read it as Utf8, unless DF engine generally replaces Utf8View with Utf8 (which i think it does not)

So, do we need a new config after all?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

4 participants