Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix: cases sensitivity for statement options keywords #2231

Merged
merged 2 commits into from
Dec 14, 2023

Conversation

seve-martinez
Copy link
Contributor

@seve-martinez seve-martinez commented Dec 8, 2023

Closes #2199 .

Prevents casing issues when using the CLI by forcing statement option keys to be lowercase:

smartinez in ~/src/glaredb on smartinez/location_casing_error λ cargo run --bin glaredb local -q "CrEaTe ExTeRnAl TaBlE supplier fRoM lOcAl (LoCaTiOn 'testdata/parquet/userdata1.parquet')"

Table created

@seve-martinez
Copy link
Contributor Author

@universalmind303 I had originally made this change in the parser logic, but that caused failures in the case-sensitive slt checks. Is #2199 only specific to CLi-based inputs?

@universalmind303
Copy link
Contributor

@universalmind303 I had originally made this change in the parser logic, but that caused failures in the case-sensitive slt checks. Is #2199 only specific to CLi-based inputs?

This should definitely happen in the parser only for the location keyword. If we lowercase the entire query, it'll result in some undesired results

select * from "MyTable" should not be evaluated the same as select * from "mytable". SQL is not case sensitive on keywords, but identifiers are allowed to be case sensitive. the sqlparser crate already handles all of the non custom keywords, so we should just need to handle the parsing of the location keyword.

@universalmind303 universalmind303 changed the title bug: forcing cli query input to lowercase fix: cases sensitivity for location keyword Dec 8, 2023
@seve-martinez
Copy link
Contributor Author

seve-martinez commented Dec 9, 2023

@universalmind303 I had originally made this change in the parser logic, but that caused failures in the case-sensitive slt checks. Is #2199 only specific to CLi-based inputs?

This should definitely happen in the parser only for the location keyword. If we lowercase the entire query, it'll result in some undesired results

select * from "MyTable" should not be evaluated the same as select * from "mytable". SQL is not case sensitive on keywords, but identifiers are allowed to be case sensitive. the sqlparser crate already handles all of the non custom keywords, so we should just need to handle the parsing of the location keyword.

If i'm following the logic correctly, location is a statement option that's acquired using remove_required or remove_optional:

let account = m.remove_required_or("account_name", account)?;
let access_key = m.remove_required_or("access_key", access_key)?;
let location: String = m.remove_required("location")?;

None => m.remove_required("location")?,

let host = m.remove_required("host")?;
let port = m.remove_optional("port")?;
let user = m.remove_required("user")?;
let password = m.remove_optional("password")?;

I did not see any calls to those methods that didn't use a lowercase key, so in that case, would it be best to handle this in remove_required/optional method on StmtOpts?

pub fn remove_required<T>(&mut self, k: &str) -> Result<T, ParserError>

Or is it as simple as just checking whether this identifier is location? :

let key = self.parser.parse_identifier()?.value;

@universalmind303
Copy link
Contributor

I think it'd be safe to assume all keys inside options should be case insensitive. I'd expect it to follow SQL semantics where a key of location would be case insensitive, but a key of "Location" would preserve it's casing. Since we don't support quoted keys inside options anyways, I wouldn't worry too much about the latter case for now.

I think we can just .to_lowercase() all of the keys before inserting into the object map.

@seve-martinez seve-martinez force-pushed the smartinez/location_casing_error branch from 20aa893 to 73ed435 Compare December 10, 2023 01:20
@universalmind303
Copy link
Contributor

@seve-martinez PR looks good, can we add a slt (sql logic tests) for various casings?

@seve-martinez seve-martinez force-pushed the smartinez/location_casing_error branch from 73ed435 to 8df60a4 Compare December 13, 2023 00:49
@seve-martinez
Copy link
Contributor Author

@seve-martinez PR looks good, can we add a slt (sql logic tests) for various casings?

I added some to the external_table.slt file. Let me know if those look okay.

Copy link
Contributor

@tychoish tychoish left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

  • the title of the pr seems like it should be updated as it's really all option keywords
  • are there other or additional tests that might be useful (I think we're good, but I wanted to check).

@seve-martinez seve-martinez changed the title fix: cases sensitivity for location keyword fix: cases sensitivity for statement options keywords Dec 14, 2023
@tychoish tychoish merged commit 691db7c into GlareDB:main Dec 14, 2023
8 of 9 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

external table location keyword shouldn't be case sensitive.
3 participants