Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Unable to read s3 table: Unknown scheme: s3 #2065

Closed
rspears74 opened this issue Jan 10, 2024 · 3 comments
Closed

Unable to read s3 table: Unknown scheme: s3 #2065

rspears74 opened this issue Jan 10, 2024 · 3 comments
Labels
bug Something isn't working

Comments

@rspears74
Copy link

Environment

Delta-rs version: main branch (I'm currently using this to develop because of bugs in datafusion 32, so I need datafusion 33+)

Binding: Rust

Environment:

  • Cloud provider: AWS
  • OS: MacOS/AWS Lambda
  • Other:

Bug

What happened: The specified table_uri is not valid: InvalidTableLocation("Unknown scheme: s3") when trying to read a table with open_table_with_storage_options.

What you expected to happen: The table to be opened.

How to reproduce it: Try to read a table from s3 with open_table_with_storage_options. Something like this:

async fn read_table() -> Result<DeltaTable> {
    deltalake::open_table_with_storage_options(
        path,
        storage_options
    ).await?
}

More details: Yes, I am using the "s3" feature

@rspears74 rspears74 added the bug Something isn't working label Jan 10, 2024
@rtyler
Copy link
Member

rtyler commented Jan 10, 2024

This is not a "bug" so much as a "feature" 😄

main has unreleased changes which fully remove the AWS support code into the deltalake-aws crate. In order to make that possible, the user must now let the core code what available storage handlers exist. Here's an example of this being done in the Python code here

@rspears74
Copy link
Author

Gotcha, thanks! I'll give this a shot!

@rspears74
Copy link
Author

Looks like adding deltalake::aws::register_handlers(None) does the trick

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

No branches or pull requests

2 participants