Encountering NotATable("No snapshot or version 0 found, perhaps xxx is an empty dir?") #1831
Labels
Milestone
Comments
|
First thing I noticed is that version is set to -1 which results in a funky offset_path "_delta_log/-0000000000000000001.json" But even when I update this function as following: pub(crate) fn commit_uri_from_version(version: i64) -> Path {
let versionToUse = if (version <= 0) { 0 } else { version };
let version = format!("{versionToUse:020}.json");
DELTA_LOG_PATH.child(version.as_str())
}
Still no files are found.. |
dimonchik-suvorov
pushed a commit
to dimonchik-suvorov/delta-rs
that referenced
this issue
Nov 19, 2023
According to the issue test should fail to load table without snapshot (version 0) but test is written to test that it is possible to read and load Delta Table with version 0 into the Rust (functions `open_table` and `open_table_with_version` work)
dimonchik-suvorov
pushed a commit
to dintegrity/delta-rs
that referenced
this issue
Nov 19, 2023
According to the issue test should fail to load table without snapshot (version 0) but test is written to test that it is possible to read and load Delta Table with version 0 into the Rust (functions open_table and open_table_with_version work)
dimonchik-suvorov
pushed a commit
to dimonchik-suvorov/delta-rs
that referenced
this issue
Nov 19, 2023
According to the issue test should fail to load table without snapshot (version 0) but test is written to test that it is possible to read and load Delta Table with version 0 into the Rust (functions `open_table` and `open_table_with_version` work)
dimonchik-suvorov
pushed a commit
to dimonchik-suvorov/delta-rs
that referenced
this issue
Nov 19, 2023
# Description According to the issue test should fail to load table without snapshot (version 0) but test is written to test that it is possible to read and load Delta Table with version 0 into the Rust (functions `open_table` and `open_table_with_version` work) # Related Issue(s) - closes delta-io#1831
|
Unfortunately, according to the test I've added I can't reproduce this issue... but this is my first Rust experience so perhaps I'm missing something... |
|
Always nice to see a friendly face 👋 @dimonchik-suvorov! I had tried this out with a Python install of the @timvw There shouldn't need to be a snapshot in the directory, when I cloned your repo I was able to load successfully with our 0.13 Python deltalake release: |
dimonchik-suvorov
pushed a commit
to dimonchik-suvorov/delta-rs
that referenced
this issue
Nov 20, 2023
# Description According to the issue test should fail to load table without snapshot (version 0) but test is written to test that it is possible to read and load Delta Table with version 0 into the Rust (functions `open_table` and `open_table_with_version` work) # Related Issue(s) - closes delta-io#1831
rtyler
pushed a commit
to dimonchik-suvorov/delta-rs
that referenced
this issue
Nov 30, 2023
According to the issue test should fail to load table without snapshot (version 0) but test is written to test that it is possible to read and load Delta Table with version 0 into the Rust (functions `open_table` and `open_table_with_version` work)
rtyler
pushed a commit
to dimonchik-suvorov/delta-rs
that referenced
this issue
Nov 30, 2023
According to the issue test should fail to load table without snapshot (version 0) but test is written to test that it is possible to read and load Delta Table with version 0 into the Rust (functions `open_table` and `open_table_with_version` work)
rtyler
pushed a commit
that referenced
this issue
Nov 30, 2023
According to the issue test should fail to load table without snapshot (version 0) but test is written to test that it is possible to read and load Delta Table with version 0 into the Rust (functions `open_table` and `open_table_with_version` work)
github-project-automation
bot
moved this from Todo
to Done
in Linux Foundation Delta Lake Roadmap
ion-elgreco
pushed a commit
to ion-elgreco/delta-rs
that referenced
this issue
Dec 1, 2023
According to the issue test should fail to load table without snapshot (version 0) but test is written to test that it is possible to read and load Delta Table with version 0 into the Rust (functions `open_table` and `open_table_with_version` work)
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Environment
Delta-rs version: 0.16.3
Binding: rust
Environment:
Bug
What happened:
Error NotATable("No snapshot or version 0 found, perhaps xxx is an empty dir?")
What you expected to happen:
No such error because the directory exists and has all (required) data. Reading just works (tm) when using apache spark or when using an earlier delta-rs version
How to reproduce it:
Simply try to load the table using the simple api:
More details:
I have this (valid) delta table available in a git repository: https://github.com/timvw/arrow-testing/tree/master/data/delta/COVID-19_NYT.
It does not contain a snaphot, it does contain a valid version 0.json file
Because there is no snapshot, the code ends up in get_latest_version
and tries to list files the files with _delta_log prefix
And there no files are found..
The text was updated successfully, but these errors were encountered: