Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merge on IS NULL condition doesn't work for empty table #2148

Closed
echai58 opened this issue Jan 30, 2024 · 1 comment
Closed

Merge on IS NULL condition doesn't work for empty table #2148

echai58 opened this issue Jan 30, 2024 · 1 comment
Labels
bug Something isn't working

Comments

@echai58
Copy link

echai58 commented Jan 30, 2024

Environment

Delta-rs version: 0.15.1

Binding: python


Bug

What happened:
When you try to merge on a predicate like " t.a IS NULL" and the length of the table is 0, you get a PanicException: index out of bounds: the len is 0 but the index is 0.

What you expected to happen:
This should not throw an error, like how s.a = t.a works correctly when the table is empty.

How to reproduce it:

from deltalake import DeltaTable, write_deltalake
import pyarrow as pa
import pandas as pd

data = pd.DataFrame.from_dict(
    {
        "a": [],
        "b": [],
        "c": [],
    }
)
schema = pa.schema(
    [
        ("a", pa.string()),
        ("b", pa.int32()),
        ("c", pa.int32()),
    ]
)

table = pa.Table.from_pandas(data, schema=schema)


write_deltalake(
    "test",
    table,
    mode="overwrite",
)


new_data = pd.DataFrame.from_dict(
    {
        "a": ["a", "a", "a"],
        "b": [None, 2, 4],
        "c": [5, 6, 7],
    }
)
new_table = pa.Table.from_pandas(new_data, schema=schema)

dt = DeltaTable("test")
dt.merge(
    source=new_table,
    predicate="t.a IS NULL",
    source_alias="s",
    target_alias="t",

).when_matched_update_all().when_not_matched_insert_all().execute()

More details:

@echai58 echai58 added the bug Something isn't working label Jan 30, 2024
@Blajda
Copy link
Collaborator

Blajda commented Feb 2, 2024

Fixed in #2149

@Blajda Blajda closed this as completed Feb 2, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

No branches or pull requests

2 participants