Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merge on predicate throw error on date colum: Unable to convert expression to string #2420

Closed
ibabeo opened this issue Apr 15, 2024 · 1 comment
Labels
bug Something isn't working

Comments

@ibabeo
Copy link

ibabeo commented Apr 15, 2024

Environment

Delta-rs version: 0.16.4

Binding: python

Environment:

  • OS: windows
  • python: 3.8.17

Bug

What happened:
Merge on predicate throw error when the merge key is a date column, which table is partitioned by.

How to reproduce it:

import pandas as pd
from datetime import date
from deltalake import DeltaTable, write_deltalake
import tempfile 

f  = tempfile.TemporaryDirectory().name
df = pd.DataFrame(data={"Date": [date(2023,1,1)] *10 + [date(2024,1,1)]*10, "B":range(20), "C":"constant"})
write_deltalake(f, df, mode='append', partition_by=["Date"])
dt = DeltaTable(f)
df_new = pd.DataFrame(data={"Date":[date(2023,1,1)], "B":[1], "C":"new value"})
dt.merge(source=df_new, predicate='t.Date = s.Date AND t.B = s.B', target_alias='t', source_alias='s').when_matched_update_all().execute()

DeltaError Traceback (most recent call last)
... \lib\site-packages\deltalake\table.py] in execute(self)
1776 Dict: metrics
1777 """
-> 1778 metrics = self.table._table.merge_execute(
1779 source=self.source,
1780 predicate=self.predicate,

DeltaError: Generic DeltaTable error: Unable to convert expression to string

More details:
The same code works in deltalake 0.16.1

@ibabeo ibabeo added the bug Something isn't working label Apr 15, 2024
@ion-elgreco
Copy link
Collaborator

This is already fixed here #2357

And will be available in the next release

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

No branches or pull requests

2 participants