LockClient requiered error for S3 backend in 0.15.1 python

[djouallah]

Environment

0.15.1

Binding:

Environment:
Python

---

Bug

it was working fine, but got this error when upgrading to 0.15.1

_internal.DeltaProtocolError: Atomic rename requires a LockClient for S3 backends. Either configure the LockClient, or set AWS_S3_ALLOW_UNSAFE_RENAME=true to opt out of support for concurrent writers.

currently I downgraded to 0.15

fwiw, I use this option with R2 " "copy_if_not_exists": "header: cf-copy-destination-if-none-match: *"

[dispanser]

@djouallah , have you tried setting AWS_S3_ALLOW_UNSAFE_RENAME=true as the error message suggests? This should still work, IIUC.

[djouallah]

but why, it used to works fine with Cloudflare R2 using "copy_if_not_exists": "header: cf-copy-destination-if-none-match: *"

[djouallah]

@roeap just to be very clear, does Cloudflare R2 support concurrent write without the use of an external lock tool ?

[ion-elgreco]

@djouallah no, also R2 doesn't support concurrent writes without locks since it uses S3 api.

I believe the copy-if-not-exists has been added back here: https://github.com/delta-io/delta-rs/pull/2083/files

A fix release will be pushed soon, I hope today