Support custom table properties via TableAlterer and write/merge

[echai58]

Description

Delta Lake via spark supports storing custom metadata as a table property via both CREATE and ALTER commands. Currently, delta-rs only supports custom configuration on table creation. TableAlterer exists, but it only supports add_constrant, not arbitrary custom metadata. Because add_constraint exists, I imagine there exists the framework already to support this and it would not be a difficult addition.

Also, merge should allow for fields like the newly added custom_metadata in write_deltalake, as people can use merge as an upsert-style write.

Use Case
There are many scenarios where we'd want to support table-level metadata that can be accessed via DeltaTable.metadata(), and they should be able to be altered in the future, not just on table creation.

Ideally, this would also allow for the configuration field to be used in write_deltalake, so it could be done as one atomic transaction, when applicable, rather than having to do a write, then an alter.

[ion-elgreco]

Yes this is on the backlog, we are tracking it here: #1909

I will take a look at exposing custom_metadata in merge, can you create a separate issue for that? 😄

[echai58]

Yes this is on the backlog, we are tracking it here: #1909

I will take a look at exposing custom_metadata in merge, can you create a separate issue for that? 😄

@ion-elgreco Great, thanks! Here's a new issue for exposing custom_metadata in merge: #2031