WIP: [GA Merge-on-Read] refactoring :apps build and adding spark3.5/iceberg1.5 artifact

[cbb330]

Summary

problem: enabling Merge-on-Read for customer tables requires the ability to compact delete files. Compacting delete files is supported in spark3.1/iceberg1.2 but requires us to test it. We cannot test it, because creating delete files is not supported in spark3.1 (only compacting existing ones).

solution: we add spark3.5/iceberg1.5 compaction application, which extends the existing compaction test suite. in a follow up PR, we will add tests which create delete files and compacts the delete files under the latest versions.

Changes

• Client-facing API Changes
• Internal API Changes
• Bug Fixes
• New Features
• Performance Improvements
• Code Style
• Refactoring
• Documentation
• Tests

For all the boxes checked, please include additional details of the changes made in this pull request.

Testing Done

• Manually Tested on local docker setup. Please include commands ran, and their output.
• Added new tests for the changes made.
• Updated existing tests to reflect the changes made.
• No tests added or updated. Please explain why. If unsure, please feel free to ask for help.
• Some other form of testing like staging or soak time in production. Please explain.

For all the boxes checked, include a detailed description of the testing done for the changes made in this pull request.

Additional Information

• Breaking Changes
• Deprecations
• Large PR broken into smaller PRs, and PR plan linked in the description.

For all the boxes checked, include additional details of the changes made in this pull request.

[autumnust]

Is this ready for review?

[cbb330]

Is this ready for review?

@autumnust waiting on #221 to merge so i can rebase. this PR is forked from that.

if you want you can compare the commit delta here: https://github.com/linkedin/openhouse/pull/255/files/770a6d0497b9af7b1bedb876cbf639b71b3a9822..94c5100d5aa780a373ee5d0e14394ecdcec2f561