Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[SPARK-50285] Metrics for commits to StagedTable instances #48830

Open
wants to merge 6 commits into
base: master
Choose a base branch
from

Conversation

olaky
Copy link
Contributor

@olaky olaky commented Nov 12, 2024

What changes were proposed in this pull request?

Commands that commit using the StagedTable interface do not have any metrics at the moment because the interface does not support retrieving metrics after a commit. This PR

  • Adds a new interface StagedTableWithCommitMetrics that allows to retrieve metrics
  • Add a method StagingTableCatalog to indicate that the catalog supports metrics
  • Support metric retrieval in the commands that use the StagedTable interface

Why are the changes needed?

Many create table commands currently return no metrics at all

Does this PR introduce any user-facing change?

No

How was this patch tested?

New tests with a test catalog for the affected commands

Was this patch authored or co-authored using generative AI tooling?

No

@github-actions github-actions bot added the SQL label Nov 12, 2024
@olaky
Copy link
Contributor Author

olaky commented Nov 14, 2024

cc @cloud-fan Can you take a look? I saw you originally implemented the Staged Table interface

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant