Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[SPARK-50311][PYTHON] Support (add|remove|get|clear)Tag(s) APIs for PySpark #48843

Open
wants to merge 5 commits into
base: master
Choose a base branch
from

Conversation

itholic
Copy link
Contributor

@itholic itholic commented Nov 14, 2024

What changes were proposed in this pull request?

This PR proposes to support (add|remove|get|clear)Tag(s) for PySpark

Why are the changes needed?

For improving the compatibility between Spark Connect and Spark Classic. These tag-related APIs are currently only supported from Spark Connect.

Does this PR introduce any user-facing change?

New APIs are added

  • addTag
  • removeTag
  • getTags
  • clearTags

How was this patch tested?

Added UTs

Was this patch authored or co-authored using generative AI tooling?

No

@github-actions github-actions bot added the DOCS label Nov 14, 2024
@itholic itholic changed the title [SPARK-50311][PYTHON] Support (add|remove|get|clear)Tag(s) for PySpark [SPARK-50311][PYTHON] Support (add|remove|get|clear)Tag(s) APIs for PySpark Nov 14, 2024
from pyspark.testing.connectutils import ReusedConnectTestCase


class JobCancellationParityTests(JobCancellationTestsMixin, ReusedConnectTestCase):
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Moved this suite from pyspark.sql.tests.connect.test_session and integrated with JobCancellationTests

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant