-
Notifications
You must be signed in to change notification settings - Fork 3.6k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
PIP 110: Topic metadata #12629
Comments
LGTM +1 |
We should prevent the user to set properties on individual partitions in a partitioned topic (because partitions are like topic for many aspects) Please also explain the permissions needed to update such properties (and add authz coverage in the test cases ) |
@Technoboy- did you see my comment? |
Ok, I will update this in the pr. |
codelipenghui
pushed a commit
that referenced
this issue
Jan 25, 2022
…#12818) Fixes #12629 ## Motivation The original discussion mail : https://lists.apache.org/thread/m9dkhq1fs6stsdwh78h84fsl5hs5v67f Introduce the ability to store metadata about topics. This would be very useful as with metadata you could add labels and other pieces of information that would allow defining the purpose of a topic, custom application-level properties. This feature will allow application-level diagnostic tools and maintenance tools to not need external databases to store such metadata. Imagine that we could add a simple key value map (String keys and String values) to the topic. These metadata could be set during topic creation and also updated.
5 tasks
2 tasks
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Motivation
The original discussion mail :
https://lists.apache.org/thread/m9dkhq1fs6stsdwh78h84fsl5hs5v67f
Introduce the ability to store metadata about topics.
This would be very useful as with metadata you could add labels and other
pieces of information that would allow defining the purpose of a topic,
custom application-level properties.
This feature will allow application-level diagnostic tools and maintenance
tools to not need external databases to store such metadata.
Imagine that we could add a simple key value map (String keys and String
values) to the topic.
These metadata could be set during topic creation and also updated.
API Changes
We need to add a map type field called
metadata
to the below methods signature:Implementation
For PartitionedTopic
We will put these metadata to the PartitionedTopicMetadata, serialized to store in configurationMetadataStore, like partitioned-topic partitions.
For NonPartitionedTopic
We will store the metadata to ML(ManagedLedger) which already supports storing custom properties.
Compatibility
The proposal will not introduce any compatibility issues.
Tests Plan
Unit tests & integration tests
The text was updated successfully, but these errors were encountered: