Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

When querying metadata for a single topic, don't always autocreate if nonexistant. #1889

Closed
llamahunter opened this issue Feb 24, 2021 · 2 comments
Labels
stale Issues and pull requests without any recent activity

Comments

@llamahunter
Copy link

llamahunter commented Feb 24, 2021

Versions

Please specify real version numbers or git SHAs, not just "Latest" since that changes fairly regularly.

Sarama Kafka Go
v1.26.4 2.4.0 1.12
Configuration

N/A

Logs

N/A

Problem Description

When requesting metadata on a single topic, the sarama library will request the broker to create the topic if it does not exist. This in not always desirable, since the topic might get made with some weird broker defined configuration. Would rather that the topic were not created, and that it was just absent from the locally cached cluster metadata.

Otherwise, this forces people to use the non topic specific metadata request, which refreshes the entire cluster's metadata. If there are thousands of topics, this can be quite a performance hit.

There should be a way to request metadata on a single topic while still suppressing auto topic creation.

@llamahunter llamahunter changed the title When querying metadata for a single topic, don't autocreate if nonexistant. When querying metadata for a single topic, don't always autocreate if nonexistant. Feb 24, 2021
@github-actions
Copy link

Thank you for taking the time to raise this issue. However, it has not had any activity on it in the past 90 days and will be closed in 30 days if no updates occur.
Please check if the main branch has already resolved the issue since it was raised. If you believe the issue is still valid and you would like input from the maintainers then please comment to ask for it to be reviewed.

@github-actions github-actions bot added the stale Issues and pull requests without any recent activity label Aug 25, 2023
@dnwe
Copy link
Collaborator

dnwe commented Aug 29, 2023

This is supported since v1.30 and #1983 by setting the AllowAutoTopicCreation bool in config:

sarama/config.go

Lines 164 to 167 in 6162a43

// Whether to allow auto-create topics in metadata refresh. If set to true,
// the broker may auto-create topics that we requested which do not already exist,
// if it is configured to do so (`auto.create.topics.enable` is true). Defaults to true.
AllowAutoTopicCreation bool

@dnwe dnwe closed this as completed Aug 29, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
stale Issues and pull requests without any recent activity
Projects
None yet
Development

No branches or pull requests

2 participants