Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Could anyBroker() be made more relient? #2488

Closed
vinhtbcasoft opened this issue Jul 10, 2023 · 3 comments
Closed

Could anyBroker() be made more relient? #2488

vinhtbcasoft opened this issue Jul 10, 2023 · 3 comments
Labels
stale Issues and pull requests without any recent activity

Comments

@vinhtbcasoft
Copy link

Versions
Sarama Kafka Go
v1.38.1 2.3.0 1.18
Configuration

What configuration values are you using for Sarama and Kafka?

config := sarama.NewConfig()
config.Version = version
config.Consumer.Offsets.Initial = sarama.OffsetOldest
ConsumerGrpClient, err =  sarama.NewConsumerGroup(core.KafkaInitialBrokers, consumerGrpId, config)
Logs
[sarama] 2023/06/18 18:12:34 tryRefreshMetadata(): client/metadata fetching metadata for [__consumer_offsets] from broker 10.8.13.1:9092
[sarama] 2023/06/18 18:12:34 tryRefreshMetadata(): successfully fetched metadata for topics.
[sarama] 2023/06/18 18:12:34 Unexpected topic-level metadata error: kafka server: Replication-factor is invalid.
[sarama] 2023/06/18 18:12:34 ERROR:  tryRefreshMetadata(): successfully fetched metadata but failed on updateMetadata, err: kafka server: Replication-factor is invalid.
[sarama] 2023/06/18 18:12:34 ERROR:  client/coordinator the __consumer_offsets topic is not initialized completely yet. Waiting 2 seconds...
[sarama] 2023/06/18 18:12:34 tryRefreshMetadata(): successfully fetched metadata for topics.
[sarama] 2023/06/18 18:12:34 Unexpected topic-level metadata error: kafka server: Replication-factor is invalid.
[sarama] 2023/06/18 18:12:34 ERROR:  tryRefreshMetadata(): successfully fetched metadata but failed on updateMetadata, err: kafka server: Replication-factor is invalid.
[sarama] 2023/06/18 18:12:34 ERROR:  client/coordinator the __consumer_offsets topic is not initialized completely yet. Waiting 2 seconds...

Problem Description

Within client.go::findCoordinator() and tryRefreshMetadata(), it asks anyBroker() for the endpoint of the request. anyBroker() will return the same endpoint (first one of the client.seedBrokers list). This is an issue if the chosen endpoint is misconfigured (it will return a error response). In this use case, the broker is not evicted from the client.seedBrokers list.

Could anyBroker() be more resilient with the following enhancement?

@dnwe
Copy link
Collaborator

dnwe commented Jul 28, 2023

Although the code gives the appearance of always choosing the first seed broker, there are a few code paths that randomise the seed brokers slice via client.randomizeSeedBrokers such that this should be a different order at various points in time

@github-actions
Copy link

Thank you for taking the time to raise this issue. However, it has not had any activity on it in the past 90 days and will be closed in 30 days if no updates occur.
Please check if the main branch has already resolved the issue since it was raised. If you believe the issue is still valid and you would like input from the maintainers then please comment to ask for it to be reviewed.

@github-actions github-actions bot added the stale Issues and pull requests without any recent activity label Oct 30, 2023
@dnwe
Copy link
Collaborator

dnwe commented Nov 2, 2023

I believe that the changes made under #2645 have improved this area so I'm going to go ahead and close this issue as fixed by that. Feel free to re-open if you have further concerns

@dnwe dnwe closed this as completed Nov 2, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
stale Issues and pull requests without any recent activity
Projects
None yet
Development

No branches or pull requests

2 participants