Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Issue 11496][C++] Allow partitioned producers to start lazily #11570

Merged
merged 6 commits into from
Aug 16, 2021

Commits on Aug 10, 2021

  1. Allow partitioned producers to start lazily

    Avoids performing lookups and producer registration on all partitions of a partitioned
    topic when using SinglePartition routing without keyed messages
    Jack Vanlightly authored and Vanlightly committed Aug 10, 2021
    Configuration menu
    Copy the full SHA
    0fa7215 View commit details
    Browse the repository at this point in the history
  2. Lazy producers start send timeout timer on first message

    Lazy producers connect on demand on their first message.
    The send timeout timer must be started on the first
    message as there is no guarantee the connection will
    complete
    Vanlightly committed Aug 10, 2021
    Configuration menu
    Copy the full SHA
    1f54174 View commit details
    Browse the repository at this point in the history

Commits on Aug 11, 2021

  1. Configuration menu
    Copy the full SHA
    adf13f6 View commit details
    Browse the repository at this point in the history
  2. Fixed additional race condition in lazy producers

    Ensure that the state is checked always in
    PartitionedProducer::sendAsync to avoid buffering of
    messages after closeAsync called.
    
    Removed sequential locking in ProducerImpl::
    handleCreateProducer that allowed for state to go
    back to Ready after closeAsync called.
    Vanlightly committed Aug 11, 2021
    Configuration menu
    Copy the full SHA
    eb79fb0 View commit details
    Browse the repository at this point in the history

Commits on Aug 13, 2021

  1. Ensure one producer starts when lazy

    In order to ensure than authz errors are returned
    during producer creation, start one producer
    eagerly when using lazy partitioned producers.
    When UseSinglePartition mode is used, the
    eagerly created producer will be the only producer
    created (when non-keyed messages are sent).
    Vanlightly committed Aug 13, 2021
    Configuration menu
    Copy the full SHA
    87224b1 View commit details
    Browse the repository at this point in the history
  2. Fail pending messages if producer creation fails

    In the case that an internal producer is created
    lazily and it fails, then fail any pending
    requests immediately.
    Vanlightly committed Aug 13, 2021
    Configuration menu
    Copy the full SHA
    59f989a View commit details
    Browse the repository at this point in the history