[Core feature] Kafka Authentication

[pingsutw]

Motivation: Why do you think this is important?

In flyteadmin, we send a cloud event to Kafka. However, we don't have any authentication for Kafka right now. we should expose auth config, use Sarama api to read the config, and send the request to Kafka.

Goal: What should the final outcome look like, ideally?

CloudEvent Config

cloudevents:
    enable: true
    kafka:
      brokers: <kafka_ip>
      version:  1.2.1
      security_ protocol: SASL_SSL
      sasl_config: <config for SASL>

Describe alternatives you've considered

NA

Propose: Link/Inline OR Additional context

Slack Thread:

• https://flyte-org.slack.com/archives/CP2HDHKE1/p1667307468058299
• https://flyte-org.slack.com/archives/CP2HDHKE1/p1667412056307629
• https://flyte-org.slack.com/archives/CP2HDHKE1/p1670577684352189
• https://flyte-org.slack.com/archives/CP2HDHKE1/p1667412056307629

Authentication Methods Overview

Are you sure this issue hasn't been raised already?

• Yes

Have you read the Code of Conduct?

• Yes

[pingsutw]

cc @eapolinario

[github-actions]

Hello 👋, This issue has been inactive for over 9 months. To help maintain a clean and focused backlog, we'll be marking this issue as stale and will close the issue if we detect no activity in the next 7 days. Thank you for your contribution and understanding! 🙏

[github-actions]

Hello 👋, This issue has been inactive for over 9 months and hasn't received any updates since it was marked as stale. We'll be closing this issue for now, but if you believe this issue is still relevant, please feel free to reopen it. Thank you for your contribution and understanding! 🙏