Kerberos Authentication for Kafka

[AndreAga]

Hi guys,
I saw Beats Library doesn’t support Kerberos authentication for kafka output, but Logstash kafka input does. Any plan to add this kind of Auth?

Thanks.

[mellowonpsx]

I agree with @AndreAga, also Logstash's Kafka output plugin supports Kerberos SASL.
Kerberos Auth is a must-have feature for Bests Library.

[gmoskovicz]

@urso any news regarding this?

[urso]

@gmoskovicz Sorry, no updates on this ticket.

[miko-code]

+1

[mmirabedini]

+1

[giezer]

+1

[ioah86]

+1

[dounine]

+1

[mayank-mahajan-guavus]

@jsoriano Is there a plan to add Kerberos Support in beats?

[nathanrstacey]

+1

[smaley07]

+1

[vickhello]

+1

[Lswx2017]

+1

[Yggdrassil80]

+1

[mostlyjason]

One of our customers offered this response on why Kerberos is better than SSL:

In the case of the Confluent article [showing SSL auth], they are using a very loose term for authentication, in saying that it is performing mutual authentication of the certificates and only validates that the certificate is trusted by way of the CA certificates. In other words, this is machine authentication, and it provides no context to the user on that machine. In our security context, SSL authentication is not sufficient.

[moulisea]

1. when I give output.kafka.version: "2.5.0", I get below error: ERROR instance/beat.go:958 Exiting: error initializing publisher: unknown/unsupported kafka vesion '2.5.0' accessing 'output.kafka.version' (source:'filebeat.yml') Exiting: error initializing publisher: unknown/unsupported kafka vesion '2.5.0' accessing 'output.kafka.version' (source:'filebeat.yml') 2. Also I see below Alert in Filebeat. Known issue in version 7.8.0 The Kafka output fails to connect when using multiple TLS brokers. We advise not to upgrade to Filebeat 7.8.0 if you’re using the Kafka output in this configuration. Do you recommend to use 7.8.0 in Prod with multiple broker( we have 3 broker). I started using 7.8 mainly to support SASL_SSL with GSSAPI mechanism. Thanks, Mouli On Wed, Aug 19, 2020 at 11:05 PM chandramouli srinivasan < [email protected]> wrote:

…

I tried logging: debug in YML file but not helpful to find what is the issue. If you are aware of any debugging mechanism, let me know. Thanks. Thanks, Mouli On Wed, Aug 19, 2020 at 12:23 PM chandramouli srinivasan < ***@***.***> wrote: > > I did that and I see below two errors. In the initial logs, it says it > established kafka connection. But later I see below errors. > > DEBUG [harvester] log/log.go:107 End of file reached: > E:\Logs\file.log; Backoff now. > DEBUG [kafka] kafka/client.go:277 finished kafka batch > DEBUG [kafka] kafka/client.go:291 Kafka publish failed with: > kafka: client has run out of available brokers to talk to (Is your cluster > reachable?) > > Kafka publish failed with: circuit breaker is open > > > Thanks, > Mouli > > On Wed, Aug 19, 2020 at 10:31 AM Noémi Ványi ***@***.***> > wrote: > >> @moulisea <https://github.com/moulisea> ATM kerberos.username is >> commented out. You need to remove # from the beginning of the line. >> >> — >> You are receiving this because you were mentioned. >> Reply to this email directly, view it on GitHub >> <#5413 (comment)>, >> or unsubscribe >> <https://github.com/notifications/unsubscribe-auth/AD2UZLQIPOWRM7G24RWJDHDSBPV5TANCNFSM4EAA7QHQ> >> . >> >

[kvch]

Do you mind opening a Discuss question for these problems? This issue is about tracking Kerberos authentication for Kafka, not arbitrary Kafka issues.

[moulisea]

Thanks for response. I am not clear how to open a Discuss question for these problems. can you provide some link, so I can start discussion on this. Yes, the issue is not able to connect to Kafka kerberos from Filebeat. so I am exploring logstash now. But if we could make this work in Filebeat, that would be really great. Appreciate your help. Thanks. Thanks, Mouli

…

Sent from my iPhone On Aug 24, 2020, at 12:01 PM, Noémi Ványi <[email protected]> wrote:  Do you mind opening a Discuss question for these problems? This issue is about tracking Kerberos authentication for Kafka, not arbitrary Kafka issues. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#5413 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AD2UZLVPXZSWN2KHOQOSMLDSCKMINANCNFSM4EAA7QHQ> .

[kvch]

I meant opening a question here: https://discuss.elastic.co/c/elastic-stack/beats/28
I will find someone to help you there. Thanks in advance.

[moulisea]

Thanks. I have created one. link below. https://discuss.elastic.co/t/filebeat-connect-with-kafka-kerberos-sasl-ssl-not-working/246160 Thanks, Mouli On Wed, Aug 19, 2020 at 12:23 PM chandramouli srinivasan < [email protected]> wrote:

…

I did that and I see below two errors. In the initial logs, it says it established kafka connection. But later I see below errors. DEBUG [harvester] log/log.go:107 End of file reached: E:\Logs\file.log; Backoff now. DEBUG [kafka] kafka/client.go:277 finished kafka batch DEBUG [kafka] kafka/client.go:291 Kafka publish failed with: kafka: client has run out of available brokers to talk to (Is your cluster reachable?) Kafka publish failed with: circuit breaker is open Thanks, Mouli On Wed, Aug 19, 2020 at 10:31 AM Noémi Ványi ***@***.***> wrote: > @moulisea <https://github.com/moulisea> ATM kerberos.username is > commented out. You need to remove # from the beginning of the line. > > — > You are receiving this because you were mentioned. > Reply to this email directly, view it on GitHub > <#5413 (comment)>, > or unsubscribe > <https://github.com/notifications/unsubscribe-auth/AD2UZLQIPOWRM7G24RWJDHDSBPV5TANCNFSM4EAA7QHQ> > . >

[Ghaithjemai]

same problem... did you find the solution?

[atalukdar]

I am having same issue with filebeat-7.11.1

Any solution on sasl_ssl?

Connection to kafka(xxxxx:9093) established
[kafka] kafka/client.go:371 finished kafka batch
[kafka] kafka/client.go:385 Kafka publish failed with: kafka: client has run out of available brokers to talk to (Is your cluster reachable?)

[elafontaine]

Same problem. Searching for a link to a solution.

[urso]

Can you reproduce the issue with 7.12?

[elafontaine]

We were on 7.10 from the standard rpm repository of Red Hat I believe. I'll ask to try on filebeat 7.12

I noticed that it's mostly a Kerberos issue, there is just no log about it. I noticed we're forced to put the service_name parameter, even though the authentification is successful without it (and with it, it actually fails... I'm trying to understand why on my side.

[elafontaine]

Ok, I confirm that with 7.12, we're able to make it work with password type of authentification with kerberos (and clear text password in the configuration). However, as soon as we switch this to auth_type: keytab and pass the keytab option, it stops working. I have confirmed the keytab authentification with kinit and with the same service that filebeat should be trying to use as per the config.

We just tested with 7.13, the same; clear text password works, but keytab doesn't.

[theforcebemay]

@elafontaine
Could you share your filebeat.yml?

[JunTaoYuan80]

Same problem with 7.15. Searching for a link to a solution.

2021-10-09T17:10:45.739+0800	INFO	[file_watcher]	filestream/fswatch.go:137	Start next scan
2021-10-09T17:10:50.838+0800	ERROR	[kafka]	kafka/client.go:317	Kafka (topic=common_log_topic): kafka: client has run out of available brokers to talk to (Is your cluster reachable?)```

[jaychouuu]

@JunTaoYuan80 jun Have you solved it now? I have the same problem

[JunTaoYuan80]

@JunTaoYuan80 jun Have you solved it now? I have the same problem

no, but i change filebeat to logstash, it's ok.

[elasticmachine]

Pinging @elastic/elastic-agent-data-plane (Team:Elastic-Agent-Data-Plane)

[jlind23]

@rdner @faec does this issue rings a bell on your end?

[rdner]

Our latest documentation claims Kerberos is supported:

To use GSSAPI mechanism to authenticate with Kerberos, you must leave this field empty, and use the kerberos options.

https://www.elastic.co/guide/en/beats/filebeat/current/kafka-output.html#_sasl_mechanism

However, we still don't have an integration test to track that it's working #29430

[jlind23]

Closing this as this is supported and keeping #29430 open in order to add safety guards.

[fbaligand]

Hi,

If this issue is “done”, could you precise what will be beats release that includes this feature?

[jlind23]

@fbaligand as of 7.17 kerberos feature was available in beta mode as stated here: https://www.elastic.co/guide/en/beats/filebeat/7.17/kafka-output.html#_sasl_mechanism

[fbaligand]

Thanks for the information.