Does Fluvio have any interoperability with Kafka?

[frol]

I have found a few references about Kafka protocol in the source code, but I don't believe those few places would be enough to ensure compatibility.

P.S. I am excited to see Fluvio project (Rust and Wasm is the perfect match!)

[sehz]

Hi, Welcome to Fluvio Project.

We used to have strict compatibility with Kafka but decided to abandon it due to maintenance costs. In addition, it prevented us from adding exciting new capabilities such as SmartStream.

Currently, we still use a subset of Kafka protocol with a compatible storage format but no longer consider compatible with Kafka. We created separate repo where we still maintain Kafka compatible protocol Rust: https://github.com/infinyon/flv-kf-protocol. If there is demand, we may consider creating a Kafka connector, but only if there is sufficient demand.

We are looking forward to feedback and contribution to projects.

[sehz]

Hi, is this about Fluvio connecting to Kafka (in order to consume/produce) or providing Kafka compatible API on top of Fluvio in order to make it easy to migrate apps?

[gadLinux]

Is more about be bale to consume and produce to Kafka. We use it for streaming and there's a lot of work done processing Kafka Streams. In our side it would be great to be able test and compare Fluvio against Kafka Streams.
I don't know if others had other requirements. For us API compatibility is desiderable but not required.

[sehz]

Gotcha. We are looking to add add kafka connector to roadmap

[saminahbab]

Yeah, personally, I see this as quite important myself too.
Kafka is the dominant pub/sub tool in the industry. I feel its unreasonable not to have first class support for kafka.
Am I right in saying that there are no ways at the moment to, at least, use kafka as some kind of streaming source?

[sehz]

Thanks for the feedback. We actually had some users who had successfully able to connect Kafka with Fluvio. We are looking to add that as a supported connector.

[ajhunyady]

@frol, Kafka sink & source connectors are now ready for use: infinyon/fluvio-connectors#256.
The documentation is in the connector section: https://www.fluvio.io/connectors/

[frol]

@ajhunyady Thanks for letting me know! We have already redesigned the system, but I will keep fluvio in mind for my next projects!