Samara-cluster dependency seems to have an issue

[bellemare]

I'm getting a samara-cluster issue when I try to get and when I try to run the sample code. Both go and dep are freshly installed, OSX 10.13.6.

➜  gokatest go get -u github.com/lovoo/goka       
# github.com/bsm/sarama-cluster
../github.com/bsm/sarama-cluster/consumer.go:452:59: c.client.config.Config.Consumer.Offsets.CommitInterval undefined (type struct { AutoCommit struct { Enable bool; Interval time.Duration }; Initial int64; Retention time.Duration; Retry struct { Max int } } has no field or method CommitInterval)

[rehabaam]

It is related to bsm/sarama-cluster project code.
I had the same issue after installing it. I fixed it by changing that line
ticker := time.NewTicker(c.client.config.Consumer.Offsets.CommitInterval)
to be
ticker := time.NewTicker(c.client.config.Consumer.Offsets.AutoCommit.Interval)

[jkt628]

bsm/sarama-cluster is deprecated and who knows if this PR with exactly that fix will be merged. which leads to a bigger issue:

is anyone updating goka to Shopify/sarama >= v1.19.0?

[jkt628]

in the meantime add this to your go.mod:

replace github.com/bsm/sarama-cluster => github.com/CHneger/sarama-cluster v0.0.0-20200108115656-95c44d29f408

[db7]

@jkt628 I am pretty sure @frairon started working on removing sarama-cluster from goka/kafka subpackage. I don't know the status though.

[frairon]

@db7 is right, we're working on removing sarama-cluster and replacing goka's rebalance-dispatch-loop by Sarama's ConsumerGroup. Therefore we're refactoring most of goka's internals, not only the goka/kafka subpackage. We'll announce the release once it's in a more stable state so goka users can test it from the branch to avoid introducing too many breaking changes.
Many tests have to be rewritten and that's where we're currently at. As a rough estimate I'd say mid-february should be realistic for something that can be used for beta testing.

[erichatescoding]

bsm/sarama-cluster is deprecated and who knows if this PR with exactly that fix will be merged. which leads to a bigger issue:

is anyone updating goka to Shopify/sarama >= v1.19.0?

I asked this question in another issue, but just saw your comment, and want to ask again the same question - if I want to upgrade kafka to 2.2.1, how am I gonna to upgrade goka (sarama under the hood) to be compatible with?

[frairon]

Well @erichatescoding, it looks like there was a fix for the sarama-cluster/sarama incompatibilities of newest versions on Jan 29th. So without testing I would assume you can now use the newest sarama version again which should enable you to upgrade to Kafka 2.2.1. However, we haven't used Kafka newer than 0.10.2.1 yet so I don't know what might be issues there.

Also we're currently moving goka to use sarama's ConsumerGroup to get rid of sarama-cluster. If you mean that by "upgrade goka ... to be compatible with" then we're on it and all you need to do is wait :)
Cheers!

[frairon]

As far as I know, sarama-cluster works with newest sarama version so I'll close this issue, feel free to reopen if you think we need more discussion around that point here.

[erichatescoding]

Appreciate that, @frairon.
I did tested the the sarama version > 1.23.0 will support Kafka 2.2.1. Our application currently using goka, with kafka 0.11.0. So base on your input here, if we are about to upgrade to kafka 2.2.1 now, then you would suggest to use sarama rather continue using goka since you guys are still working on?