feat: Configurable loading of event bus producer

[timmc-edx]

This defines an API for IDAs to use when producing events that allows the
choice of Event Bus implementation to be configured rather than written in
code. Implementations can document the appropriate setting for deployers
to use.

Caching the producer instance means that we automatically get a
long-lived instance without the implementation itself having to
perform caching itself.

This is part of #87

Merge checklist:

• Version bumped
• Changelog record added
• Documentation updated (not only docstrings)
• Commits are squashed

[robrap]

Generally right direction. Just digging into details.

[robrap]

Just confirming that this replaces the previously documented send_to_event_bus, since this would probably result in code like event_bus.send(...)?
Related, do we like using the term send rather than produce, to provide overlap with the signals, since this takes a signal? Or would something like produce_from_signal be more clear?

[timmc-edx]

No strong opinion! But yeah, it seemed redundant to use the existing name.

[mphilbrick211]

Hi @timmc-edx! Just checking to see if you are planning to proceed with this PR?

[timmc-edx]

Yep! This work was deprioritized and has bit-rotted a bit, but I'll be revisiting it.

[robrap]

@timmc-edx: We have #87 which mentions wip PRs. You are welcome to close the PRs and verify the issue would help someone find them when and if the work is picked up. That said, do whatever feels right.

[timmc-edx]

I pushed up something of a rewrite of the loader code. It may be that some of it should get pushed into an internal module, but I think this will do what we need and allow us to remove Kafka references from IDAs.

[timmc-edx]

This is ready for review again, and is accompanied by openedx/event-bus-kafka#14

I'm a little unsure about what the consumer API should look like, and it might be better to move that to a new PR later. It might be better to just focus on the producer part right now.

[robrap]

@timmc-edx: I think we decided we would close/defer this? We could create a separate issue, or just use this PR as the issue, if you are ok with long-standing PRs. We should add backlog label and move to Future milestone, if you agree with status. Thanks!

[timmc-edx]

This one is just for loading of the implementation -- and for now, scoped down to just the producing side. I'm moving the consumer side into a separate branch for later consideration.

[rgraber]

I think this is worth a small how-to with examples of the kinds of things you might put in the EVENT_BUS_PRODUCER setting. The how-to could basically copy the setting description but I think a how-to would be a more accessible place for it.

[timmc-edx]

@rgraber Hmm. I feel like it wouldn't be that helpful to give an example of the setting without backing code. The producer setting already gives an example of some.module.path.EventBusImplementation but of course there's no EventBusImplementation to look at. Perhaps the documentation could instead link to event-bus-kafka's documentation as an example? It's all a bit chicken-and-egg, but I could come back and make a new PR here once https://github.com/openedx/event-bus-kafka/pull/14/files#diff-7b3ed02bc73dc06b7db906cf97aa91dec2b2eb21f2d92bc5caa761df5bbc168f is merged.

[rgraber]

@rgraber Hmm. I feel like it wouldn't be that helpful to give an example of the setting without backing code. The producer setting already gives an example of some.module.path.EventBusImplementation but of course there's no EventBusImplementation to look at. Perhaps the documentation could instead link to event-bus-kafka's documentation as an example? It's all a bit chicken-and-egg, but I could come back and make a new PR here once https://github.com/openedx/event-bus-kafka/pull/14/files#diff-7b3ed02bc73dc06b7db906cf97aa91dec2b2eb21f2d92bc5caa761df5bbc168f is merged.

I think coming back once that PR is merged makes sense

[rgraber]

LGTM

[timmc-edx]

OK, I'll add a ticky box for that on the issue.