Where to establish Bunny connections in Ruby On Rails using the Puma web server?

[kingpalethe]

I am using Ruby On Rails 7 with the Puma web server.

Puma is a multi-threaded web server.

I send RabbitMQ messages from various rails "controllers".

I know that I am not supposed to make a new connection or channel with each controller action. But at the same time, I know I am not supposed to share these between threads.

The Bunny docs seem to have recommendations for Unicorn, which was the dominant web server used before 2017 with Rails:

http://rubybunny.info/articles/connecting.html#using_bunny_with_unicorn

I can't find any best practice recommendation for Puma.

Currently, in my puma.rb initialization file, I am doing something like this, copying the recommendations for Unicorn:

on_worker_boot do
    $rabbitmq_connection = Bunny.new
    $rabbitmq_channel= $rabbitmq_connection_fw.create_channel
end  

Puma does not seem to have a concept like Unicorn's after_fork, but I have noticed that it's recommended to initialized redis connections in this on_worker_boot area, so that's why I am using it for RabbitMQ.

I have been using it like this for several months. In each rails controller action, I use the $rabbitmq_channel, like this:

queue = $rabbitmq_channel.queue('my-queue-name', durable: true)
queue.publish(...)

This has generally worked, but... recently, with higher traffic, however, I have had some worrying errors where certain messages were sent to entire the WRONG queue.

Here is some possibly related reference material:
#515
#494
https://wework.github.io/ruby/rails/bunny/rabbitmq/threads/concurrency/puma/errors/2015/11/12/bunny-threads/
https://www.cloudamqp.com/blog/part4-rabbitmq-13-common-errors.html

[michaelklishin]

Initializing connections that way is fine. However, with Puma your resources will be accessed concurrently, so access to all channel operations with Bunny have to be synchronized.

[michaelklishin]

The approach outlined in this post seems to be a good middle ground for at least some apps. Obviously there cannot be a single recommendation for all concurrency hazards.

[kingpalethe]

Thanks for this. The blog post linked is from 2015, and for reasons I don't understand, this strategy no longer works. There doesn't seem to be a way to use ConnectionPool with Bunny. Someone must have discovered the best practice for this, as likely 90% of production Rails apps use Puma now, instead of Unicorn.

[pgouv]

What is the issue with connection pool?

[michaelklishin]

If you don't end up sharing channels between threads, I don't see any reason why ConnectionPool would be "incompatible" with Bunny.

Channels must not be shared for concurrent access (for publishers and most synchronous operations such as queue.declare). That's a protocol design decision, it does not matter what client is used.

[pgouv]

ConnectionPool is a good gem and Rails 7 has a dependency to it.
I am not sure how to handle channel errors like precondition failed when using it.