"Google::Cloud::Vision.image_annotator" crashes Puma app server in Rails controller, but same code runs fine as a standalone ruby script

[sampenguin]

I've looked high and low and tried overriding the Google::Cloud::Vision module to inject debug info, but no matter what I do the app server crashes as soon as I try to instantiate a vision client with the line Google::Cloud::Vision.image_annotator.

I am at a loss as to why this runs fine, connects to the cloud API and outputs as expected in a standalone script, but the same code fails with no exceptions, no logging, nothing but an abrupt server crash in a Rails controller.

For auth, I am using a straightforward service key in a .json file just to get a basic API call to analyze an image successfully.

I've also tried enabling GCV logging per the instructions at https://cloud.google.com/ruby/docs/reference/google-cloud-vision-v1/latest and that did nothing that I could see. If anyone has any suggestions or ideas to workaround or get at further ways to debug this, I'd love to hear them as I'm at a hard block on this now. What am I missing?

Environment details

• OS: Windows 10
• Ruby version: 3.3.3-p89
• Rails 7.1.3
• Puma 6.4.2
• gem 'google-cloud-vision'

Steps to reproduce

1. Processing the route to DebugController#moderate (see below) crashes 100% of the time on vision_client = Google::Cloud::Vision.image_annotator.

Code examples

Standalone Ruby script (running this from console via "ruby debug_vision.rb" works and outputs two results as expected):

# debug_vision.rb
require "google/cloud/vision"

ENV["GOOGLE_APPLICATION_CREDENTIALS"] ||= ("D:/path_to/config/credentials/google_cloud_vision.json").to_s

Google::Cloud::Vision.configure do |config|
  config.credentials = ENV["GOOGLE_APPLICATION_CREDENTIALS"]
  puts "=== credentials: #{config.credentials}"
end

image_path = "D:/path_to/file.jpeg"

vision = Google::Cloud::Vision.image_annotator
puts "=== vision client initialized ==="


# Open the image file as an IO object
File.open(image_path, "rb") do |image_file|
  response = vision.safe_search_detection image: image_file
  safe_search = response.responses.first.safe_search_annotation

  puts "Adult: #{safe_search.adult}"
  puts "Violence: #{safe_search.violence}"
end

When running via Rails controller, the credentials are set via an initializer when the server starts:

# config/initializers/google_cloud_vision.rb
require "google/cloud/vision"
# Set the environment variable for Google Cloud credentials
ENV["GOOGLE_APPLICATION_CREDENTIALS"] ||= Rails.root.join("config/credentials/google_cloud_vision.json").to_s

Google::Cloud::Vision.configure do |config|
  config.credentials = ENV["GOOGLE_APPLICATION_CREDENTIALS"]
end

Rails controller code (this method verifies credentials are correct and that a file is uploaded, but creating the image_annotator client object crashes the server):

# controllers/debug_controller.rb

class DebugController < ApplicationController
# Attempted all kinds of variations of including google/cloud/vision, no differences
# require "google/cloud/vision"
# include Google::Cloud::Vision
...
def moderate
    Rails.logger.debug "========== DebugController#moderate..."

    uploaded_file = params[:image]

    unless uploaded_file.present?
      flash[:alert] = "No file was uploaded!"
      render :image
    end

    image_path = uploaded_file.tempfile.path
    Rails.logger.debug "=== image_path: #{image_path&.inspect}"

    begin
      # Attempted to do config within controller, no difference
      # Google::Cloud::Vision.configure do |config|
      #   config.credentials = Rails.root.join("config/credentials/google_cloud_vision.json").to_s
      # end

      # Check if the environment variable is set
      credentials_env = ENV["GOOGLE_APPLICATION_CREDENTIALS"]
      unless credentials_env
        raise "Environment variable GOOGLE_APPLICATION_CREDENTIALS not set"
      end
      Rails.logger.debug "========== GOOGLE_APPLICATION_CREDENTIALS: #{credentials_env} =========="

      ####################################################
      # THE NEXT LINE CRASHES THE APP SERVER EVERY TIME #########
      vision_client = Google::Cloud::Vision.image_annotator
      
      # vision_client = VisionDebug.image_annotator # Attempt to override module, see more details
      Rails.logger.debug "=== vision client initialized ==="
      Rails.logger.debug "#{vision_client&.inspect}"

      response = vision_client.safe_search_detection image: image_path
      Rails.logger.debug "=== response: #{response&.inspect}"

      safe_search = response.responses.first.safe_search_annotation
      Rails.logger.debug "=== safe_search: #{safe_search&.inspect}"

      render json: {
        adult: safe_search.adult,
        violence: safe_search.violence,
      }
    rescue => e
      # This code block is never reached, the server just crashes
      error_message = "Google Cloud Vision API call failed: #{e.message}"
      Rails.logger.error error_message
      flash[:alert] = error_message
      render :image
    end
  end

Backtrace

None available, just a hard stop crash of the app server.

More Details

I did somewhat successfully override the Google Cloud Vision module to inject some more debugging, which further revealed it's crashing on this line in def self.image_annotator:
result = service_module.const_get(:Client).new(&block)

This is the full override code (just copied the original source and added a debug line in between each op):

module VisionDebug
  include Google::Cloud::Vision

  def self.image_annotator version: :v1, transport: :grpc, &block
    Rails.logger.debug "=== image_annotator, version: #{version}, transport: #{transport}"
    require "google/cloud/vision/#{version.to_s.downcase}"
    Rails.logger.debug "=== after require"

    package_name = Google::Cloud::Vision
                     .constants
                     .select { |sym| sym.to_s.downcase == version.to_s.downcase.tr("_", "") }
                     .first
    Rails.logger.debug "=== package_name: #{package_name}"


    service_module = Google::Cloud::Vision.const_get(package_name).const_get(:ImageAnnotator)
    Rails.logger.debug "=== service_module: #{service_module&.inspect}"

    service_module = service_module.const_get(:Rest) if transport == :rest
    Rails.logger.debug "=== service_module after rest check: #{service_module&.inspect}"

    ####################################################
    # THE NEXT LINE CRASHES THE APP SERVER EVERY TIME #########
    result = service_module.const_get(:Client).new(&block)

    # result = original_image_annotator(version, transport, &block)
    Rails.logger.debug "=== Result: #{result&.inspect}"
    result
  end
end

The logging output of this overridden version looks like this when called from the DebugController#moderate action:

Started POST "/debug/moderate" for 127.0.0.1 at 2024-06-28 15:31:17 
Processing by DebugController#moderate as HTML
  Parameters: {"authenticity_token"=>"[FILTERED]", "image"=>#<ActionDispatch::Http::UploadedFile:0x000001f0bbc2aaf8 @tempfile=#<Tempfile:C:/path_to/file.jpeg>, @content_type="image/jpeg", @original_filename="file.jpeg", @headers="Content-Disposition: form-data; name=\"image\"; filename=\"file.jpeg\"\r\nContent-Type: image/jpeg\r\n">, "commit"=>"Analyze Image"}
========== DebugController#moderate...
=== image_path: "C:/path_to/Temp/RackMultipart20240628-48008-573tm7.jpeg"
========== GOOGLE_APPLICATION_CREDENTIALS: D:/path_to/config/credentials/google_cloud_vision.json ==========
=== image_annotator, version: v1, transport: grpc
=== after require
=== package_name: V1
=== service_module: Google::Cloud::Vision::V1::ImageAnnotator
=== service_module after rest check: Google::Cloud::Vision::V1::ImageAnnotator
*** CRASH BACK TO SHELL THAT APP SERVER IS RUNNING IN ***
D:\App>

[dazuma]

A few clarifying questions:

1. By "crash" do you mean some kind of operating system process crash, like a segmentation fault or something? (Sorry, I'm unfamiliar with what those look like in Windows.)
2. Is the difference between the succeeding standalone script and the crashing Rails app only the fact that Rails is running in the latter? i.e. are you running on the same Windows workstation in both cases? How are you launching Rails? Do you have some kind of Windows-based Rails launcher, or are you starting it with rails server? Do you have any fancy auto-reloading or process caching system like bootsnap running in your Rails setup, and if so can you try turning those off?
3. I assume you have googleauth 1.11.0 in your bundle. As an experiment, can you try downgrading the googleauth gem to version 1.8.1 and retry?

[sampenguin]

1. Sorry I could have been clearer. The Puma server shuts down immediately, without warning or logging anything. So just the app server that is responsible for executing the rails controller code halts, there is not an OS crash. The web server and other processes do not go down with it.
2. Yes both running on the same Windows workstation. My development setup uses PowerShell 7.4.2, so I'll have usually a handful of PowerShell windows open, running processes. The core stack is nginx as https, Puma 6.4.2 (ruby 3.3.3-p89) as the Rails 7.1.3 server, and Sidekiq 7.2.2 that runs a few worker tasks on startup or for mail processing (none of which are proc'ing when I do these tests). All of the processes are started manually via various bat scripts, but ultimately they boil down to the simple commands of rails server, bundle exec sidekiq etc. so there is nothing fancy or automated in my development environment.
   When I execute the successful standalone script, it's also done through a PowerShell window
3. There are two Google gems installed from my Gemfile currently: gem 'googleauth', '~> 1.11' and gem 'google-cloud-vision' (and of course all their dependencies). I will try the downgrade at next opportunity and report back!

As a bit of further info re: all other pieces of the auth chain working, I have been working around this by not using the gem features and instead doing direct https calls to the Google Cloud Vision API successfully and seeing it tracked on Cloud Console. So all the auth info seems to be correct, fwiw.