Support for Elastic Beanstalk's Periodic Tasks? (via cron.yml)

[btc]

Does this library support Elastic Beanstalk's periodic tasks via cron.yml file? I do not see documentation for such a feature.

[btc]

If not, here is my basic implementation that I am using now:

version: 1
cron:
      - name: 'HeartbeatJob'
        url: '/_jobs'
        schedule: "*/30 * * * * *"
      - name: 'FetchStreamMetricsJob'
        url: '/_jobs'
        schedule: "*/5 * * * * *"

Rails.application.routes.draw do
  post '_jobs', to: 'job_runner#perform_now'
end

class JobRunnerController < ApplicationController
  protect_from_forgery :except => :perform_now

  def perform_now
    return head(:unauthorized) if ENV['DISABLE_SQS_CONSUMER']

    job_name = request.headers['X-Aws-Sqsd-Taskname']
    job = job_name.constantize.new
    job.perform_now
    return head(:ok)
  end
end

[tawan]

Hi @briantigerchow,

Currently this gem does not support periodic tasks, but I have been evaluating this possibility of including a support for this feature. Thanks a lot for your submission and for sharing your implementation. I'll leave this issue open and I'll come back to it in the near future.

[karensg]

Good question. I am looking forward to hear your advice on how to set it up according to best practices.

[sauy7]

FWIW I've implemented the same as @briantigerchow (cron.yml) except that I wrap the post '_jobs' endpoint in a condition based on the Rails environment (instead of having to test for ENV['DISABLE_SQS_CONSUMER'] in the controller). My web tier is "production" and my worker tier is "production_worker". My routes file (simplified) then has an if condition:

if Rails.env.production_worker?
  post '_jobs' ...
else
  # regular web endpoints
end

Whilst setting up separate Rails environments may seem like overkill, I have other requirements such as my web app is configured to force SSL but the worker app responds on port 80.

UPDATE: I should add that this approach does not expose web urls in the worker tier, so if you need to refer to web tier urls (e.g., in an email), follow @briantigerchow's approach.

[karensg]

I just implemented your proposal. Thanks

[masonjeffreys]

Instead of using the methods above, I added a check to the middleware.

If a request appears to be coming from SQS but doesn't originate from the ActiveElasticJob gem, we check to see if it matches the "_jobs" route. If so, it's considered to be from the cron.yml file and we perform the action immediately. It's a bit ugly though.

This allows me to leave config.force_ssl on even for cron jobs.
For class SqsMessageConsumer

  def call(env) #:nodoc:
    request = ActionDispatch::Request.new env
    if enabled? && aws_sqsd?(request)
      if originates_from_gem?(request)
        unless request.local? || sent_from_docker_host?(request)
          m = "Accepts only requests from localhost for job processing".freeze
          return ['403', {CONTENT_TYPE_HEADER_NAME => 'text/plain' }, [ m ]]
        end
        begin
          verify!(request)
          job = JSON.load(request.body)
          ActiveJob::Base.execute(job)
        rescue ActiveElasticJob::MessageVerifier::InvalidDigest => e
          return [
            '403',
            {CONTENT_TYPE_HEADER_NAME => 'text/plain' },
            ["incorrect digest"]]
        end
        return [
          OK_RESPONSE_CODE ,
          {CONTENT_TYPE_HEADER_NAME => CONTENT_TYPE },
          [ '' ]]
      elsif env["PATH_INFO"] == "/_jobs"
        ### if enabled and from aws_sqsd but did not originate from the gem, ###
        ### it may be a cron jobs from cron.yml. Perform that here ####
        begin
          job_name = request.headers['X-Aws-Sqsd-Taskname']
          job = job_name.constantize.new
          job.perform_now
           c = "Performed a cron job from sqs".freeze
          return [OK_RESPONSE_CODE, {CONTENT_TYPE_HEADER_NAME => 'text/plain' }, [ c ]]
        rescue
          return [
            '403',
            {CONTENT_TYPE_HEADER_NAME => 'text/plain' },
            ["Unsuccessful cron job attempt sqs for task: #{request.headers['X-Aws-Sqsd-Taskname']}"]]
        end
      end
    end
    @app.call(env)
  end

[btc]

How does the new built-in support for this behavior work?

[tawan]

Hi all,

Support for periodic tasks has just been released in version v2.0.0. I also added documentation on its usage to the README.
@briantigerchow