Skip to content
This repository has been archived by the owner on Jul 13, 2023. It is now read-only.

Incorrect AWS Path URL in Paperchip 5 #2237

Closed
SylarRuby opened this issue Jun 27, 2016 · 14 comments
Closed

Incorrect AWS Path URL in Paperchip 5 #2237

SylarRuby opened this issue Jun 27, 2016 · 14 comments

Comments

@SylarRuby
Copy link

SylarRuby commented Jun 27, 2016

Gems:

gem "paperclip", "~> 5.0.0.beta1"
gem 'aws-sdk', '>= 2.0.34'
gem 'rails', '>= 5.0.0.rc1', '< 5.1'

environment rb:

  config.paperclip_defaults = {
       :s3_region => ENV['S3_REGION'],
       :storage => :s3,
       :s3_credentials => {
       :bucket => ENV['S3_BUCKET_NAME'],
       :access_key_id => ENV['AMAZON_ACCESS_KEY_ID'],
       :secret_access_key => ENV['AMAZON_SECRET_ACCESS_KEY']
  }
}

Image uploads to aws but the paperclip url is incorrect

Incorrect: https://s3.amazonaws.com // paperclip
Correct: https://s3-us-west-2.amazonaws.com // amazon - no issue here
@SylarRuby SylarRuby changed the title Image is saving with "data" in S3 Incorrect Path URL Jun 28, 2016
@brandosaurus-rex
Copy link

I am/was having that same issue. Apparently there is a config variable available now called :s3_host_name and you set it to...

:s3_host_name => 's3-us-west-2.amazonaws.com',

Have never had to do this before but could be a change made from the past couple years. I never had to set the region before either.

I found this solution here:

http://stackoverflow.com/questions/16149815/ruby-on-rails-4-paperclip-aws-image-url-is-wrong

@SylarRuby
Copy link
Author

That is with version 4. V4 works with that var. I'm pretty sure I've tried that. Are you using v5 and what gem version for aws?

@SylarRuby SylarRuby changed the title Incorrect Path URL Incorrect Path URL in Paperchip 5 Jul 8, 2016
@brandosaurus-rex
Copy link

remote: Using aws-sdk 2.3.19
remote: Installing figaro 1.1.1
remote: Installing paperclip 5.0.0

@SylarRuby
Copy link
Author

SylarRuby commented Jul 8, 2016

Ha. Maybe the aws gem was updated. I'll try the version you have later

@brandosaurus-rex
Copy link

ya i just tried this and it worked for me. i did have to restart my anvil app to make it work. so try restarting server after making change.

@ikhakoo
Copy link

ikhakoo commented Jul 25, 2016

Having a similar issue. Paperclip uses the correct AWS host name when i specify it but then it includes the bucket name again on the url

Paperclip provides this:

http://MY-BUCKET.s3.amazonaws.com/MY-BUCKET/photos/images/000/000/001/original/img-1.jpg?1469480304

Instead of this:

http://MY-BUCKET.s3.amazonaws.com/photos/images/000/000/001/original/img-1.jpg?1469480304

This is my paperclip config:

config.paperclip_defaults = { 
                                :storage => :s3,
                                :s3_region => ENV['AWS_REGION'],
                                :s3_host_name => "ENV['S3_HOST_NAME']",
                                :s3_credentials => {
                                  :bucket => ENV['S3_BUCKET_NAME'],
                                  :access_key_id => ENV['AWS_ACCESS_KEY_ID'],
                                  :secret_access_key => ENV['AWS_SECRET_ACCESS_KEY']
                                }
                              }

@tute tute changed the title Incorrect Path URL in Paperchip 5 Incorrect AWS Path URL in Paperchip 5 Aug 19, 2016
@danielbonnell
Copy link

Any progress on this issue? I'm having essentially the same problem. I have the following configuration:

config.paperclip_defaults = {
    storage: :s3,
    s3_region: 'us-standard',
    s3_host_name: 's3.amazonaws.com',
    s3_credentials: {
        bucket: "my-bucket",
        access_key_id: access_key,
        secret_access_key: secret_key
    }
}

I'd expect this to produce a url structure like so:

//s3.amazonaws.com/my-bucket/photos/images/000/000/001/original/something.jpg

Instead it looks like:

//my-bucket.s3.us-standard.amazonaws.com/my-bucket/photos/images/000/000/001/original/something.jpg

When I try to upload a file, I get the following error now:

Seahorse::Client::NetworkingError - Failed to open TCP connection to my-bucket.s3.us-standard.amazonaws.com:443 (Hostname not known: my-bucket.s3.us-standard.amazonaws.com):
  aws-sdk-core (2.6.38) lib/seahorse/client/networking_error.rb:7:in `initialize'
  aws-sdk-core (2.6.38) lib/seahorse/client/net_http/handler.rb:86:in `rescue in transmit'
  aws-sdk-core (2.6.38) lib/seahorse/client/net_http/handler.rb:68:in `transmit'
  aws-sdk-core (2.6.38) lib/seahorse/client/net_http/handler.rb:42:in `call'
  aws-sdk-core (2.6.38) lib/seahorse/client/plugins/content_length.rb:12:in `call'
  aws-sdk-core (2.6.38) lib/aws-sdk-core/plugins/s3_request_signer.rb:88:in `call'
  aws-sdk-core (2.6.38) lib/aws-sdk-core/plugins/s3_request_signer.rb:23:in `call'
  aws-sdk-core (2.6.38) lib/aws-sdk-core/xml/error_handler.rb:8:in `call'
  aws-sdk-core (2.6.38) lib/aws-sdk-core/plugins/helpful_socket_errors.rb:10:in `call'
  aws-sdk-core (2.6.38) lib/aws-sdk-core/plugins/s3_request_signer.rb:65:in `call'
  aws-sdk-core (2.6.38) lib/aws-sdk-core/plugins/s3_redirects.rb:15:in `call'
  aws-sdk-core (2.6.38) lib/aws-sdk-core/plugins/retry_errors.rb:87:in `call'
  aws-sdk-core (2.6.38) lib/aws-sdk-core/plugins/retry_errors.rb:118:in `retry_request'
  aws-sdk-core (2.6.38) lib/aws-sdk-core/plugins/retry_errors.rb:101:in `retry_if_possible'
  aws-sdk-core (2.6.38) lib/aws-sdk-core/plugins/retry_errors.rb:89:in `call'
  aws-sdk-core (2.6.38) lib/aws-sdk-core/plugins/retry_errors.rb:118:in `retry_request'
  aws-sdk-core (2.6.38) lib/aws-sdk-core/plugins/retry_errors.rb:101:in `retry_if_possible'
  aws-sdk-core (2.6.38) lib/aws-sdk-core/plugins/retry_errors.rb:89:in `call'
  aws-sdk-core (2.6.38) lib/aws-sdk-core/plugins/retry_errors.rb:118:in `retry_request'
  aws-sdk-core (2.6.38) lib/aws-sdk-core/plugins/retry_errors.rb:101:in `retry_if_possible'
  aws-sdk-core (2.6.38) lib/aws-sdk-core/plugins/retry_errors.rb:89:in `call'
  aws-sdk-core (2.6.38) lib/aws-sdk-core/plugins/s3_dualstack.rb:32:in `call'
  aws-sdk-core (2.6.38) lib/aws-sdk-core/plugins/s3_accelerate.rb:49:in `call'
  aws-sdk-core (2.6.38) lib/aws-sdk-core/plugins/s3_md5s.rb:31:in `call'
  aws-sdk-core (2.6.38) lib/aws-sdk-core/plugins/s3_expect_100_continue.rb:21:in `call'
  aws-sdk-core (2.6.38) lib/aws-sdk-core/plugins/s3_bucket_name_restrictions.rb:12:in `call'
  aws-sdk-core (2.6.38) lib/aws-sdk-core/plugins/s3_bucket_dns.rb:31:in `call'
  aws-sdk-core (2.6.38) lib/aws-sdk-core/rest/handler.rb:7:in `call'
  aws-sdk-core (2.6.38) lib/aws-sdk-core/plugins/user_agent.rb:12:in `call'
  aws-sdk-core (2.6.38) lib/seahorse/client/plugins/endpoint.rb:41:in `call'
  aws-sdk-core (2.6.38) lib/aws-sdk-core/plugins/param_validator.rb:21:in `call'
  aws-sdk-core (2.6.38) lib/seahorse/client/plugins/raise_response_errors.rb:14:in `call'
  aws-sdk-core (2.6.38) lib/aws-sdk-core/plugins/s3_sse_cpk.rb:19:in `call'
  aws-sdk-core (2.6.38) lib/aws-sdk-core/plugins/s3_dualstack.rb:24:in `call'
  aws-sdk-core (2.6.38) lib/aws-sdk-core/plugins/s3_accelerate.rb:34:in `call'
  aws-sdk-core (2.6.38) lib/aws-sdk-core/plugins/idempotency_token.rb:18:in `call'
  aws-sdk-core (2.6.38) lib/aws-sdk-core/plugins/param_converter.rb:20:in `call'
  aws-sdk-core (2.6.38) lib/seahorse/client/plugins/response_target.rb:21:in `call'
  aws-sdk-core (2.6.38) lib/seahorse/client/request.rb:70:in `send_request'
  aws-sdk-core (2.6.38) lib/seahorse/client/base.rb:207:in `block (2 levels) in define_operation_methods'
  aws-sdk-resources (2.6.38) lib/aws-sdk-resources/services/s3/file_uploader.rb:42:in `block in put_object'
  aws-sdk-resources (2.6.38) lib/aws-sdk-resources/services/s3/file_uploader.rb:49:in `open_file'
  aws-sdk-resources (2.6.38) lib/aws-sdk-resources/services/s3/file_uploader.rb:41:in `put_object'
  aws-sdk-resources (2.6.38) lib/aws-sdk-resources/services/s3/file_uploader.rb:34:in `upload'
  aws-sdk-resources (2.6.38) lib/aws-sdk-resources/services/s3/object.rb:251:in `upload_file'
  paperclip (5.1.0) lib/paperclip/storage/s3.rb:359:in `block in flush_writes'
  paperclip (5.1.0) lib/paperclip/storage/s3.rb:332:in `flush_writes'
  paperclip (5.1.0) lib/paperclip/attachment.rb:244:in `save'
   () Users/ACIDSTEALTH/.gem/ruby/2.3.0/bundler/gems/delayed_paperclip-5255c52c1f31/lib/delayed_paperclip/attachment.rb:67:in `save'
  paperclip (5.1.0) lib/paperclip/has_attached_file.rb:92:in `block in add_active_record_callbacks'
  activesupport (4.2.7) lib/active_support/callbacks.rb:446:in `block in make_lambda'
  activesupport (4.2.7) lib/active_support/callbacks.rb:228:in `block in halting_and_conditional'
  activesupport (4.2.7) lib/active_support/callbacks.rb:506:in `block in call'
  activesupport (4.2.7) lib/active_support/callbacks.rb:506:in `call'
  activesupport (4.2.7) lib/active_support/callbacks.rb:92:in `__run_callbacks__'
  activesupport (4.2.7) lib/active_support/callbacks.rb:778:in `_run_save_callbacks'
  activerecord (4.2.7) lib/active_record/callbacks.rb:302:in `create_or_update'
  activerecord (4.2.7) lib/active_record/persistence.rb:120:in `save'
  bullet (5.4.2) lib/bullet/active_record42.rb:42:in `save_with_bullet'
  activerecord (4.2.7) lib/active_record/validations.rb:37:in `save'
  activerecord (4.2.7) lib/active_record/attribute_methods/dirty.rb:21:in `save'
  activerecord (4.2.7) lib/active_record/transactions.rb:286:in `block (2 levels) in save'
  activerecord (4.2.7) lib/active_record/transactions.rb:351:in `block in with_transaction_returning_status'
  activerecord (4.2.7) lib/active_record/connection_adapters/abstract/database_statements.rb:213:in `block in transaction'
  activerecord (4.2.7) lib/active_record/connection_adapters/abstract/transaction.rb:184:in `within_new_transaction'
  activerecord (4.2.7) lib/active_record/connection_adapters/abstract/database_statements.rb:213:in `transaction'
  activerecord (4.2.7) lib/active_record/transactions.rb:220:in `transaction'
  activerecord (4.2.7) lib/active_record/transactions.rb:348:in `with_transaction_returning_status'
  activerecord (4.2.7) lib/active_record/transactions.rb:286:in `block in save'
  activerecord (4.2.7) lib/active_record/transactions.rb:301:in `rollback_active_record_state!'
  activerecord (4.2.7) lib/active_record/transactions.rb:285:in `save'
  app/controllers/employers/photos_controller.rb:76:in `block in upload'
  actionpack (4.2.7) lib/action_controller/metal/mime_responds.rb:212:in `respond_to'
  app/controllers/employers/photos_controller.rb:75:in `upload'
  actionpack (4.2.7) lib/action_controller/metal/implicit_render.rb:4:in `send_action'
  actionpack (4.2.7) lib/abstract_controller/base.rb:198:in `process_action'
  actionpack (4.2.7) lib/action_controller/metal/rendering.rb:10:in `process_action'
  actionpack (4.2.7) lib/abstract_controller/callbacks.rb:20:in `block in process_action'
  activesupport (4.2.7) lib/active_support/callbacks.rb:117:in `call'
  activesupport (4.2.7) lib/active_support/callbacks.rb:555:in `block (2 levels) in compile'
  activesupport (4.2.7) lib/active_support/callbacks.rb:505:in `call'
  activesupport (4.2.7) lib/active_support/callbacks.rb:92:in `__run_callbacks__'
  activesupport (4.2.7) lib/active_support/callbacks.rb:778:in `_run_process_action_callbacks'
  activesupport (4.2.7) lib/active_support/callbacks.rb:81:in `run_callbacks'
  actionpack (4.2.7) lib/abstract_controller/callbacks.rb:19:in `process_action'
  actionpack (4.2.7) lib/action_controller/metal/rescue.rb:29:in `process_action'
  actionpack (4.2.7) lib/action_controller/metal/instrumentation.rb:32:in `block in process_action'
  activesupport (4.2.7) lib/active_support/notifications.rb:164:in `block in instrument'
  activesupport (4.2.7) lib/active_support/notifications/instrumenter.rb:20:in `instrument'
  activesupport (4.2.7) lib/active_support/notifications.rb:164:in `instrument'
  actionpack (4.2.7) lib/action_controller/metal/instrumentation.rb:30:in `process_action'
  actionpack (4.2.7) lib/action_controller/metal/params_wrapper.rb:250:in `process_action'
  activerecord (4.2.7) lib/active_record/railties/controller_runtime.rb:18:in `process_action'
  actionpack (4.2.7) lib/abstract_controller/base.rb:137:in `process'
  actionview (4.2.7) lib/action_view/rendering.rb:30:in `process'
  actionpack (4.2.7) lib/action_controller/metal.rb:196:in `dispatch'
  actionpack (4.2.7) lib/action_controller/metal/rack_delegation.rb:13:in `dispatch'
  actionpack (4.2.7) lib/action_controller/metal.rb:237:in `block in action'
  actionpack (4.2.7) lib/action_dispatch/routing/route_set.rb:74:in `dispatch'
  actionpack (4.2.7) lib/action_dispatch/routing/route_set.rb:43:in `serve'
  actionpack (4.2.7) lib/action_dispatch/routing/mapper.rb:49:in `serve'
  actionpack (4.2.7) lib/action_dispatch/journey/router.rb:43:in `block in serve'
  actionpack (4.2.7) lib/action_dispatch/journey/router.rb:30:in `serve'
  actionpack (4.2.7) lib/action_dispatch/routing/route_set.rb:817:in `call'
  omniauth (1.2.2) lib/omniauth/strategy.rb:186:in `call!'
  omniauth (1.2.2) lib/omniauth/strategy.rb:164:in `call'
  omniauth (1.2.2) lib/omniauth/strategy.rb:186:in `call!'
  omniauth (1.2.2) lib/omniauth/strategy.rb:164:in `call'
  omniauth (1.2.2) lib/omniauth/strategy.rb:186:in `call!'
  omniauth (1.2.2) lib/omniauth/strategy.rb:164:in `call'
  omniauth (1.2.2) lib/omniauth/strategy.rb:186:in `call!'
  omniauth (1.2.2) lib/omniauth/strategy.rb:164:in `call'
  omniauth (1.2.2) lib/omniauth/strategy.rb:186:in `call!'
  omniauth (1.2.2) lib/omniauth/strategy.rb:164:in `call'
  bullet (5.4.2) lib/bullet/rack.rb:12:in `call'
  rack (1.6.5) lib/rack/deflater.rb:35:in `call'
  warden (1.2.6) lib/warden/manager.rb:35:in `block in call'
  warden (1.2.6) lib/warden/manager.rb:34:in `call'
  rack (1.6.5) lib/rack/etag.rb:24:in `call'
  rack (1.6.5) lib/rack/conditionalget.rb:38:in `call'
  rack (1.6.5) lib/rack/head.rb:13:in `call'
  actionpack (4.2.7) lib/action_dispatch/middleware/params_parser.rb:27:in `call'
  actionpack (4.2.7) lib/action_dispatch/middleware/flash.rb:260:in `call'
  rack (1.6.5) lib/rack/session/abstract/id.rb:225:in `context'
  rack (1.6.5) lib/rack/session/abstract/id.rb:220:in `call'
  actionpack (4.2.7) lib/action_dispatch/middleware/cookies.rb:560:in `call'
  activerecord (4.2.7) lib/active_record/query_cache.rb:36:in `call'
  activerecord (4.2.7) lib/active_record/connection_adapters/abstract/connection_pool.rb:653:in `call'
  actionpack (4.2.7) lib/action_dispatch/middleware/callbacks.rb:29:in `block in call'
  activesupport (4.2.7) lib/active_support/callbacks.rb:88:in `__run_callbacks__'
  activesupport (4.2.7) lib/active_support/callbacks.rb:778:in `_run_call_callbacks'
  activesupport (4.2.7) lib/active_support/callbacks.rb:81:in `run_callbacks'
  actionpack (4.2.7) lib/action_dispatch/middleware/callbacks.rb:27:in `call'
  actionpack (4.2.7) lib/action_dispatch/middleware/reloader.rb:73:in `call'
  actionpack (4.2.7) lib/action_dispatch/middleware/remote_ip.rb:78:in `call'
  rollbar (2.13.3) lib/rollbar/middleware/rails/rollbar.rb:24:in `block in call'
  rollbar (2.13.3) lib/rollbar.rb:145:in `scoped'
  rollbar (2.13.3) lib/rollbar/middleware/rails/rollbar.rb:22:in `call'
  better_errors (2.1.1) lib/better_errors/middleware.rb:84:in `protected_app_call'
  better_errors (2.1.1) lib/better_errors/middleware.rb:79:in `better_errors_call'
  better_errors (2.1.1) lib/better_errors/middleware.rb:57:in `call'
  actionpack (4.2.7) lib/action_dispatch/middleware/debug_exceptions.rb:17:in `call'
  rollbar (2.13.3) lib/rollbar/middleware/rails/show_exceptions.rb:22:in `call_with_rollbar'
  web-console (2.2.1) lib/web_console/middleware.rb:39:in `call'
  actionpack (4.2.7) lib/action_dispatch/middleware/show_exceptions.rb:30:in `call'
  railties (4.2.7) lib/rails/rack/logger.rb:38:in `call_app'
  railties (4.2.7) lib/rails/rack/logger.rb:20:in `block in call'
  activesupport (4.2.7) lib/active_support/tagged_logging.rb:68:in `block in tagged'
  activesupport (4.2.7) lib/active_support/tagged_logging.rb:26:in `tagged'
  activesupport (4.2.7) lib/active_support/tagged_logging.rb:68:in `tagged'
  railties (4.2.7) lib/rails/rack/logger.rb:20:in `call'
  actionpack (4.2.7) lib/action_dispatch/middleware/request_id.rb:21:in `call'
  rack (1.6.5) lib/rack/methodoverride.rb:22:in `call'
  rack (1.6.5) lib/rack/runtime.rb:18:in `call'
  activesupport (4.2.7) lib/active_support/cache/strategy/local_cache_middleware.rb:28:in `call'
  rack (1.6.5) lib/rack/lock.rb:17:in `call'
  actionpack (4.2.7) lib/action_dispatch/middleware/static.rb:120:in `call'
  rack (1.6.5) lib/rack/sendfile.rb:113:in `call'
  railties (4.2.7) lib/rails/engine.rb:518:in `call'
  railties (4.2.7) lib/rails/application.rb:165:in `call'
  rack (1.6.5) lib/rack/deflater.rb:35:in `call'
  rack (1.6.5) lib/rack/content_length.rb:15:in `call'
  puma (3.6.2) lib/puma/configuration.rb:225:in `call'
  puma (3.6.2) lib/puma/server.rb:578:in `handle_request'
  puma (3.6.2) lib/puma/server.rb:415:in `process_client'
  puma (3.6.2) lib/puma/server.rb:275:in `block in run'
  puma (3.6.2) lib/puma/thread_pool.rb:116:in `block in spawn_thread'

I'm using paperclip 5.1.0, aws-sdk 2.6.38, and delayed_paperclip 3.0.1.

@charlesbedrosian
Copy link

I have been having the same issue with paperclip 5.1.0, aws-sdk 2.6.38.

I resolved by setting my defaults as:

  config.paperclip_defaults = {
      storage: :s3,
      url: ':s3_domain_url',
      path: ':class/:attachment/:id/:style.:extension',
      s3_region: ENV.fetch('AWS_REGION'),
      s3_credentials: {
          bucket: ENV.fetch('S3_BUCKET_NAME'),
          access_key_id: ENV.fetch('AWS_ACCESS_KEY_ID'),
          secret_access_key: ENV.fetch('AWS_SECRET_ACCESS_KEY'),
          s3_host_name: ENV.fetch('S3_ENDPOINT'),
      }
  }

with S3_ENDPOINT=s3.amazonaws.com"

The key difference was putting the url & path as shown.

Perhaps this will work for you

@danielbonnell
Copy link

@charlesbedrosian I matched your setup exactly but it made no difference. I'm still stuck on the same error even after restarting the server. Here's my setup:

config.paperclip_defaults = {
    storage: :s3,
    url: ':s3_domain_url',
    path: ':class/:attachment/:id_partition/:style/:filename',
    s3_region: 'us-standard',
    s3_credentials: {
        bucket: ENV['s3_bucket_name'],
        access_key_id: ENV['s3_access_id'],
        secret_access_key: ENV['s3_access_key'],
        s3_host_name: 's3.amazonaws.com'
    }
}

@danielbonnell
Copy link

I solve my problem. According to the docs they renamed the us-standard region to us-east-1. The docs claim that you don't need to modify your application, however once I changed my region from us-standard to us-east-1 everything worked.

@rafbgarcia
Copy link

rafbgarcia commented Jan 9, 2017

thanks bro, you setup worked for me @danielbonnell

been fighting this the entire day

@tcg
Copy link

tcg commented Jun 16, 2017

Still a bit new to Ruby stuff, but it wasn't obvious to me that setting the Region wouldn't automatically change the host name. I set it manually (storing things in West, not East) and it works fine. Possibly related documentation clarity: #2379

@TargetXOps
Copy link

I'm on v5 and the solution posted by progus-florian on s3_host_name worked for me, using s3-us-west-2.amazonaws.com

@dijonkitchen
Copy link
Contributor

FYI #2379 is merged!

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

No branches or pull requests

10 participants