Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

API uploads images via URL #3573

Merged
merged 1 commit into from
Apr 21, 2020

Conversation

calebhaye
Copy link
Contributor

@calebhaye calebhaye commented Mar 31, 2020

Description

This enhancement give the images API the ability to create images from URLs. A
simple check determines whether or not the attachment parameter is a URL. If it
is a URL, OpenURI is used to download the image and it subsequently undergoes
the standard post upload processing (via paperclip). If it is not a URL, then
the attachment parameter is handled in the same manner it was prior to this
commit.

Fixes #3572

Checklist:

  • I have followed Pull Request guidelines
  • I have added a detailed description into each commit message
  • I have updated Guides and README accordingly to this change (if needed)
  • I have added tests to cover this change (if needed)
  • I have attached screenshots to this PR for visual changes (if needed)

@kennyadsl
Copy link
Member

@calebhaye I think that HoundCI is complaining at valid things, can you please take a look?

@calebhaye
Copy link
Contributor Author

calebhaye commented Apr 3, 2020 via email

@calebhaye calebhaye force-pushed the api-upload-image-by-url branch from 579794a to 1f95dde Compare April 5, 2020 01:33
@calebhaye
Copy link
Contributor Author

calebhaye commented Apr 5, 2020

Okay, all ready :) @kennyadsl

@calebhaye calebhaye force-pushed the api-upload-image-by-url branch 2 times, most recently from 4bbd9f2 to 844903a Compare April 5, 2020 04:15
Copy link
Member

@kennyadsl kennyadsl left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks a lot!

I left a couple of notes, can we also add some documentation in this API endpoint description?


def valid_url?(url)
uri = URI.parse url
uri.is_a?(URI::HTTP) && uri.host.present?
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why do we need to check the host here?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe we don't? Will fix.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

fixed in latest commit.

expect do
post spree.api_product_images_path(product.id), params: {
image: {
attachment: 'https://www.google.com/images/branding/googlelogo/2x/googlelogo_color_92x30dp.png',
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What about using an image that belongs to Solidus, e.g. https://github.com/solidusio/brand/raw/1827e7afb7ebcf5a1fc9cf7bf6cf9d277183ef11/PNG/solidus-logo-dark.png ?

This way the test suite is safe, whatever Google decides to do with their assets.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I like that suggestion. I was definitely unsure about this one. Will fix

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

fixed in latest commit.

@calebhaye
Copy link
Contributor Author

Thanks a lot!

I left a couple of notes, can we also add some documentation in this API endpoint description?

Hi @kennyadsl,

The notes you left have been addressed.

The current implementation has zilch for documentation in the API endpoint description, which is probably why neither of us know if it really works or not. I could add something for docs, but I honestly don't know where/how to do that. Will you please point me in the right direction? I'd like to wrap this one up soon if we can 👍, and at present it seems like the only blocker for merging is your request for documentation.

Thank you

Copy link
Member

@kennyadsl kennyadsl left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm figuring out the best place where to add that info to the documentation, in the meantime, I've probably found a small issue with the code. Let me know if it's a valid concern, thanks again!

@@ -15,7 +15,11 @@ def show

def create
authorize! :create, Image
@image = scope.images.create(image_params)
if valid_url? image_params[:attachment]
@image = scope.images.create(attachment: URI.open(image_params[:attachment]))
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looking twice, is this removing the other attributes in image_params and creating an image with only the attachment? I think we should update the image_params instead, preserving the rest of the attributes.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think it may be. I'll do some debugging soon and get back to you.

Copy link
Contributor Author

@calebhaye calebhaye Apr 15, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@kennyadsl Good catch. I added a fix that addresses your concern. The line following the line you commented on now updates the resultant Spree::Image object with the other attributes within image_params, except for :attachment

I tried updating image_params instead, as per your suggestion, but that resulted in the following error when running the spec:

Paperclip::AdapterRegistry::NoHandlerError:
       No handler found for "https://github.com/solidusio/brand/raw/1827e7afb7ebcf5a1fc9cf7bf6cf9d277183ef11/PNG/solidus-logo-dark.png"

So, I opted to create the image first using image_params[:attachment], and then update all the other attributes using ActionController::Parameters except method.

I also updated the spec so that it fails without the newly added line.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It sounds weird... I'll try to take a look soon and see if I find why this happens. I'd prefer if we can make a single INSERT query here, instead of also updating the record later.

Copy link
Member

@kennyadsl kennyadsl Apr 17, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm trying locally and I got what you mean now. It looks like a bug in paperclip: if it doesn't have the adapter with create and all the rest of attributes I can't get why it has the adapter if you create the record only with the attachment one...

Anyway, I think we can avoid the double query with this code:

# frozen_string_literal: true

module Spree
  module Api
    class ImagesController < Spree::Api::BaseController
      def create
        authorize! :create, Image

        @image = scope.images.build(image_params.except(:attachment))
        @image.attachment = prepared_attachment
        @image.save

        respond_with(@image, status: 201, default_template: :show)
      end

      private

      def prepared_attachment
        uri = URI.parse image_params[:attachment]

        if uri.is_a? URI::HTTP
          URI.open(image_params[:attachment])
        else
          image_params[:attachment]
        end
      rescue URI::InvalidURIError
        image_params[:attachment]
      end
    end
  end
end

I'm also wondering if we need something similar in the update method but that could be part of another PR if you don't have time and need for that to work in your use case.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@calebhaye I was wondering if you had a chance to look at this? Would be really cool to get this merged. 🙂

Copy link
Contributor Author

@calebhaye calebhaye Apr 19, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@aldesantis yep! Just pushed the latest, which includes an updated update method and a new spec for testing the update method when updating the attachment property.

@kennyadsl please note: I originally updated the update method to look like this:

@image.assign_attributes(image_params.except(:attachment))
@image.attachment = prepared_attachment if image_params[:attachment].present?
@image.save

But I changed it to the following because I feel as though it reads more logically:

if image_params[:attachment].present?
  @image.assign_attributes(image_params.except(:attachment))
  @image.attachment = prepared_attachment
  @image.save
else
  @image.update image_params
end

I'd rather have it make more sense to developers who reference the code in the future, than to have fewer lines

@kennyadsl
Copy link
Member

For the documentation, I mean adding a small description here, something like:

      attachment:
        type: string
        description: |
          The URL of the image to attach.

Also, can you please squash commits into a single one?

Thanks again.

@calebhaye
Copy link
Contributor Author

calebhaye commented Apr 14, 2020

For the documentation, I mean adding a small description here, something like:

      attachment:
        type: string
        description: |
          The URL of the image to attach.

Also, can you please squash commits into a single one?

Thanks again.

Thanks for the pointer. If that's the prescribed methodology, I'll add the docs there. I noticed that the other API controllers (like the create method in the ProductsController, for instance), just have code comments and no actual docs that manifest on Stoplight. I'll leave it up to you, should I add the docs as a comment, add them to the aforementioned reference point suggested by you, or both? Please advise @kennyadsl

And yes of course I'll squash the commits -- thx

@calebhaye calebhaye force-pushed the api-upload-image-by-url branch 3 times, most recently from 3415f2b to 61a86be Compare April 15, 2020 20:23
@calebhaye
Copy link
Contributor Author

Should be good to go 🤞

@kennyadsl kennyadsl added changelog:solidus_api Changes to the solidus_api gem type:enhancement Proposed or newly added feature labels Apr 17, 2020
@calebhaye calebhaye force-pushed the api-upload-image-by-url branch from 61a86be to f30bd3e Compare April 19, 2020 17:41
This enhancement give the images API the ability to create and update images from
URLs.  A simple check determines whether or not the attachment parameter is a URL.

If it is a URL: OpenURI is used to download the image and it subsequently
undergoes the standard post upload processing (via paperclip).  After
processing the image, the other attributes (besides :attachment) are updated on
the resultant Spree::Image.

If it is not a URL: the attachment parameter is handled in the same manner
it was prior to this commit.

API documentation added.

Uses static image from solidus repo, to avoid external dependency
@calebhaye calebhaye force-pushed the api-upload-image-by-url branch from f30bd3e to 4a1824a Compare April 19, 2020 18:12
Copy link
Member

@kennyadsl kennyadsl left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you!

@kennyadsl kennyadsl requested a review from aldesantis April 20, 2020 11:53
@kennyadsl kennyadsl merged commit f38a7a4 into solidusio:master Apr 21, 2020
calebhaye added a commit to droprx/solidus that referenced this pull request Apr 22, 2020
Paperclip’s CVE-2017–0889 Server Side Request Forgery (SSRF)
vulnerability (https://medium.com/in-the-weeds/all-about-paperclips-cve-2017-0889-server-side-request-forgery-ssrf-vulnerability-8cb2b1c96fe8) was likely introduced in solidusio#3573.  This patch
ensures that redirection does not take place in the context of
uploading an image via the api.
calebhaye added a commit to droprx/solidus that referenced this pull request Apr 22, 2020
Paperclip’s CVE-2017–0889 Server Side Request Forgery (SSRF)
vulnerability (https://medium.com/in-the-weeds/all-about-paperclips-cve-2017-0889-server-side-request-forgery-ssrf-vulnerability-8cb2b1c96fe8) was likely introduced in solidusio#3573.  This patch
ensures that redirection does not take place in the context of
uploading an image via the api.
calebhaye added a commit to droprx/solidus that referenced this pull request Apr 22, 2020
Paperclip’s CVE-2017–0889 Server Side Request Forgery (SSRF)
vulnerability (https://medium.com/in-the-weeds/all-about-paperclips-cve-2017-0889-server-side-request-forgery-ssrf-vulnerability-8cb2b1c96fe8) was likely introduced in solidusio#3573.  This patch
ensures that redirection does not take place in the context of
uploading an image via the api.
calebhaye added a commit to droprx/solidus that referenced this pull request Apr 24, 2020
Paperclip’s CVE-2017–0889 Server Side Request Forgery (SSRF)
vulnerability (https://medium.com/in-the-weeds/all-about-paperclips-cve-2017-0889-server-side-request-forgery-ssrf-vulnerability-8cb2b1c96fe8) was likely introduced in solidusio#3573.  This patch
ensures that redirection does not take place in the context of
uploading an image via the api, and ensures that a private address
is not used for the attachment parameter.
kennyadsl added a commit that referenced this pull request May 15, 2020
Revert "API uploads images via URL" (#3573)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
changelog:solidus_api Changes to the solidus_api gem type:enhancement Proposed or newly added feature
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Image API should allow upload via URL
4 participants