Thumbor version forked and fixed to work on the latest lambda enviroment #127

jodevsa · 2019-07-24T14:55:19Z

Since AWS have updated the lambda execution environment which broke old deployments (versions <=3.0) of this solution, and newer versions of this solution doesn't correctly implement the Thumbor interface (they shifted to sharpJS). I decided to Fork version 3.1 (Thumbor version) and make the required changes for it to work on lambda again

https://github.com/jodevsa/serverless-image-handler
Published solution template is available in the README

timkelty · 2019-07-24T15:55:17Z

Thank you!

scoates · 2019-07-24T16:39:34Z

Is this working for other 3rd parties? I tried updating our template to the published one here and CFN complained with:

2019-07-24 12:30:51 UTC-0400	ServerlessImageHandler	UPDATE_ROLLBACK_IN_PROGRESS	The following resource(s) failed to update: [CustomResource].
2019-07-24 12:30:50 UTC-0400	CustomResource	UPDATE_FAILED	Error occurred while GetObject. S3 Error Code: NoSuchBucket. S3 Error Message: The specified bucket does not exist (Service: AWSLambdaInternal; Status Code: 400; Error Code: InvalidParameterValueException; Request ID: REDACTED)

jasonobrown · 2019-07-24T16:41:39Z

Same for me. Really need something to work asap. Multiple production sites are failing right now. Jason O Brown

…

On Wed, Jul 24, 2019 at 11:39 AM Sean Coates ***@***.***> wrote: Is this working for other 3rd parties? I tried updating our template to the published one here and CFN complained with: 2019-07-24 12:30:51 UTC-0400 ServerlessImageHandler UPDATE_ROLLBACK_IN_PROGRESS The following resource(s) failed to update: [CustomResource]. 2019-07-24 12:30:50 UTC-0400 CustomResource UPDATE_FAILED Error occurred while GetObject. S3 Error Code: NoSuchBucket. S3 Error Message: The specified bucket does not exist (Service: AWSLambdaInternal; Status Code: 400; Error Code: InvalidParameterValueException; Request ID: REDACTED) — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub <#127>, or mute the thread <https://github.com/notifications/unsubscribe-auth/ACCTAVQWVLBUSWSZ7UVDSSDQBCAUTANCNFSM4IGQ6ATA> .

jodevsa · 2019-07-24T16:58:13Z

@jasonobrown which region are you deploying to ?

jasonobrown · 2019-07-24T16:58:41Z

@jodevsa us-east-2

scoates · 2019-07-24T17:00:00Z

I'm us-east-1, FWIW.

jasonobrown · 2019-07-24T17:00:57Z

Will it work if I deploy it in us-east-1 but my bucket is in us-east-2?

jodevsa · 2019-07-24T17:07:55Z

@jasonobrown @scoates it should work now for us-east-1, us-east-2 and eu-west-1

scoates · 2019-07-24T17:11:02Z

Thank you. You saved the rest of the week.

jasonobrown · 2019-07-24T17:12:23Z

It still failed for me in us-east-2

jodevsa · 2019-07-24T17:14:19Z

Wrong bucket permission.., should work now 👍

jasonobrown · 2019-07-24T17:17:08Z

Negative, failed again. I really appreciate the help by the way. Sorry to be such a nagging pain in the butt.

scoates · 2019-07-24T17:20:06Z

To be clear for others who might be following: this is working for me, now, on an updated stack (previously updated in February!), deployed in us-east-1 with a bucket in us-east-1.

timkelty · 2019-07-24T17:21:51Z

@jasonobrown @scoates it should work now for us-east-1, us-east-2 and eu-west-1

I have a stack in us-west-1…any way you could make it deployable there? 👼

I'd also like to nominate @jodevsa as person of the year!

timkelty · 2019-07-24T17:39:20Z

@jodevsa I'm seeing the same thing @jasonobrown is – failing in us-east-2 (and us-west-1). I've confirmed its working in us-east-1, though.

jasonobrown · 2019-07-24T17:48:52Z

@jodevsa @timkelty It's failing for me on us-east-1 as well.

jodevsa · 2019-07-24T17:56:17Z

Hi guys
The problem is that lambda requires the files to be in the same region which requires me to create multiple buckets for each region. there might be an easier way, not sure

@jasonobrown for some reason I'm unable to create a bucket in us-east-2 ( might be due to the fact i just deleted a bucket with that name in different region. I'll try again later
@timkelty us-west-1 should work now

working regions:
eu-west-1, us-east-1 and us-west-1

I'll continue on supporting all regions when i come back

timkelty · 2019-07-24T18:07:59Z

@jodevsa you are a saint. Confirmed that us-west-1 is now working.

jasonobrown · 2019-07-24T18:12:36Z

Can anyone tell me why I can't launch in us-east-1? Am I supposed to be changing the template url to say us-east-1 or leaving it as eu-west-1? I've tried both, but obviously something is wrong.

timkelty · 2019-07-24T18:21:47Z

@jasonobrown Should be able to just leave the template url as eu-west-1 (that's what I did).

jasonobrown · 2019-07-24T18:24:29Z

@timkelty I've tried that. The bucket should be able to be in a different region right?

idioteque1025 · 2019-07-24T18:44:57Z

@jasonobrown I am having the same issue as you. can not use us-east-1 as an origin. Seeing the rollback error.

jasonobrown · 2019-07-24T18:46:45Z

I just tried using a bucket in the same region and I'm still getting the error. I'm not setting an iam policy, but I didn't have one on the previous stack either (it's supposed to default to my user). Does that line up with what everyone else is doing (those who it has worked for)?

dlosie · 2019-07-24T18:47:46Z

I was able to deploy this to us-east-1 about an hour ago, but I can't anymore using the same steps. Not sure what's changed.

jasonobrown · 2019-07-24T18:49:57Z

Trying to deploy to us-east-1, I get this error.
Error occurred while GetObject. S3 Error Code: AuthorizationHeaderMalformed. S3 Error Message: The authorization header is malformed; the region 'us-east-1' is wrong; expecting 'us-east-2' (Service: AWSLambdaInternal; Status Code: 400; Error Code: InvalidParameterValueException; Request ID: e935fa07-2bc6-4efc-92b0-1128c970075b)

timkelty · 2019-07-24T18:56:25Z

@jasonobrown what has been working for me is:

go to my CloudFormation Stack
click Update
"replace template"
paste in https://cf-templates-nestrom.s3-eu-west-1.amazonaws.com/serverless-image-handler/1.0/serverless-image-handler.template

so far has worked for me in us-east-1 and us-west-1

jodevsa · 2019-07-24T19:00:28Z

@dlosie @jasonobrown Not sure what changed. us-east-1 used to work

hoantran-it · 2019-07-29T01:17:42Z

@jasonobrown what has been working for me is:

go to my CloudFormation Stack

click Update

"replace template"

paste in https://cf-templates-nestrom.s3-eu-west-1.amazonaws.com/serverless-image-handler/1.0/serverless-image-handler.template

so far has worked for me in us-east-1 and us-west-1

@timkelty I'm working on ap-southeast-1, and this solution can not work. Can you please to give advice for that. Thank you!

chuyuou · 2019-07-29T01:44:14Z

I am in the same situation... region is ap-southeast-2. Thanks!

hoantran-it · 2019-07-29T02:58:57Z

@jodevsa Could you help on other regions please. I have production on ap-southeast-1,. Thanks!

oakesjosh · 2019-07-29T03:13:40Z

I ran into the issue of my region not being supported the other day. I managed to figure out how to get it setup in my own S3 account and have the template pull from there.

Build the repository or download it already built - Download link (unzip before putting in S3)
Create an S3 bucket with your deployment region appended to the name. So if you're deploying in us-west-2 your bucket name would be my-bucket-name-us-west-2
Upload the 3 zip files into your S3 bucket.
Edit the locations in the .template that reference the bucket/location of the zip files. These references are on lines 135 & 136, 146 & 147, 160 & 161 Note the directory structure on lines 136, 147, and 161. If you uploaded the files in the root of the S3 bucket these lines will have to be adjusted to reflect that.
Now upload this edited template into cloud formation and your deployment should work

ahmetcetin39 · 2019-07-29T06:20:07Z

@jasonobrown what has been working for me is:

go to my CloudFormation Stack

click Update

"replace template"

paste in https://cf-templates-nestrom.s3-eu-west-1.amazonaws.com/serverless-image-handler/1.0/serverless-image-handler.template

so far has worked for me in us-east-1 and us-west-1

I had changed the original template file, and when I check the old template and this new template I do not see much difference. Could you please explain which part is changed in the template so that I can change it manually?
Thanks.

Is the change actually these two lines:
S3Bucket: nestrom-production-image-handler-files
S3Key: serverless-image-handler/1.0/serverless-image-handler.zip

chuyuou · 2019-07-29T06:25:03Z

@oakesjosh Hi, thanks for your solution. I have tried to update 135 & 136, 146 & 147, 160 & 161. put the 3 zip files under the root of the bucket directly. It says

Error occurred while GetObject. S3 Error Code: NoSuchBucket. S3 Error Message: The specified bucket does not exist (Service: AWSLambdaInternal; Status Code: 400; Error Code: InvalidParameterValueException; Request ID: blabla)

Is there other things that I need to change?

hoantran-it · 2019-07-29T06:39:16Z

@oakesjosh I tried with your solution but I have the same issue like @chuyuou. So sad!

ahmetcetin39 · 2019-07-29T06:43:07Z

@oakesjosh Hi, thanks for your solution. I have tried to update 135 & 136, 146 & 147, 160 & 161. put the 3 zip files under the root of the bucket directly. It says

Error occurred while GetObject. S3 Error Code: NoSuchBucket. S3 Error Message: The specified bucket does not exist (Service: AWSLambdaInternal; Status Code: 400; Error Code: InvalidParameterValueException; Request ID: blabla)

Is there other things that I need to change?

I am having the same problem from eu-north-1 region. @jodevsa

chuyuou · 2019-07-29T06:54:58Z

@oakesjosh my region is ap-southeast-2, The only difference between my template and the original template is the bucket name.

ahmetcetin39 · 2019-07-29T07:52:55Z

@oakesjosh Hi, thanks for your solution. I have tried to update 135 & 136, 146 & 147, 160 & 161. put the 3 zip files under the root of the bucket directly. It says

Error occurred while GetObject. S3 Error Code: NoSuchBucket. S3 Error Message: The specified bucket does not exist (Service: AWSLambdaInternal; Status Code: 400; Error Code: InvalidParameterValueException; Request ID: blabla)

Is there other things that I need to change?

@oakesjosh I am having the same problem. Did you change something else besides these?

jaeyow · 2019-07-29T09:07:51Z

Just wanted to add that I was able to make this work by:
deploying the cloudformation stack to us-east-1,
while my images bucket is located in ap-southeast-2

I got similar errors like the above 'NoSuchBucket'...

The critical bit was the first 2 Parameters of the template:

Origin S3 Bucket => 'my-bucket-name'
Origin S3 Bucket Region => 'ap-southeast-2'

I used the supplied template as is, including the following bucket and key for the image handler code:

S3Bucket: nestrom-production-image-handler-files
S3Key: serverless-image-handler/1.0/serverless-image-handler.zip

Next I plan to point it to my own bucket, so that I am not dependent on this 3rd party resource that I don't have control over.

Edit: will follow instructions by @oakesjosh above and see if this can work with the S3 bucket and the zip files within my infrastructure in ap-southeast-2.

Edit: Yes I can confirm that @oakesjosh instructions above worked for me:

Image Handler is deployed in ap-southeast-2
zip files are in s3 bucket 1 also in ap-southeast-2
My images are in s3 bucket 2 also in ap-southeast-2
My edited template in in s3 bucket 3 also in ap-southeast-2

as @chuyuou said:

for example, {bucket-name}-ap-southeast-2 in s3, then in the template the Origin S3 Bucket should written as :{bucket-name} , without region

The only issues I had was not having the correct bucket policy to allow CF to get the zip files from my bucket 1.

Thanks!

vpont · 2019-07-29T10:22:05Z

Not working for eu-west-3:

Error occurred while GetObject. S3 Error Code: NoSuchBucket. S3 Error Message: The specified bucket does not exist (Service: AWSLambdaInternal; Status Code: 400; Error Code: InvalidParameterValueException; Request ID: removed)

Please advice.
Thank you!

Edit: Fixed following @oakesjosh instructions.

rpagliuca · 2019-07-29T12:00:00Z

Thanks @timkelty, worked for me on us-west-2

chuyuou · 2019-07-29T13:35:08Z

Worked out..Thanks @oakesjosh ! This can work!! Saved my tomorrow!!

ahmetcetin39 · 2019-07-29T13:44:09Z

Worked out..Thanks @oakesjosh ! This can work!! Saved my tomorrow!!

@chuyuou Did you do something else rather than changing the bucket names?

chuyuou · 2019-07-29T13:46:10Z

for example, {bucket-name}-ap-southeast-2 in s3, then in the template the Origin S3 Bucket should written as :{bucket-name} , without region..Follow @oakesjosh instruction and use those three zip files.

hoantran-it · 2019-07-29T14:17:18Z

@chuyuou You save my life!
Thank you all! @jodevsa @oakesjosh @timkelty

SilverFoxA · 2019-07-30T06:27:54Z

I'm getting this error

Custom Resource failed to stabilize in expected time any thoughts? followed the above steps.

panovitch · 2019-07-30T08:40:44Z

Thanks a lot for the fix @jodevsa!
By the way, if anyone else here is struggling with errors like invalid elf header, for me building through the docker image included in the PR didnt work, I had to start an EC2 running AWS linux and build it there.

hayesry · 2019-07-30T19:06:31Z

Hello all - This version of the solution has been patched and is now available at:
(https://solutions-reference.s3.amazonaws.com/serverless-image-handler/v3.1.1/serverless-image-handler.template)
This patch will fix any issues stemming from the Lambda Execution Environment updates that have been rolling out. More details on that update available here:(https://aws.amazon.com/blogs/compute/upcoming-updates-to-the-aws-lambda-execution-environment/)

wcdit-dev · 2019-08-01T10:42:24Z

@hayesry Thank you! Solved the problem on us-west-2.

thisjeremiah · 2019-08-05T05:39:26Z

In case anyone else is managing their own template based on the original template: the only fix I needed in my template was to switch all instances of "3.1.0" to "3.1.1". This will target the correct solution image handler zip.

peterkuiper · 2019-08-07T08:30:39Z

@hayesry Can you share the updated build process? The new solution does not work for us since we have to support animated GIFs :(

Note: I have already updated my fork (including Docker build), for those who like to do the builds themselves (I am not using CF). Thanks to @jodevsa for sharing the correct value for PYCURL_SSL_LIBRARY!

aablain · 2019-08-16T23:05:27Z

@hayesry Thank you for linking that! I used that template with the steps in #127 (comment) and it solved the issue for me. VERY MUCH appreciate it!

rromanchuk · 2019-12-23T03:32:02Z

This entire section https://github.com/awslabs/serverless-image-handler#building-distributable-for-customization should be removed until any sort of semblance of a deterministic build can be achieved.

You can brick you entire deployment by simply changing a Tag value and end up replacing your imagehandler function with an architecture that is unsupported. It will allow you do that without a peep from the build process.

My current strategy is deploy the solution, and drift the stack while documenting the changes. To upgrade, I redeploy the solution, reapply my drift, then adjust my dns aliases. I'd be thrilled with just an AMI of environment aws solutions team uses to build https://aws.amazon.com/solutions/serverless-image-handler/

Also thinking of ways to completely decouple the lambda function from this stack completely, way to much risk exposure building image libraries.

beomseoklee · 2020-02-06T21:51:53Z

We are closing this issue, but please feel free to open the issue again if you any other support.

onassar · 2021-03-22T18:06:56Z

Has anyone here successfully mitigated the risk posed by AWS no longer supporting botocore.requests through the cfn-response module?

I'm a bit confused as to what actually needs to be done.

This is the email I have from them in January 2021:

jodevsa changed the title ~~Thumbor version forked and fixed~~ Thumbor version forked and fixed to work on the latest lambda enviroment Jul 24, 2019

This was referenced Jul 24, 2019

images in subfolders doesn't work #112

Closed

libcurl link-time ssl backend (openssl) is different from compile-time ssl backend (nss) #128

Closed

hayesry mentioned this issue Aug 4, 2019

New JSON format for v4.0 is not working #124

Closed

fanyizhen1995 mentioned this issue Dec 13, 2019

需要重新安装pycurl JoeShi/aws-serverless-image-handler#1

Closed

georgebearden closed this as completed Feb 7, 2020

Thumbor version forked and fixed to work on the latest lambda enviroment #127

Thumbor version forked and fixed to work on the latest lambda enviroment #127

Comments

jodevsa commented Jul 24, 2019 • edited Loading

timkelty commented Jul 24, 2019

scoates commented Jul 24, 2019

jasonobrown commented Jul 24, 2019 via email

jodevsa commented Jul 24, 2019

jasonobrown commented Jul 24, 2019

scoates commented Jul 24, 2019

jasonobrown commented Jul 24, 2019

jodevsa commented Jul 24, 2019 • edited Loading

scoates commented Jul 24, 2019

jasonobrown commented Jul 24, 2019

jodevsa commented Jul 24, 2019

jasonobrown commented Jul 24, 2019

scoates commented Jul 24, 2019

timkelty commented Jul 24, 2019 • edited Loading

timkelty commented Jul 24, 2019

jasonobrown commented Jul 24, 2019

jodevsa commented Jul 24, 2019 • edited Loading

timkelty commented Jul 24, 2019

jasonobrown commented Jul 24, 2019

timkelty commented Jul 24, 2019

jasonobrown commented Jul 24, 2019

idioteque1025 commented Jul 24, 2019

jasonobrown commented Jul 24, 2019

dlosie commented Jul 24, 2019

jasonobrown commented Jul 24, 2019

timkelty commented Jul 24, 2019 • edited Loading

jodevsa commented Jul 24, 2019 • edited Loading

hoantran-it commented Jul 29, 2019

chuyuou commented Jul 29, 2019

hoantran-it commented Jul 29, 2019 • edited Loading

oakesjosh commented Jul 29, 2019 • edited Loading

ahmetcetin39 commented Jul 29, 2019 • edited Loading

chuyuou commented Jul 29, 2019 • edited Loading

hoantran-it commented Jul 29, 2019

ahmetcetin39 commented Jul 29, 2019 • edited Loading

chuyuou commented Jul 29, 2019

ahmetcetin39 commented Jul 29, 2019 • edited Loading

jaeyow commented Jul 29, 2019 • edited Loading

vpont commented Jul 29, 2019 • edited Loading

rpagliuca commented Jul 29, 2019

chuyuou commented Jul 29, 2019

ahmetcetin39 commented Jul 29, 2019

chuyuou commented Jul 29, 2019 • edited Loading

hoantran-it commented Jul 29, 2019

SilverFoxA commented Jul 30, 2019

panovitch commented Jul 30, 2019 • edited Loading

hayesry commented Jul 30, 2019 • edited Loading

wcdit-dev commented Aug 1, 2019

thisjeremiah commented Aug 5, 2019

peterkuiper commented Aug 7, 2019 • edited Loading

aablain commented Aug 16, 2019

rromanchuk commented Dec 23, 2019

beomseoklee commented Feb 6, 2020

onassar commented Mar 22, 2021

jodevsa commented Jul 24, 2019 •

edited

Loading

jodevsa commented Jul 24, 2019 •

edited

Loading

timkelty commented Jul 24, 2019 •

edited

Loading

jodevsa commented Jul 24, 2019 •

edited

Loading

timkelty commented Jul 24, 2019 •

edited

Loading

jodevsa commented Jul 24, 2019 •

edited

Loading

hoantran-it commented Jul 29, 2019 •

edited

Loading

oakesjosh commented Jul 29, 2019 •

edited

Loading

ahmetcetin39 commented Jul 29, 2019 •

edited

Loading

chuyuou commented Jul 29, 2019 •

edited

Loading

ahmetcetin39 commented Jul 29, 2019 •

edited

Loading

ahmetcetin39 commented Jul 29, 2019 •

edited

Loading

jaeyow commented Jul 29, 2019 •

edited

Loading

vpont commented Jul 29, 2019 •

edited

Loading

chuyuou commented Jul 29, 2019 •

edited

Loading

panovitch commented Jul 30, 2019 •

edited

Loading

hayesry commented Jul 30, 2019 •

edited

Loading

peterkuiper commented Aug 7, 2019 •

edited

Loading