Magento 2.3 images cache generation generates wrong hash for some images with a custom theme that has the keep_frame var set to 0

[carolineboivin]

Summary (*)

I just upgrade my Magento 2 project from 2.2.6 to 2.3. After the upgrade, some images in the catalog are showing placeholders instead of the images. I have regenerated the images cache with the catalog:images:resize but some images are still missing (swatch images in category pages and image thumbnail in category pages).

After some investigation, it seems that is it a problem with the hash used for the image cache directory. When generating the images cache in the CLI, all images have in their misc params used to build the hash a value of "keep_frame" => 1, although in the view.xml file of my custom theme, the param "product_image_white_borders" is set to 0. When generating the image path for the catalog pages, the keep frame param is 0.

My investigation led me to the ParamsBuilder class (module-catalog/Model/Product/Image/ParamsBuilder) that uses its viewConfig property to get the value of the var "product_image_white_borders" in the "hasDefaultFrame" function. Seemingly, when running the CLI command, the view config that is injected in the ParamsBuilder is not the same that is used by the ImageResize command, and it has a wrong "product_image_white_borders" value of 1. It seems that it does not use the theme that the ImageResize command is currently processing, and its scope is global.

Environment

Magento 2.3 Open Source with custom frontend theme

Examples (*)

Assign images to products in the catalog.
Set the value of the variable "product_image_white_borders" to 0 in your custom theme view.xml file.
Regenerate product images cache with the CLI command catalog:images:resize.

Proposed solution

The product thumbnail in category pages should appear but instead the placeholder image is shown.

[magento-engcom-team]

Hi @carolineboivin. Thank you for your report.
To help us process this issue please make sure that you provided the following information:

• Summary of the issue
• Information on your environment
• Steps to reproduce
• Expected and actual results

Please make sure that the issue is reproducible on the vanilla Magento instance following Steps to reproduce. To deploy vanilla Magento instance on our environment, please, add a comment to the issue:

@magento-engcom-team give me $VERSION instance

where $VERSION is version tags (starting from 2.2.0+) or develop branches (for example: 2.3-develop).
For more details, please, review the Magento Contributor Assistant documentation.

@carolineboivin do you confirm that you was able to reproduce the issue on vanilla Magento instance following steps to reproduce?

• yes
• no

[carolineboivin]

@magento-engcom-team give me 2.3.0 instance

[magento-engcom-team]

Hi @carolineboivin. Thank you for your request. I'm working on Magento 2.3.0 instance for you

[magento-engcom-team]

Hi @carolineboivin, here is your Magento instance.
Admin access: https://i-19710-2-3-0.instances.magento-community.engineering/admin
Login: admin Password: 123123q
Instance will be terminated in up to 3 hours.

[carolineboivin]

Seems to be a duplicate of #4622 after all

[magento-engcom-team]

Hi @engcom-backlog-nazar. Thank you for working on this issue.
In order to make sure that issue has enough information and ready for development, please read and check the following instruction: 👇

• 1. Verify that issue has all the required information. (Preconditions, Steps to reproduce, Expected result, Actual result).

  Details

  If the issue has a valid description, the label Issue: Format is valid will be added to the issue automatically. Please, edit issue description if needed, until label Issue: Format is valid appears.

• 2. Verify that issue has a meaningful description and provides enough information to reproduce the issue. If the report is valid, add Issue: Clear Description label to the issue by yourself.

• 3. Add Component: XXXXX label(s) to the ticket, indicating the components it may be related to.

• 4. Verify that the issue is reproducible on 2.3-develop branch

  Details

  - Add the comment @magento-engcom-team give me 2.3-develop instance to deploy test instance on Magento infrastructure.
  - If the issue is reproducible on 2.3-develop branch, please, add the label Reproduced on 2.3.x.
  - If the issue is not reproducible, add your comment that issue is not reproducible and close the issue and stop verification process here!

• 5. Verify that the issue is reproducible on 2.2-develop branch.

  Details

  - Add the comment @magento-engcom-team give me 2.2-develop instance to deploy test instance on Magento infrastructure.
  - If the issue is reproducible on 2.2-develop branch, please add the label Reproduced on 2.2.x

• 6. Add label Issue: Confirmed once verification is complete.

• 7. Make sure that automatic system confirms that report has been added to the backlog.

[ghost]

Hi @carolineboivin thank you for you report. I'm trying reproduce this on clean magento instance but unsuccessful. Can you provide more detail, or maybe this related to some custom modules ?

[carolineboivin]

I could not reproduce on the provided vanilla Magento instance, since it happens when you have a custom theme with a view.xml file setting the "product_image_white_borders" to 0 and then regenerating the images with the CLI.
Although, I was able to circumvent the issue by adding a "0" value to all my problematic images types in my view.xml file. Although, this is redundant and I believe the "product_image_with_borders" var should work as expected.

[ghost]

@carolineboivin I was debugging on custom theme, and actually i set break point, and the value of product_image white borders always be set to 1, but in my config files it was 0.

[magento-engcom-team]

@engcom-backlog-nazar Thank you for verifying the issue. Based on the provided information internal tickets MAGETWO-97097 were created

[navinbhudiya]

@magento-engcom-teamI am using a custom theme and i have upgrade Magento version 2.2.6 to 2.3.0 then i am facing same issue.
Image Cache generation generates wrong has and all image showing 404. any one have solution

[navinbhudiya]

@carolineboivin did you get any solution for issue?

[carolineboivin]

Hi @navinbhudiya, I was able to fix my images with adding <frame>0</frame> to all my images types in my custom view.xml file. This is the only workaround I have found until this issue is solved. Have a nice day!

[Userr017]

@carolineboivin You mean example this? I have still same issue after this.

<image id="category_page_grid" type="small_image">
        <width>580</width>
        <height>720</height>
	<frame>0</frame>
</image>

[carolineboivin]

Yes, I am using the swatches in the listing, so the most important format were not in my view.xml file at first, I copied the values from the module swatches too in my file, maybe you're missing those ?

            <image id="product_swatch_image_large" type="image"/>
            <image id="product_swatch_image_medium" type="image">
                <width>330</width>
                <height>230</height>
                <frame>0</frame>
            </image>
            <image id="product_swatch_image_small" type="thumbnail">
                <width>88</width>
                <height>116</height>
                <frame>0</frame>
            </image>

[pauldonnelly]

I just upgraded from 2.2.6 to 2.3 and I am having the same issue. The product images are using the wrong hash.

Has there been any official fix for this?

Edit:
My images are now showing. I noticed when I ran bin/magento catalog:images:resize an error occurred saying XX images where missing and the regen did not complete. It seems catalog_product_entity_media_gallery had a lot of old deleted products/images (not sure why they were not removed). I removed those and tried again and my images appeared. There was no need for the frame node in view.xml.

[annakoehler]

I have the same issue with white borders and my changes in theme view.xml don't work. It helps me only if I create etc/view.xml in some module (this module has sequence on Magento_Catalog) with
<vars module="Magento_Catalog"> <var name="product_image_white_borders">0</var> </vars>

[Userr017]

I have was same issue. So I restored back to 2.2.7 because this big issue. Hope so someone fix this issue for magento 2.3.0.

[johnny-longneck]

Any new fixes for this? I upgraded from Magento 2.3.1 to 2.4.2 and in live environment all product images are only showing placeholder. Tried all solutions several times.

EDIT: I finally found a "workaround" I can live with.
There are two ways to generate the url of the images, hash or via parameters. I switched to parameters and it worked.

Find the settings under store > settings > configuration > catalog media url format

This is definitly a bug.

[davidduong90]

Hello @johnny-longneck , Please update configure to:
Admin panel > Stores > Settings > Configuration > GENERAL > Web > Url Options > Catalog media URL format

[johnny-longneck]

Hello @johnny-longneck , Please update configure to:
Admin panel > Stores > Settings > Configuration > GENERAL > Web > Url Options > Catalog media URL format

Hello, In my answer, I said this was already my workaround.
Fact is, Catalog media url format with hash mechanism is not working, since it gives wrong hash path and
Catalog media url format with parameters is not optimal since you need a resize mechanism via CDN or webserver.

[davidduong90]

Did you find any other solution?

[VyacheslavCh]

Same issue happened to our store when we migrated to Magento 2.4.2+ . After week of research we found the issue,
vendor\magento\module-media-storage\App\Media.php has wrong parameter in there.

Replace with return preg_replace('|^.*((?:/[^/]+){3})$|', '$1', $resizedImagePath);

[johnny-longneck]

@VyacheslavCh did this resolve the issue totally for you?
In my case it is not working. The original image path is not the problem but the hash path.

Using your option destroys also my category images, before it was just product images.
Seems like the behaviour is really unique.

Really frustrating. We are live with full sized catalog images for more than 4 weeks now.

[Nuranto]

Hello,

I reproduce it too on 2.4.2.

I will try to PR a fix.

[dvershinin]

Fix by @pauldonnelly , or simply running n98-magerun2 catalog:images:resize works fine.

[Den4ik]

@carolineboivin @Nuranto Please follow comment #33439 (comment)

If you still able reproduce issue on fresh 2.4-develop installation please provide step by step instruction

[pmathbliss]

I fixed a 2.2 by commenting out the first is_array of an or in Magento\Catalog\Model\Product;\Image::isCached()

https://github.com/magento/magento2/blob/2.4-develop/app/code/Magento/Catalog/Model/Product/Image.php

Does any one know what isCached means in this context? Does it mean the url/uri is a cached path? The hash will always be a hash since it includes the configurations.
Should the file exists be first?

[razvan-optaros]

@carolineboivin @Nuranto Please follow comment #33439 (comment)

If you still able reproduce issue on fresh 2.4-develop installation please provide step by step instruction

Why this comment has 1 like? because it asked for step by step and then closed this ticket.

If we would not close tickets in this manner, we will have 100.000 bugs in Magento 2.

@Den4ik why did you close this? Is everything ok?

[CadenceLabs-AndrewS]

We resolved our issue on 2.4.3-p1 ( upgraded from 2.3.4 ) by patching the regex function in:
vendor/magento/module-media-storage/App/Media.php
It's missing the case-insensitive flag... 'i' on end of the preg_replace regex

- return preg_replace('|^.*?((?:/([^/])/([^/])/\2\3)?/?[^/]+$)|', '$1', $resizedImagePath);
+ return preg_replace('|^.*?((?:/([^/])/([^/])/\2\3)?/?[^/]+$)|i', '$1', $resizedImagePath);

Our hash issue was resolved with a composer update (make sure your dependencies update)

Images are now loading on our front-end.

[harrigo]

We resolved our issue on 2.4.3-p1 ( upgraded from 2.3.4 ) by patching the regex function in: vendor/magento/module-media-storage/App/Media.php It's missing the case-insensitive flag... 'i' on end of the preg_replace regex

- return preg_replace('|^.*?((?:/([^/])/([^/])/\2\3)?/?[^/]+$)|', '$1', $resizedImagePath); + return preg_replace('|^.*?((?:/([^/])/([^/])/\2\3)?/?[^/]+$)|i', '$1', $resizedImagePath);

Our hash issue was resolved with a composer update (make sure your dependencies update)

Images are now loading on our front-end.

I also have issue however 2.4.3-p2, the above did not resolve my issue unfortunately. For me all my main store have correct images however secondary show placeholder for all generated images.

Tried also changing to "query param" for "Catalog media URL format" as suggested earlier but will need to build that into the webserver or a CDN as just serves original files with params so careful of that one.

[pimruiter]

I also had the same issue on 2.4.3-p1, the image cache URL's were generated wrong. For me it was a wrong Watermark configuration, removing these settings all the cache images were generated right again. Maybe this will help someone.

[miteshdhaduk1]

@pimruiter Thanks...!!! Watermark configuration was causing issues for me in GraphQL images for cart data and products. Currently removing Watermarks images fixes my issue.

Any idea why it's causing it?

[simoncrewe]

My issue was that the resizer doesn't work for products set to not visible individually.
The fix is below.
https://github.com/magento/magento2/blob/2.4-develop/app/code/Magento/MediaStorage/Service/ImageResize.php#L183

[reniraj]

Same issue happened to our store when we migrated to Magento 2.4.2+ . After week of research we found the issue, vendor\magento\module-media-storage\App\Media.php has wrong parameter in there.

Replace with return preg_replace('|^.*((?:/[^/]+){3})$|', '$1', $resizedImagePath);

This is for EE ?

[mdiab1996]

After a lot of debugging , I found that .htaccess was changed in pub/media ( this file is very import because each not found image forwarded to pub/get.php to generate the cache image immediately if its not found on the cache , so i just restored the .htaccess file and my problem was solved .

if your problem still not solved you have to make sure that the get.php is called so you can add echo "test image"; exit; At the beginning of the file after this open any not found image on the frontend ; if get.php was called correctly the output must be "test image" if not then you problem is get.php is not called.

Please note that if the imges still not appear and get.php called correctly you can debug get.php and add a echo test on each step to find what exactly is the problem , maybe its a permissions issue.

[Episodio1]

@mdiab1996
I can't test it because I already deleted that project from my computer. But your solution makes a lot of sense, and not the previously posted workarounds.
Thank you!

[paradoxos]

After a lot of debugging , I found that .htaccess was changed in pub/media ( this file is very import because each not found image forwarded to pub/get.php to generate the cache image immediately if its not found on the cache , so i just restored the .htaccess file and my problem was solved .

if your problem still not solved you have to make sure that the get.php is called so you can add echo "test image"; exit; At the beginning of the file after this open any not found image on the frontend ; if get.php was called correctly the output must be "test image" if not then you problem is get.php is not called.

Please note that if the imges still not appear and get.php called correctly you can debug get.php and add a echo test on each step to find what exactly is the problem , maybe its a permissions issue.

After trying everything above mdiab1996 was right the .htaccess was broken
thanks

[hatemdeeb]

This issue still exists in M 2.4.6
The problem originates from
\Magento\Catalog\Model\Product\Image\ParamsBuilder::build()
When it checks for hashing parameters for generating resized images it behaves wrongly for different themes that have different view.xml configurations
For me, it seems like this function particularly causes the issue:
\Magento\Catalog\Model\Product\Image\ParamsBuilder::overwriteDefaultValues
on this line
$frame = $imageArguments['frame'] ?? $this->hasDefaultFrame();

The workaround is to explicitly specify <frame>0</frame> in all custom themes view.xml files.
Need more time to debug and determine what fix need to be applied.