[4.2.2]: Duplicate image transformation with filename "null"

[kringkaste]

What happened?

Description

We were wondering why the job GeneratePendingTransforms processes an increasing number of transformations over time. We have checked the transformation table in the database and found that duplicate entries are created in the database almost simultaneously for the same transformationString and assetId, whose only difference is that one entry have null as filename. But this does not always happen.

Example:

+-------+---------+----------------------------------------+---------------------------------------------------+--------+---------------------------------+------------+------------+-------+-------------+---------------------+---------------------+--------------------------------------+
| id    | assetId | transformer                            | filename                                          | format | transformString                 | fileExists | inProgress | error | dateIndexed | dateCreated         | dateUpdated         | uid                                  |
+-------+---------+----------------------------------------+---------------------------------------------------+--------+---------------------------------+------------+------------+-------+-------------+---------------------+---------------------+--------------------------------------+
| 72057 | 9201783 | craft\imagetransforms\ImageTransformer | banner-Symbolbild-Schlagloch-imago0150089693h.jpg | NULL   | _120x33_crop_center-center_none |          1 |          0 |     0 | NULL        | 2022-08-30 08:10:39 | 2022-08-30 08:12:11 | c0fdac71-7013-4caa-88f9-d05a6fb171dd |
| 72058 | 9201783 | craft\imagetransforms\ImageTransformer | NULL                                              | NULL   | _120x33_crop_center-center_none |          0 |          0 |     0 | NULL        | 2022-08-30 08:10:39 | 2022-08-30 08:10:39 | 433c510b-d20a-4eb6-9a9e-4d49c49a9646 |
| 72059 | 9201783 | craft\imagetransforms\ImageTransformer | banner-Symbolbild-Schlagloch-imago0150089693h.jpg | NULL   | _240x66_crop_center-center_none |          1 |          0 |     0 | NULL        | 2022-08-30 08:10:39 | 2022-08-30 08:13:52 | d5ae04c6-8ed5-46ca-a9a3-1d94dd5d14dd |
| 70973 | 9201783 | craft\imagetransforms\ImageTransformer | banner-Symbolbild-Schlagloch-imago0150089693h.jpg | NULL   | _34x9_crop_center-center_none   |          1 |          0 |     0 | NULL        | 2022-08-30 08:09:27 | 2022-08-30 08:11:08 | 8884b1bf-a144-48b2-b274-00fd938e9786 |
| 70974 | 9201783 | craft\imagetransforms\ImageTransformer | banner-Symbolbild-Schlagloch-imago0150089693h.jpg | NULL   | _68x18_crop_center-center_none  |          1 |          0 |     0 | NULL        | 2022-08-30 08:09:27 | 2022-08-30 08:11:09 | dc9a37fb-bfbb-4e66-9bfd-a6900f14ba1b |
| 70975 | 9201783 | craft\imagetransforms\ImageTransformer | NULL                                              | NULL   | _68x18_crop_center-center_none  |          0 |          0 |     0 | NULL        | 2022-08-30 08:09:27 | 2022-08-30 08:09:27 | 8d3f8842-6c61-4462-a811-f68cef163e63 |
|  1628 | 9201783 | craft\imagetransforms\ImageTransformer | banner-Symbolbild-Schlagloch-imago0150089693h.jpg | NULL   | _banner                         |          1 |          0 |     0 | NULL        | 2022-08-29 21:25:54 | 2022-08-29 21:25:55 | 1c633a16-c7d3-4b05-8b27-349d640721c7 |
|  2395 | 9201783 | craft\imagetransforms\ImageTransformer | banner-Symbolbild-Schlagloch-imago0150089693h.jpg | NULL   | _teaserListThumb                |          1 |          0 |     0 | NULL        | 2022-08-29 21:29:21 | 2022-08-29 21:29:22 | 22f585c7-8adb-4056-88ba-cb1386382909 |
+-------+---------+----------------------------------------+---------------------------------------------------+--------+---------------------------------+------------+------------+-------+-------------+---------------------+---------------------+--------------------------------------+

The function getTransformIndex finds and returns the already existing transformation. But the wrong transformation will not be deleted, so on the next run, the transformation will be processed again.

But we think the real problem is the transformation with the filename with null. We have not been able to trace where these entries are generated and why.

Craft CMS version

4.2.2

PHP version

8.1.7

Operating system and version

Linux 5.13.0-1031-aws

Database type and version

MySQL 5.7.38

Image driver and version

Imagick 3.7.0 (ImageMagick 6.9.10-23)

Installed plugins and versions

[brandonkelly]

Thanks for reporting that! We just released Craft 4.2.4 which we believe will resolve this going forward (#11863). Let us know if you continue to see it, though.

[kringkaste]

Hi Brandon, we upgraded another customer directly from 3.7 to 4.2.4 and deleted the whole image transform index of the site before the migration. Now, after some days, we do have the same issue here: About 200 transformation rows in the imagetransforminx with filename = null. And not in every case a correct row in the table exists. We just deleted these rows now and see if they will popup again.

[brianjhanson]

@kringkaste it's worth noting that filename is expected to be null while the transform is being created. Do the rows with filename = null have any usual values in the fileExists, inProgress or error columns?

[kringkaste]

@brianjhanson Here is an example of the customer:

| 222241 |  136032 | craft\imagetransforms\ImageTransformer | NULL     | NULL   | _68x38_crop_center-center_none   |          0 |          0 |     0 | NULL        | 2022-09-19 09:13:35 | 2022-09-19 09:13:35 | e7d64765-7852-4e4c-b078-3919083e14dd |
| 222243 |  142212 | craft\imagetransforms\ImageTransformer | NULL     | NULL   | _34x19_crop_center-center_none   |          0 |          0 |     0 | NULL        | 2022-09-19 09:13:35 | 2022-09-19 09:13:35 | 31d62d05-b3f8-45fc-b2a1-c8ff293607d8 |
| 222245 |  142212 | craft\imagetransforms\ImageTransformer | NULL     | NULL   | _68x38_crop_center-center_none   |          0 |          0 |     0 | NULL        | 2022-09-19 09:13:35 | 2022-09-19 09:13:35 | 142ab57c-7be3-4f6f-ac02-0f423025e38e |
| 222248 |  147165 | craft\imagetransforms\ImageTransformer | NULL     | NULL   | _68x38_crop_center-center_none   |          0 |          0 |     0 | NULL        | 2022-09-19 09:13:35 | 2022-09-19 09:13:35 | 642a8466-da2d-49f5-97b8-72d5f7aa051c |
| 222251 |  161262 | craft\imagetransforms\ImageTransformer | NULL     | NULL   | _68x38_crop_center-center_none   |          0 |          0 |     0 | NULL        | 2022-09-19 09:13:35 | 2022-09-19 09:13:35 | 1c22cefe-76cc-44e4-a371-6b60d6862547 |

fileExists, inProgress and error are always false.

[brianjhanson]

Are there any "Generating Transforms" jobs stuck in the queue by chance? Or are there any errors in the storage/logs files that might be related to this?

[kringkaste]

No errors in log, no stuck jobs in the queue :( But these rows are from a customer still running the 4.2.2 version. I deleted the rows from the 4.2.4 installation, so we have to wait for new ones.

[kringkaste]

@brianjhanson I think the original bug is fixed with the update to 4.2.4. We could no longer find duplicate rows with filename = null and filename filled in. The other seems to be the normal behavior: The pending transformations are created without filename and when the job has run the filename is updated. We will update the other customer also to 4.2.4 and I think this will eliminate these duplicate rows as well.

Another thing I added to the feature request: The job should ignore image transformations already failed with an error #11970.