Asset composer.lock is invalid...

[drupol]

Hello,

Today I investigated how to reproduce correctly psysh from the sources given the composer.lock file you're providing in the release asset (this is an outcome of our discussion in #767).

I succeeded to reproduce an issue locally with simple PHP and Composer commands, here we go:

Shell session

❯ git clone [email protected]:bobthecow/psysh.git
Cloning into 'psysh'...
remote: Enumerating objects: 17102, done.
remote: Counting objects: 100% (3804/3804), done.
remote: Compressing objects: 100% (1155/1155), done.
remote: Total 17102 (delta 2851), reused 3452 (delta 2612), pack-reused 13298
Receiving objects: 100% (17102/17102), 3.98 MiB | 2.74 MiB/s, done.
Resolving deltas: 100% (11683/11683), done.

❯ cd psysh/

❯ php --version
PHP 8.1.29 (cli) (built: Jun  5 2024 05:51:57) (NTS)
Copyright (c) The PHP Group
Zend Engine v4.1.29, Copyright (c) Zend Technologies
    with Zend OPcache v8.1.29, Copyright (c), by Zend Technologies

❯ git checkout v0.12.4
Note: switching to 'v0.12.4'.
HEAD is now at 2fd717af Merge branch 'release/0.12.4'

❯ wget https://github.com/bobthecow/psysh/releases/download/v0.12.4/composer-v0.12.4.lock
--2024-07-01 17:18:20--  https://github.com/bobthecow/psysh/releases/download/v0.12.4/composer-v0.12.4.lock
Resolving github.com (github.com)... 140.82.121.4
Connecting to github.com (github.com)|140.82.121.4|:443... connected.
HTTP request sent, awaiting response... 302 Found
Location: https://objects.githubusercontent.com/github-production-release-asset-2e65be/4549925/86a8224a-9c62-4999-aa9e-62450abf9f12?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=releaseassetproduction%2F20240701%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Date=20240701T151820Z&X-Amz-Expires=300&X-Amz-Signature=1d55b836ce08fa356802b23cb7f9ad4f54de8aaee93214f13c87a423a49b0837&X-Amz-SignedHeaders=host&actor_id=0&key_id=0&repo_id=4549925&response-content-disposition=attachment%3B%20filename%3Dcomposer-v0.12.4.lock&response-content-type=application%2Foctet-stream [following]
--2024-07-01 17:18:20--  https://objects.githubusercontent.com/github-production-release-asset-2e65be/4549925/86a8224a-9c62-4999-aa9e-62450abf9f12?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=releaseassetproduction%2F20240701%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Date=20240701T151820Z&X-Amz-Expires=300&X-Amz-Signature=1d55b836ce08fa356802b23cb7f9ad4f54de8aaee93214f13c87a423a49b0837&X-Amz-SignedHeaders=host&actor_id=0&key_id=0&repo_id=4549925&response-content-disposition=attachment%3B%20filename%3Dcomposer-v0.12.4.lock&response-content-type=application%2Foctet-stream
Resolving objects.githubusercontent.com (objects.githubusercontent.com)... 185.199.109.133, 185.199.111.133, 185.199.110.133, ...
Connecting to objects.githubusercontent.com (objects.githubusercontent.com)|185.199.109.133|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 87637 (86K) [application/octet-stream]
Saving to: ‘composer-v0.12.4.lock’

2024-07-01 17:18:21 (3,06 MB/s) - ‘composer-v0.12.4.lock’ saved [87637/87637]

❯ mv composer-v0.12.4.lock composer.lock

❯ ll
drwxr-xr-x    - pol  1 jui 17:17 bin
.rw-r--r--  295 pol  1 jui 17:17 box.json.dist
.rw-r--r--  159 pol  1 jui 17:17 codecov.yml
.rw-r--r-- 1,7k pol  1 jui 17:17 composer.json
.rw-r--r--  88k pol 10 jun 03:19 composer.lock
.rw-r--r-- 1,1k pol  1 jui 17:17 LICENSE
.rw-r--r-- 2,9k pol  1 jui 17:17 Makefile
.rw-r--r--  576 pol  1 jui 17:17 phpstan.neon.dist
.rw-r--r--  373 pol  1 jui 17:17 phpunit.xml.dist
.rw-r--r-- 2,4k pol  1 jui 17:17 README.md
.rw-r--r-- 5,4k pol  1 jui 17:17 scoper.inc.php
drwxr-xr-x    - pol  1 jui 17:17 src
drwxr-xr-x    - pol  1 jui 17:17 test
drwxr-xr-x    - pol  1 jui 17:17 vendor-bin

❯ composer validate --strict
./composer.json is valid but your composer.lock has some errors
# Lock file errors
- The lock file is not up to date with the latest changes in composer.json, it is recommended that you run `composer update` or `composer update <package name>`.

❯ composer require --no-update symfony/polyfill-iconv symfony/polyfill-mbstring
Using version ^1.30 for symfony/polyfill-iconv
Using version ^1.30 for symfony/polyfill-mbstring
./composer.json has been updated

❯  composer require --no-update --dev roave/security-advisories:dev-latest
./composer.json has been updated

❯ composer update --lock --no-install
Loading composer repositories with package information
Updating dependencies
Lock file operations: 0 installs, 0 updates, 0 removals
Writing lock file
13 packages you are using are looking for funding.
Use the `composer fund` command to find out more!
No security vulnerability advisories found.

❯ composer install
Installing dependencies from lock file (including require-dev)
Verifying lock file contents can be installed on current platform.
- Required package "symfony/polyfill-iconv" is in the lock file as "v1.29.0" but that does not satisfy your constraint "^1.30".
- Required package "symfony/polyfill-mbstring" is in the lock file as "v1.29.0" but that does not satisfy your constraint "^1.30".
This usually happens when composer files are incorrectly merged or the composer.json file is manually edited.
Read more about correctly resolving merge conflicts https://getcomposer.org/doc/articles/resolving-merge-conflicts.md
and prefer using the "require" command over editing the composer.json file directly https://getcomposer.org/doc/03-cli.md#require-r

~/C/t/psysh > HEAD > [email protected] +5 -2 [!] > [email protected] ✘    

Basically, the issue comes from the fact that during the Makefile step build/psysh, you build a composer.lock with the very latest version of symfony/polyfill-iconv and symfony/polyfill-mbstring. At the time of creating the composer.lock of version 0.12.4, it was 1.29.0.

Turns out that new releases of those two packages were released in the meantime, so reproducing the package locally with your composer.lock is going to fail.

To fix this, I would suggest fixing the version of symfony/polyfill-iconv and symfony/polyfill-mbstring in the Makefile or simply add them to composer.json.

Can you please advise on this and let me know what you think?

IMHO, we should at all cost modify the composer.* files during the CI steps, this is really making things harder when it comes to reproduce the package.

[bobthecow]

Yeah, I can see where that would break things.

They're not in composer.json (and they're added during CI) because they're not dependencies of PsySH (the library), they're dependencies of psysh (the compiled binary).

I've thought a bit about this in the past, but i don't love any of the options i've come up with:

• Adding them to package composer.json means people who don't care about the binary will now have unnecessary and possibly incompatible dependencies installed when they install the library.
• Fixing the version in the Makefile means not only do we have weird dependencies that only affect CI, we have weird dependency versions that only affect CI.
• Adding a derived sub-project composer.json isn't supported by Composer.
• Splitting the entire binary off into its own repo would solve 👆 but introduces other issues. Cross-repo dependencies, losing the ability to test library + phar hermetically against HEAD, etc.