[4.x]: Installing plugins via CP fails

[realjoshharrison]

What happened?

Description

Installing plugins via the CP's plugin store is failing with the following error:

A fatal error has occurred:

Status: undefined
Response: undefined

Steps to reproduce

1. Clean install of Craft (I'm using MAMP Pro)
2. Go to plugin store and attempt to install something. I tried with CKEditor, and experienced the same with Redactor too.

Expected behavior

The plugin installs as usual.

Actual behavior

The above error message is shown, the attempted plugin is added to composer.json's require list, but the files are not present in vendor. On the above error message page, the dev console shows:

:8890/index.php?p=craft%2Factions%2Fpluginstore%2Finstall%2Fcomposer-install&v=1694448094397:1
Failed to load resource: the server responded with a status of 500 (Internal Server Error)

After this occurs, all requests fail to load with 500 Internal Server Error pages (in both CP and front end) until I restart the server.

I can't see any errors in craft's logs, Apache logs, or PHP logs... the last entry in Craft's web.log is a seemingly successful web.INFO request for admin/actions/pluginstore/install/precheck.

Initially this seemed like it might be something to do with my server setup, but the odd thing is I hadn't changed anything when I first experienced this error today. I since updated my system composer and MAMP Pro to the latest versions, but still get the same errors.

Everything I'd think was relevant checks out in the System Report's requirements list. The only warnings are mysql timezone support, @web is not explicitly overwritten, and no OPcache.

The same operations work fine when using e.g. composer require craftcms/ckeditor -w && php craft plugin/install ckeditor directly on the command line. It's only via the CP where this occurs.

Thanks for your help with this. Please let me know if I can provide any specific files.

Craft CMS version

4.5.3

PHP version

8.2.0

Operating system and version

MAMP Pro 6.8.1, Darwin 22.6.0 (MacOS 13.5.1)

Database type and version

MySQL 5.7.39

Image driver and version

Imagick 3.7.0 (ImageMagick 6.9.6-2)

Installed plugins and versions

• None

[brandonkelly]

Just tested installing CKEditor from the CP using MAMP Pro 6.8.1 and it’s working on both Craft 3.9.1 and 4.5.3.

Can you share your composer.json and composer.lock files from before installing CKEditor?

[realjoshharrison]

Hi Brandon, thanks so much for taking a look.

Strange! Sure, here they are. It's literally a clean install created using composer create-project craftcms/craft and set up using the CLI wizard.

And here they are after the error occurs if that's useful. You can see craftcms/ckeditor was added to composer.json but there's nothing about it in composer.lock (and nothing for it in vendor either).

The last entry in web.log is the request to admin/actions/pluginstore/install/precheck with nothing further logged after that...

[realjoshharrison]

I've solved part of the mystery after noticing the undefined error was always occurring after exactly 30 seconds of "Updating Composer dependencies (this may take a minute)…". Resolved that timeout by editing MAMP's File > Open Template > Apache > httpd.conf, and replacing the line:

MAMP_FastCgiServer_MAMP

with

MAMP_FastCgiServer_MAMP -idle-timeout 3600

as per https://stackoverflow.com/questions/76079982/how-to-stop-30s-timeouts-in-mamp

The undefined error doesn't occur after 30 seconds now. However, "Updating Composer dependencies (this may take a minute)…" runs for a very long time. Just left it for 6 mins and it's still "Updating Composer dependencies"... still nothing new in web.log, php error log, or apache error log.

PHP's max_execution_time is set to 300.

I don't know if this is relevant, but all web requests on the site stop responding after the install attempt, and as soon as I stop the servers in MAMP, the composer.json file reverts to its state as before the update was attempted (i.e. craftcms/ckeditor disappears again from composer.json).

[realjoshharrison]

I've just found that this issue also applies to updating the CMS via the control panel. I guess any composer operation via the CP is getting stuck on "Updating Composer dependencies".

I've narrowed down the Craft update where this issue began to occur on my environment.

On a fresh Craft 4.4.17 install, tested just now, I can still install plugins via the CP, and update Craft via the CP too.

On a fresh Craft 4.5.0 install, the issue occurs.

Perhaps this is somehow related to switching to the bundled composer.phar in 4.5.0?

I realise you can't replicate this on your environment... so please let me know if there's anything I can check on for you – file permissions etc?

Clutching at straws here, but in MAMP Pro under 'Ports & User' I am running the servers as my own username rather than Unix users if that makes any difference...

[brandonkelly]

Do Composer operations take longer than 30 seconds on 4.4.17 for you?

In general they’re always taking under 10 seconds on my end, which is probably why I’m not affected by the 30 second default timeout.

[realjoshharrison]

No, they’d typically only take the usual 5-10 seconds. 4.4.17 updated to the latest version via the CP in about 10 seconds.

[angrybrad]

@realjoshharrison Can you put a phpinfo.php in your public web folder with these contents:

<?php phpinfo();

Then load yoursite.com/phpinfo.php in a browser and share the results here as a large image/PDF/etc.

[cloudgrayau]

I am not entirely sure if this is related, but I just received the same "undefined" response when updating plugins in the CP using MAMP Pro. Looking at the logs, it appeared to be a "memory" issue -

2023-09-18 00:21:59 [web.ERROR] [yii\web\HttpException:400] yii\web\BadRequestHttpException in /Users/cloudgray/Sites/cloudgray/vendor/craftcms/cms/src/controllers/BaseUpdaterController.php:72
Stack trace:
#0 /Users/cloudgray/Sites/cloudgray/vendor/craftcms/cms/src/controllers/UpdaterController.php(41): craft\controllers\BaseUpdaterController->beforeAction(Object(yii\base\InlineAction))
#1 /Users/cloudgray/Sites/cloudgray/vendor/yiisoft/yii2/base/Controller.php(176): craft\controllers\UpdaterController->beforeAction(Object(yii\base\InlineAction))
#2 /Users/cloudgray/Sites/cloudgray/vendor/yiisoft/yii2/base/Module.php(552): yii\base\Controller->runAction('finish', Array)
#3 /Users/cloudgray/Sites/cloudgray/vendor/craftcms/cms/src/web/Application.php(304): yii\base\Module->runAction('updater/finish', Array)
#4 /Users/cloudgray/Sites/cloudgray/vendor/craftcms/cms/src/web/Application.php(607): craft\web\Application->runAction('updater/finish', Array)
#5 /Users/cloudgray/Sites/cloudgray/vendor/craftcms/cms/src/web/Application.php(283): craft\web\Application->_processActionRequest(Object(craft\web\Request))
#6 /Users/cloudgray/Sites/cloudgray/vendor/yiisoft/yii2/base/Application.php(384): craft\web\Application->handleRequest(Object(craft\web\Request))
#7 /Users/cloudgray/Sites/cloudgray/web/index.php(12): yii\base\Application->run()
#8 {main} {"memory":32528656,"exception":"[object] (yii\\web\\BadRequestHttpException(code: 0):  at /Users/cloudgray/Sites/cloudgray/vendor/craftcms/cms/src/controllers/BaseUpdaterController.php:72)"} 
2023-09-18 00:21:59 [web.INFO] [application] Request context:
$_GET = [
    'p' => 'backdoor/actions/updater/finish'
    'site' => 'cloudgray'
    'v' => '1694996516535'
]

I've since updated my PHP memory. I will advise if it happens again.
Whilst the updates do succeed for me (composer is updated successfully), it just appears to fail on the finish command.

[realjoshharrison]

Thanks @angrybrad – here's a phpinfo.

And this log file shows everything from when I click Update in admin/utilities/updates, to when I shut down the MAMP servers after a few minutes of Updating composer dependencies. The signal 15 error and everything after it is only logged when I shut down the servers so not sure they are related to the actual issue.

MAMP's PHP has 512MB memory, so I'm not sure that would be the issue in my case...

[angrybrad]

@realjoshharrison huh... nothing really jumping out at me in either of those.

Can you try a few things to see if the behavior changes?

• Swap from PHP 8.2 to PHP 8.1
• Swap from MAMP's CGI mode to Module mode (it looks like you're currently on CGI mode)

[realjoshharrison]

Good ideas... module mode (still with PHP 8.2) worked! 😀

I was able to install a plugin, and update Craft, via the CP.

Based on this I've found a local fix that works on CGI mode.

I see that Symfony\Component\Process\PhpExecutableFinder's first port of call is to look for PHP_BINARY.

Without setting anything in .env, PHP_BINARY returns /Applications/MAMP/bin/php/php8.2.0/bin/php-cgi. So something with that particular one must not be playing nicely.

I just tried setting PHP_BINARY=/Applications/MAMP/bin/php/php8.2.0/bin/php in .env, and the composer operations then worked, even on CGI mode. So I can just do this for all my local projects.

If I set PHP_BINARY=/opt/homebrew/bin/php – the output of which php in my terminal – the operations also work.

Not sure if there's a better solution, but I'm happy to set PHP_BINARY for my local environments for now. If you'd like me to try anything else out let me know!

[stanislavprokopov]

After updating to craft 4.5 I cannot update any other plugin from CP:

Error: The command "'/usr/lib64/php8.1/bin/php-cgi' '/data01/domains/www/craft/storage/runtime/composer.phar' 'update' 'craftcms/feed-me' '--with-all-dependencies' '--working-dir' '/data01/domains/www/craft' '--no-scripts' '--no-ansi' '--no-interaction'" failed.

Exit Code: 255(Unknown error)

Working directory: /data01/domains/www/acc

Output:
================
<b>Security Alert!</b> The PHP CGI cannot be accessed directly.

<p>This PHP CGI binary was compiled with force-cgi-redirect enabled. This
means that a page will only be served up if the REDIRECT_STATUS CGI variable is
set, e.g. via an Apache Action directive.</p>
<p>For more information as to <i>why</i> this behaviour exists, see the <a href="http://php.net/security.cgi-bin">manual page for CGI security</a>.</p>
<p>For more information about changing this behaviour or re-enabling this webserver,
consult the installation file that came with this distribution, or visit
<a href="http://php.net/install.windows">the manual page</a>.</p>

Error Output:
================

Composer output: Security Alert! The PHP CGI cannot be accessed directly.

This PHP CGI binary was compiled with force-cgi-redirect enabled. This
means that a page will only be served up if the REDIRECT_STATUS CGI variable is
set, e.g. via an Apache Action directive.
For more information as to why this behaviour exists, see the manual page for CGI security.
For more information about changing this behaviour or re-enabling this webserver,
consult the installation file that came with this distribution, or visit
the manual page.

Before the update everything worked and I can ssh into the machine and run composer update manually without any issues.

[angrybrad]

@realjoshharrison

Without setting anything in .env, PHP_BINARY returns /Applications/MAMP/bin/php/php8.2.0/bin/php-cgi. So something with that particular one must not be playing nicely.

php-cgi is meant to be accessed through web requests, and php through CLI requests. Even though you're updating through the Craft control panel in a browser, we're using Symfony's Process component to execute composer.phar as a CLI request. Which would explain the error your getting and the workaround you found.

By setting MAMP to “module” mode, I think you’re effectively bypassing php-cgi altogether.

@stanislavprokopov basically the same issue, except your fix would be to set a PHP_BINARY env var to /usr/lib64/php8.1/bin/php

[realjoshharrison]

Thanks for the explanation @angrybrad, that makes sense.

So it sounds like any environments using the php-cgi binary for web requests won’t be able to use any CP operations that need to execute CLI requests, such as plugin installations and Craft updates, because Symfony’s PhpExecutableFinder is going to return the unusable web binary on these environments? And the workaround is to explicitly set the path to any viable php binary in .env.

I’m ok to continue with that myself, but I wonder if it would be worth documenting somewhere, or perhaps checking in the CMS whether php-cgi is going to be used for any CP initiated CLI operations, and throwing an early error?

[stanislavprokopov]

Setting PHP_BINARY as proposed by @angrybrad resolved my issue, thx.

[brandonkelly]

We’ve made a couple changes for the next Craft 3 and 4 releases, which should help:

• $_SERVER['PHP_BINARY'] is now respected.
• If $_SERVER['PHP_BINARY']/getenv('PHP_BINARY') and getenv('PHP_BINARY') are both empty, we will check if PHP is running via FastCGI, and if the PHP_BINARY PHP constant ends in php-cgi. If so, we’ll check for a php executable in the same directory, and use that if so.

[eugenebroeren]

@brandonkelly, Tommy pointed to this thread and asked if i could add PHP_BINARY.
Here some add information in this thread

I've got the same problem

• Updating in CraftCMS UI does not work anymore after ~4.5.x or 3.9.1 (and installing new apps from the store)

Error (running on Plesk)
//
The command "'php' '/craft/storage/runtime/composer.phar' 'update' 'craftcms/cms' '--with-all-dependencies' '--working-dir' '/var/www/vhosts//craft' '--no-scripts' '--no-ansi' '--no-interaction'" failed.
//

• Adding PHP_BINARY = /opt/plesk/php/8.1/bin/php in .env does something but then you get an opendir failure.

• Removing PHP_BINARY in .env and change the php opendir setting in Plesk to :

  {WEBSPACEROOT}{/}{:}{TMP}{/}:/opt/plesk/php/8.1/bin/php
  (adding :/opt/plesk/php/8.1/bin/php )

  ... does work. I don't know if that's a good fix / workaround.

So basically the CraftCMS UI hasn't got a clue where to find the php command anymore after > 4.5.X
Restoring an older project < 4.5 is working fine.

I hope it can be fixed in the next release.

I can update with CLI, that's is not a problem.

[brandonkelly]

So basically the CraftCMS UI hasn't got a clue where to find the php command anymore after > 4.5.X

It’s more that, Craft < 3.9/4.5 didn’t need to know the path to the appropriate php executable in the first place, because they were performing Composer operations directly within Craft web requests.

Unfortunately, that’s not how Composer was designed to work, and Composer 2.3+ made it technically impossible to continue using it that way. So we could either continue shipping an old, unmaintained copy of Composer with Craft, with an increasing number of security holes and incompatibilities (#13396, #13441), or switch to bundling a composer.phar file and executing that via a subprocess.

This issue aside, option 2 is a clear improvement, so we aren’t going to be rolling it back. We’ll just have to keep working to improve support for different environments/configs.

[eugenebroeren]

This issue aside, option 2 is a clear improvement, so we aren’t going to be rolling it back. We’ll just have to keep working to improve support for different environments/configs.

Thanks and clear

1. Can you confirm that (in my case) changing this in the opendir setting in Plesk is a fine solution (security) ?
2. Maybe i can make a symbolic link somewhere in craft to /opt/plesk/php/8.1/bin/php or link a composer.phar (as a general solution for other users ) ?

///////////////////
{WEBSPACEROOT}{/}{:}{TMP}{/}:/opt/plesk/php/8.1/bin/php
(adding :/opt/plesk/php/8.1/bin/php )

... does work. I don't know if that's a good fix / workaround.
///////////////////

[angrybrad]

@eugenebroeren I think that workaround is fine. It's basically saying that the web version of PHP (php-cgi), can access your website's root path, the system temp folder, and the CLI version of PHP that lives at /opt/plesk/php/8.1/bin/php.

The slightly inconvenient part is if you upgrade this site to PHP 8.2 in the future, you'll need to remember to update that path.

I'm pretty sure that open_basedir takes into account symlinks as well.

[brandonkelly]

Craft 3.9.4 and 4.5.6 are out with those improvements.

This should solve Composer operations for MAMP on CGI mode. No PHP_BINARY environment variable or other configuration needed.

[realjoshharrison]

Looks like a great solution and should work on my MAMP environment. Not sure off the top of my head whether MAMP sets $_SERVER['PHP_BINARY'] but I do know the php binary is in the same directory as php-cgi so that’ll catch it. Thanks for the fix! ❤️

[brandonkelly]

We tested it in MAMP + CGI and confirmed it works :)

[eugenebroeren]

I tried 4.5.6 and in my case (Plesk env) i still getting "cant find php" (installing a package and perhaps updating packages in craft ui)

The command "'php' '/craft/storage/runtime/composer.phar' 'update' 'doublesecretagency/craft-cpcss' '--with-all-dependencies' '--working-dir' '/craft' '--no-scripts' '--no-ansi' '--no-interaction'" failed.
sh: line 0: exec: php: not found

So for now my solution is this :
Php settings ->openbasedir {WEBSPACEROOT}{/}{:}{TMP}{/}:/opt/plesk/php/8.1/bin/php

or always update / install via CLI (that did not break)

[brandonkelly]

Yeah if the command was using 'php' as the executable, that means PhpExecutableFinder::find() couldn’t find a PHP executable candidate it liked, so Craft is just doing a hail mary, hoping there’s a working php executable in the PATH.

[eugenebroeren]

@brandonkelly

Ah, I see what goes wrong due to PhpExecutableFinder (in my case), so i'll stick to adding php in openbase php :

The problem is if (!is_executable($php)) {
The php cmd is_exacutable returns the false because of openbase.

You can set PHP_BINARY="/opt/plesk/php/8.1/bin/php" in .env (or PHP_PATH)
If you return the PHP_BINARY of PHP_PATH directly (without is_is_executable) , it is working as before.

############
public function find(bool $includeArgs = true): string|false
{
return getenv('PHP_BINARY') ; // THIS WORKS

    if ($php = getenv('PHP_BINARY')) {

.....
###################

[angrybrad]

The problem is if (!is_executable($php)) {
The php cmd is_exacutable returns the false because of openbase.

If I'm following you, and is_executable returns false because it is out of the limits of what is in open_basedir, then that seems like a reasonable response and expected behavior to me.

[eugenebroeren]

If I'm following you, and is_executable returns false because it is out of the limits of what is in open_basedir, then that seems like a reasonable response and expected behavior to me.

Correct, but the strange thing is that when you just return $PHP_BINARY (/opt/plesk/php/8.1/bin/php) or return "/opt/plesk/php/8.1/bin/php" in the first line of of PhpExecutableFinder everything is fine and no need for adding /opt/plesk/php/8.1/bin/php in php settings openbasedir.

Best option in my case would be if $PHP_BINARY (or another var $PHP_DIRECT_PATH for example) is set in .env then simply return it en do not launch PhpExecutableFinder. I don't know of more people running into this.