Passenger can't start Rails app with Forest

[Lukasz-Ziemnicki]

Expected behavior

Start Rails 5 application.

Actual behavior

Passenger does not boot Rails 5 app.

Failure Logs

[ E 2020-12-29 00:42:39.0632 21407/T1c age/Cor/App/Implementation.cpp:305 ]: Could not spawn process for application /var/www/app/current: An error occurred while starting up the preloader. It sent an unknown response type "socket: unix:/tmp/passenger.G06gso5/apps.s/preloader.1bwmyct\n". Error ID: a1fe365c Error details saved to: /tmp/passenger-error-xm0eQ3.html Message from application: An error occurred while starting up the preloader. It sent an unknown response type &quot;socket: unix:/tmp/passenger.G06gso5/apps.s/preloader.1bwmyct\n&quot;.<br> Raw process output: ... [ E 2020-12-29 00:42:39.1126 21407/Tf age/Cor/Con/CheckoutSession.cpp:285 ]: [Client 5-2] Cannot checkout session because a spawning error occurred. The identifier of the error is a1fe365c. Please see earlier logs for details about the error. client_loop: send disconnect: Broken pipe

Context

• Phusion Passenger: 5.0.29
• Package Version: forest_liana gem -> 4
• Rails Version: 5.2.2
• Database Dialect: -
• Database Version:

What it is interesting everything worked fine till yesterday. After removing Forest Liana from Gemfile (and initializers) Passenger correctly boots my Rails app. No new code was released neither DevOps work on server made. It's sudden issue.

[arnaudbesnier]

Hi @Lukasz-Ziemnicki,

Let me try to understand:

What it is interesting everything worked fine till yesterday.

Do you mean everything was working fine with the Forest Liana integrated properly in your Rails server?

It's sudden issue.

What was the last operation or contributions done prior to the sudden issue?

Thanks for your help 🙏

[Lukasz-Ziemnicki]

@arnaudbesnier Thank you for answer.

Do you mean everything was working fine with the Forest Liana integrated properly in your Rails server?

Yes, everything worked fined.

What was the last operation or contributions done prior to the sudden issue?

We did absolutely nothing. No Rails app deploys performed nor DevOps works made. We have none shell scripts set up.
One thing I am aware of is that database backup service provided by our server deliverer runs in background. But backups where made in the past and we did not report any issues.

---

Similar problem is reported here: phusion/passenger#2026

We see the same message in logs as in above issue:

App 6416 stdout: App 6416 stdout: [2020-12-28 12:04:13] Forest 🌳🌳🌳 A newer version of Forest liana exists, you should upgrade for a better experience. Run the following command to upgrade: bundle update --source forest_liana App 6416 stdout: !> Ready
And afterwards error is displayed and app can't boot.

Do you think that output message from Forest can crash Passenger? Maybe there is possiblity to remove that message somehow?

[rrodrigu3z]

Hi @Lukasz-Ziemnicki @arnaudbesnier

I can confirm this too. Passenger failed to start suddenly. We simply redeployed the instances and didn't start, so we added as a temporary solution the ENV var FOREST_DISABLE_AUTO_SCHEMA_APPLY=true.

Interesting though is the fact that if we use Puma (rails s instead of passenger start) the problem doesn't appear. This might be related to the Passenger version of Elasticbeanstalk (4.0.60) and the message with the tree icons, maybe?.

Also, I wasn't able to reproduce locally using passenger 6 (didn't have time to install an older version in OSX).

[RafalWalczak]

@Lukasz-Ziemnicki @rrodrigu3z as I noticed also upgrade passenger to version 5.3.7 and above solves the problem.