Sharp as a dependency of Ghost, require('sharp') causes a full process crash

[ErisDS]

We're seeing several people reporting this same issue, all on systems where sharp was previously installed OK.

I don't have access to servers, I can't guarantee prerequisites match and I also haven't yet managed to reproduce, but we've had enough reports and enough detail now that I am fairly stuck and need some insider knowledge.

Relevant issue in Ghost is here: TryGhost/Ghost#11129

Ghost has sharp as an optional dependency, as we know the binary install could fail.

To further mitigate this, we wrap require in a try-catch block:

const makeSafe = fn => (...args) => {
    try {
        require('sharp');
    } catch (err) {
       ...
    }

One of the affected users tried adding console logs to this code:

const makeSafe = fn => (...args) => {
    common.logging.info('log 1');
    try {
        common.logging.info('log 2');
        require('sharp');
    } 
    ...

On their system, every time this code is executed, "log 1" is output and then the process crashes with no further logs or errors.

We're trying to get more information about what state their system is in, and add more debug info to both Ghost and Ghost-CLI to help find the problem, but we're stabbing in the dark.

The point of the try/catch should be that the require is safe. Sharp is present on these machines, and so I don't understand in what case require could cause a full process crash, but I assume that sharp must be installed but in a bad state?

[lovell]

Please can you try setting the npm_config_loglevel=verbose environment variable at https://github.com/TryGhost/Ghost-CLI/blob/master/lib/tasks/yarn-install.js#L67 , which will make prebuild-install output more of its decision making about prebuilt binaries.

[lovell]

It looks like the yarn wrapper logic in https://github.com/TryGhost/Ghost-CLI/blob/master/lib/utils/yarn.js is missing support for the stderr Stream, so some install-time error message will be lost.

[ErisDS]

Not arguing at all that we need to improve the CLI output so that errors are visible, and we are working on that 👍

The problem here is that we have sharp as an optional dependency and wrapped in a try-catch to account for install failures, because we can’t guarantee the environment, but requiring sharp crashes the whole process instantly.

There’s nothing more we can do to account for this than we are already doing. If the install fails, we’re OK with that (even if we don’t show it), if require throws an exception, we’re OK with that, but this a full process crash which is coming from sharp.

[lovell]

I notice there have been some new details/discussion on TryGhost/Ghost#11129 so I've asked a few questions there in the hope we can narrow down what's causing the crash.

I've also submitted an improvement to prebuild-install, which sharp uses at install time, to add support for the yarn --verbose flag (it already supports npm --verbose) - see prebuild/prebuild-install#111

[lovell]

It looks like the upstream issue at TryGhost/Ghost#11129 has now been closed, with people reporting via 🎉 emoji that upgrading sharp to the latest version has fixed this.

Even though we never quite found out exactly what was causing this, there were a number of improvements to install-time logging added to both prebuild-install and Ghost, so should something similar come up again we're in a better place to find it.

If you hadn't already seen #1282, the next v0.25.0 release of sharp is migrating to N-API internally, which should remove a whole class of common upgrade problems.

I'll close this but please feel free to re-open if people keep reporting this problem against Ghost with the latest version of sharp.

[ErisDS]

Was a super weird and nigh-impossible to reproduce issue, thanks for your time & 🤞 it's def resolved now.

[idmacdonald]

Unfortunately, I am still seeing this bug. I am running Ghost on Ubuntu 20.04.3 LTS, I followed the instructions at https://ghost.org/docs/install/ubuntu/ to install Ghost. It looks like anytime Ghost tries to use Sharp to resize images, Ghost crashes with an error like this:
traps: node[190951] trap invalid opcode ip:7f51fe938dfb sp:7ffcdffdfd60 error:0 in libvips-cpp.so.42[7f51fe62b000+aac000]

This is with Sharp 0.29.3 on amd64.

[lovell]

@idmacdonald Please can you open a new issue, additionally providing the output of cat /proc/cpuinfo