Apache / PHP segfaults (PHP 8.0 and 8.1, but also 7.4) #8159
Comments
|
a backtrace would help, if I recall correctly, openbsd tends to generate a coredump as |
|
There is no php.core, only httpd2.core. I tried looking but there are not enough debug symbols within ... perhaps need a binary package with DEBUG=on ... This crash is very weird. |
|
This is the output of GDB gdb /usr/local/sbin/httpd2 /httpd2.coreGNU gdb 6.3 Core was generated by `httpd2'. There is nothing more ... in the GDB output ! |
|
Also maybe this can help you: (gdb) list |
|
(gdb) bt full |
|
Even if i use just this in opcache.ini or if I use: It crashes with any of these configs (default or custom) for opcache.ini ... |
Core
Core |
pcre
pcre |
Zend OPcache
Zend OPcache |
|
The only difference I see between restarting via rcctl says in log: caught SIGWINCH, shutting down gracefully but with kill -HUP says in log: |
thanks however would be best to focus on PHP 8.x, as said in another ticket PHP 7.4 is no longer supported. |
That would be ideal if that is possible. |
PHP 8.0 and PHP 8.1 are also crashing on OpenBSD 7.0 amd64 with Apache 2.4.52 ! |
PHP 8.0 GDB$ gdb /usr/local/sbin/httpd2 /httpd2.core Core was generated by `httpd2'. |
This is the GDB output for PHP 8.0.15(gdb) list |
PHP 8.0.15Zend OPcache
Zend OPcache |
Just to answer for the fact you wrote that PHP 7.4 is no more supported ...I repeat any of: PHP 8.1.2 / PHP 8.0.15 / PHP 7.4.27 are affected by this issue I reported.Looks like the only unaffected version is PHP 7.3.33 !I have all 4 versions installed on OpenBSD 7.0, I can switch and use any of these versions. If I disable in opcache.ini the opcache.so no crash happen in any of these versions !Thus this makes me think this is something related with the opcache module of PHP.If I do When the apache logs are rotating, as usual it must be: I am using OpenBSD since 10 years or so. Log rotation using I will install over the weekend PHP 7.4.28 / 8.0.16 and 8.1.3 and test again but I am sure they will also be crashing because I had not seen any fix regarding the opcache area in the changes between. This is not an OpenBSD 7.0 or Apache 2.4.52 issue ... PHP 7.3.33 is not crashing under the exact conditions, configurations, OS version and Apache version ... Perhaps you are confused how different PHP versions can coexist ... under OpenBSD the official packages can coexist all installed and can activate any of them. So this is how I am able to test any PHP version by just switching the Apache PHP module on-the-fly keeping the same configs... |
|
Question: which version of PHP would you like me to compile with DEBUG on ? 8.0.16 or 8.1.3 ? |
|
since it s most likely a bug fix to consider, PHP 8.0 seems the way to go, at least for starter. we ll see if this is duplicated existing bug or not. |
unix-world
changed the title
|
OK. |
|
This above (debug) build was using the official OpenBSD 7.0 / PHP 8.0 package (includding specific platform patches), except I added one extra flag: "--enable-debug=yes". |
Here is a second debug log that the OpenBSD package maintainer sent me, using a newer GDB version:PHP 8.0.15 (debug) with newer GDB 9.2 |
|
Would it be possible for you to create an additional patch in your repository named patch- diff --git a/ext/opcache/ZendAccelerator.c b/ext/opcache/ZendAccelerator.c
and try with a new build ? Just to check something |
|
I was looking over the changes between PHP 7.3 / 7.4 / 8.0 in ZendAccelerator.c, especially on the parts that handle the SIGKILL. In 7.3 appears: |
|
@unix-world can you please check if graceful Apache restart (kill -USR1) works or not. |
|
I've just checked. |
|
but |
|
OpenBSD is not using systemd. The content of /etc/rc.d/apache2 startup script: Thus when doing If I try |
|
When building from the source the PHP 8.0, |
|
Differences between PHP 7.3.33 and 8.0.16 in ext/opcache/ZendAccelerator.c that may cause the crashing issue: in PHP 8.0.16 (crashing) there is: in PHP 7.3.33 (not crashing) In my opinion (not sure yet) somewhere at these lines the crash is producing for PHP 8.0.16, perhaps in some circumstances cwd is NULL somehow ... because in PHP 8.0 this part is in post-deactivate (crashing) while in PHP 7.3.33 (not crashing) is in a different method : PHP 7.3.33 |
|
This bug is somehow similar with the PHP behavior by calling cwd in a class destructor that also (still) fails ... Thus in my opinion calling (ZCG(cwd)) in post_deactivate (as in PHP 8.0.x) produces the crash ... instead calling it in a separate method like in PHP 7.3.x (accel_deactivate). I presume this bug is not visible on linux platform (but is still there ...) because the systemd is much more slower (compared with rc.d) on restarting a service like apache ... |
|
https://httpd.apache.org/docs/2.4/stopping.html Until this bug is fixed inside PHP opcache, I can only do |
I was also tried a different compile scenario.I compiled PHP 8.0.16 as above, win an extra additional flag: --enable-ztsIn this scenario apache2 refuses to load the PHP opcache. If ZTS enabled, everything works OK except opcache ... If I try to load opcache.so apache does not start at all and display the following error in the errors_log: If ZTS flag disabled, without opcache it works just fine. With PHP opcache crashes. PS: with ZTS enabled, the php-cli also fails with |
Update: To make the opcache to function when PHP 8.0 is build with --enable-zts I had to add this flag too: --disable-opcache-jit.Otherwise apache or php-cli exit with errors if PHP ZTS try to load opcache extension: |
|
@dstogov Could you reproduce the apache crash on SIGHUP or SIGUSR1 bug ? |
|
@devnexen is the |
|
you highlighted a lot of concerns not related with the issue at hand but would you be able to create a patch which would replace (same file name as above)
by
? normally it should not crash however SIGTERM had been chosen because it can be blocked but just to verify something. |
|
I have tried this and still crashing. But I observed that in PHP 8.0 2 extra lines have been added and in PHP 7.3.33 they are not present: and I am trying with this 4 lines commented out, ... I have an idea that here might be the clue, after issuing SIGTERM or SIGKILL the apache may halt the access to logging and here might be a crash perhaps ... because the other log situations in this code block I could not reproduce but they might be also crashing, but never got yet entered in the Let me try this and I get back. |
|
I was trying any of the above scenarios. I was talking with an internal guy here that have some limited C skills and he told me that from what he can see in the debug log the crashing issue comes from Thus I have noticed only one major difference in the logic between PHP 7.3.33 and 8.0.16: In 8.0.16 the accel_activate method is called in a completely different place: What is your opinion on this ? I am not aware too much of Zend internal code ... |
|
I do not have opinion yet, I ll try to set up this weekend hopefully a separate openbsd VM to be able to "mess up" with packages and debugging unless someone find some clue in the meantime. |
|
OK. By the way, you can install the binary ports, but in order to compile it from source there are some things you have to know if you don't already know:
I can confirm that also the original ports built by the package maintainers have this bug. I get the email messages with one of the maintainers. He told me that this is for sure something inside PHP opcache that changed. PS: also be sure to use the llvm 11, which is the default compiler (not gcc) R. |
|
@unix-world I suspect, the crash is related to Apache graceful restart and dynamic loading of php and opcache. According to backtrace opcache can't call back to PHP from accel_startup(). I don't have OpenBSD installation and can't reproduce the problem, |
|
The best backtrace shows crash under start_accel_module(), but this function just forwards to zend_startup_module(), that is not in backtrace. Try to attach GDB/LLDB to the main Apache process, set breakpoint on start_accel_module() and then try to understand what is going wrong after restart. It's also possible to add some instrumentation code into start_accel_module() to write address of |
|
@dstogov All 4 versions are built with exactly the same build options and same platform patches (of course adapted by version). I suspect is something about this change ... because the opcache JIT is just in 8.0.x and 8.1.x but 7.4.x still crashes and does not have yet the opcache JIT... |
|
PS: |
I am really confused. You closed this bug without any explanation.Look at the below bug, it was an older bug reporting approximately the same issue, perhaps still not solved: |
|
** And I think the following FreeBSD bug is exactly the same after they switch from PHP 7.3 to 7.4 ... and still unsolved / unfixed in PHP 8.0 / 8.1 !: ** |
|
@devnexen I don't know why this ticket was closed without an answer. In my opinion the bug I reported here is because of the new changes in the opcache, in PHP 7.4 / 8.0 and 8.1. I am not a C expert, but this clear looks to me like a "use-after-free error" ... In 7.3.33 the accel_activate method is called in ext/opcache/ZendAccelerator.c, line # 3156 In 8.0.16 the accel_activate method is called in a completely different place: Please not that I included the PHP 7.4 in this discussion just to exclude the possibility this bug have nothing to do with the opcache JIT (introduced only since 8.0) ! |
|
@dstogov , as you said:
But please look at this part of the code and you see that accel_activate is called here !!: PHP 8.0 / 8.1 / 7.4But in PHP 7.3 (not crashing) is called in completely different place In PHP 7.3 the accel_activate is called here |
|
I see |
|
I suspect calling the accel_activate() from the new place in PHP 7.4 / 8.0 / 8.1 which also occurs when calling start_accel_module() could be a cause ... not sure. |
|
You may revert the the commit 34a491d1ed5 to check this. |
Description
PHP 7.4.27 / 8.0.15 / 8.1.3
Apache 2.4.52
OpenBSD 7.0 / amd64
If running any of the above PHP versions under Apache (as module), I noticed that every time when apache rotate it's logs it crashes. To be sure of this issue I set the log rotation at a fixed hour (not by size).
In the last 3 days, each day at the exactly same hour apache crashed and did not crashed in the rest.
Finally I did not loaded the opcache.so and it did not crashed on log rotation.
After further investigation I noticed this:
The problem is somwehere in opcache.so.
All the above versions are affected.
The PHP 7.3.33 works OK, it does not crash.
How to reproduce the issue:
If I do not load opcache.so it does not crashes.
PHP Version
8.0.15
Operating System
OpenBSD 7.0
The text was updated successfully, but these errors were encountered: