Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

openJcePlusTests_0 TestRSAPSSInterop>BaseTestRSAPSSInterop.testRSASignatureWithPSSMultiByteSize_IBM2BC2:265 Segmentation error vmState=0x00020019 #20947

Open
JasonFengJ9 opened this issue Jan 15, 2025 · 4 comments
Open

openJcePlusTests_0 TestRSAPSSInterop>BaseTestRSAPSSInterop.testRSASignatureWithPSSMultiByteSize_IBM2BC2:265 Segmentation error vmState=0x00020019 #20947

JasonFengJ9 opened this issue Jan 15, 2025 · 4 comments
Labels
comp:crypto comp:gc comp:jit test failure

Comments

@JasonFengJ9
Copy link
Member

JasonFengJ9 commented Jan 15, 2025 •
edited
Loading

Failure link

From internal Test_openjdk17_j9_extended.functional_ppc64le_linux_fips140_3_openjceplusfips (rtj-sles15le-svl-test-sik33-1)

java version "17.0.14-beta" 2025-01-21
IBM Semeru Runtime Certified Edition 17.0.14+6-202501111534 (build 17.0.14-beta+6-202501111534)
Eclipse OpenJ9 VM 17.0.14+6-202501111534 (build master-e0743f2495, JRE 17 Linux ppc64le-64-Bit Compressed References 20250111_855 (JIT enabled, AOT enabled)
OpenJ9   - e0743f2495
OMR      - 9e718a6f9
JCL      - cbb8509130 based on jdk-17.0.14+6)

Rerun in Grinder - Change TARGET to run only the failed test targets

Optional info

Failure output (captured from console output)

[2025-01-11T16:31:26.285Z] variation: NoOptions
[2025-01-11T16:31:26.285Z] JVM_OPTIONS:   -Dsemeru.fips=true -Dsemeru.customprofile=OpenJCEPlusFIPS

[2025-01-11T17:50:40.764Z] Testing doFinal
[2025-01-11T17:50:40.764Z] Test finished: ibm.jceplus.junit.openjceplusfips.multithread.TestAESCopySafe
[2025-01-11T17:50:40.764Z] Test calling: ibm.jceplus.junit.openjceplusfips.multithread.TestAESGCMLong
[2025-01-11T17:50:40.764Z] Test finished: ibm.jceplus.junit.openjceplusfips.multithread.TestAESGCMLong
[2025-01-11T17:50:40.764Z] Test calling: ibm.jceplus.junit.openjceplusfips.multithread.TestAESGCMNonExpanding
[2025-01-11T17:50:40.764Z] Test finished: ibm.jceplus.junit.openjceplusfips.multithread.TestAESGCMNonExpanding
[2025-01-11T17:50:40.764Z] Test calling: ibm.jceplus.junit.openjceplusfips.multithread.TestAESGCMSameBuffer
[2025-01-11T17:50:40.764Z] Test finished: ibm.jceplus.junit.openjceplusfips.multithread.TestAESGCMSameBuffer
[2025-01-11T17:50:40.764Z] Test calling: ibm.jceplus.junit.openjceplusfips.multithread.TestAESGCMUpdate
[2025-01-11T17:50:46.226Z] Unhandled exception
[2025-01-11T17:50:46.226Z] Type=Segmentation error vmState=0x00020019
[2025-01-11T17:50:46.226Z] J9Generic_Signal_Number=00000018 Signal_Number=0000000b Error_Value=00000000 Signal_Code=00000003
[2025-01-11T17:50:46.226Z] Handler1=00007FFFBA3832C0 Handler2=00007FFFBA2A8840
[2025-01-11T17:50:46.226Z] R0=00007FFFB99CB17C R1=00007FFF94F3AEF0 R2=00007FFFB9BA5500 R3=00007FFFB9707800
[2025-01-11T17:50:46.226Z] R4=00007FFF94F3AF58 R5=0000000000000100 R6=00007FFFB9B05728 R7=00007FFF94F3B0B8
[2025-01-11T17:50:46.226Z] R8=000000005EC2C2C3 R9=0000000000000000 R10=000000003F42C2C3 R11=0000000000000010
[2025-01-11T17:50:46.226Z] R12=000960020000372B R13=00007FFF94F468E0 R14=0000000000B65870 R15=0000000000AF4C00
[2025-01-11T17:50:46.226Z] R16=00007FFFB3400038 R17=00000000E39BD150 R18=000000000000188F R19=00000000000017F8
[2025-01-11T17:50:46.226Z] R20=0000000000000001 R21=00000000E39D5708 R22=0000000000001888 R23=6161616161616161
[2025-01-11T17:50:46.226Z] R24=00007FFFB40D7390 R25=00007FFFB40D71D8 R26=00007FFF94F3B510 R27=00007FFFB40D72B8
[2025-01-11T17:50:46.226Z] R28=0000000000000001 R29=00007FFF94F3AF58 R30=00007FFFB968DDC0 R31=00007FFFB9707800
[2025-01-11T17:50:46.226Z] NIP=0009600200003728 MSR=800000000280D033 ORIG_GPR3=00007FFFB99AFEC8 CTR=000960020000372B
[2025-01-11T17:50:46.226Z] LINK=00007FFFB99AFECC XER=0000000000000000 CCR=0000000084882846 SOFTE=0000000000000001
[2025-01-11T17:50:46.226Z] TRAP=0000000000000480 DAR=0009600200003728 dsisr=0000000040000000 RESULT=FFFFFFFFFFFFFFF2
[2025-01-11T17:50:46.226Z] FPR0=00000000000000ee (f: 238.000000, d: 1.175876e-321)
[2025-01-11T17:50:46.226Z] FPR1=406dc068a0000000 (f: 2684354560.000000, d: 2.380128e+02)
[2025-01-11T17:50:46.226Z] FPR2=43d8585860000000 (f: 1610612736.000000, d: 7.016997e+18)
[2025-01-11T17:50:46.226Z] FPR3=3fce840b4ac4e4d2 (f: 1254417664.000000, d: 2.384047e-01)
[2025-01-11T17:50:46.226Z] FPR4=bff0000000000000 (f: 0.000000, d: -1.000000e+00)
[2025-01-11T17:50:46.226Z] FPR5=3fe62e42fefa39ef (f: 4277811712.000000, d: 6.931472e-01)
[2025-01-11T17:50:46.226Z] FPR6=bfd2bef0a7c06ddb (f: 2814406144.000000, d: -2.929041e-01)
[2025-01-11T17:50:46.226Z] FPR7=bff0000000000000 (f: 0.000000, d: -1.000000e+00)
[2025-01-11T17:50:46.226Z] FPR8=bfdffffef20a4123 (f: 4060758272.000000, d: -4.999997e-01)
[2025-01-11T17:50:46.226Z] FPR9=3fd5575b0be00b6a (f: 199232368.000000, d: 3.334568e-01)
[2025-01-11T17:50:46.226Z] FPR10=bfd00ea348b88334 (f: 1220051712.000000, d: -2.508934e-01)
[2025-01-11T17:50:46.226Z] FPR11=bfdf946c1829d4b0 (f: 405394624.000000, d: -4.934340e-01)
[2025-01-11T17:50:46.226Z] FPR12=0000000000000000 (f: 0.000000, d: 0.000000e+00)
[2025-01-11T17:50:46.226Z] FPR13=05aeff6e0f29f023 (f: 254406688.000000, d: 2.668238e-281)
[2025-01-11T17:50:46.226Z] FPR14=0000000000000000 (f: 0.000000, d: 0.000000e+00)
[2025-01-11T17:50:46.226Z] FPR15=0000000000000000 (f: 0.000000, d: 0.000000e+00)
[2025-01-11T17:50:46.226Z] FPR16=0000000000000000 (f: 0.000000, d: 0.000000e+00)
[2025-01-11T17:50:46.226Z] FPR17=0000000000000000 (f: 0.000000, d: 0.000000e+00)
[2025-01-11T17:50:46.226Z] FPR18=0000000000000000 (f: 0.000000, d: 0.000000e+00)
[2025-01-11T17:50:46.226Z] FPR19=0000000000000000 (f: 0.000000, d: 0.000000e+00)
[2025-01-11T17:50:46.226Z] FPR20=0000000000000000 (f: 0.000000, d: 0.000000e+00)
[2025-01-11T17:50:46.226Z] FPR21=0000000000000000 (f: 0.000000, d: 0.000000e+00)
[2025-01-11T17:50:46.226Z] FPR22=0000000000000000 (f: 0.000000, d: 0.000000e+00)
[2025-01-11T17:50:46.226Z] FPR23=0000000000000000 (f: 0.000000, d: 0.000000e+00)
[2025-01-11T17:50:46.226Z] FPR24=0000000000000000 (f: 0.000000, d: 0.000000e+00)
[2025-01-11T17:50:46.226Z] FPR25=0000000000000000 (f: 0.000000, d: 0.000000e+00)
[2025-01-11T17:50:46.226Z] FPR26=0000000000000000 (f: 0.000000, d: 0.000000e+00)
[2025-01-11T17:50:46.227Z] FPR27=0000000000000000 (f: 0.000000, d: 0.000000e+00)
[2025-01-11T17:50:46.227Z] FPR28=0000000000000000 (f: 0.000000, d: 0.000000e+00)
[2025-01-11T17:50:46.227Z] FPR29=0000000000000000 (f: 0.000000, d: 0.000000e+00)
[2025-01-11T17:50:46.227Z] FPR30=0000000000000000 (f: 0.000000, d: 0.000000e+00)
[2025-01-11T17:50:46.227Z] FPR31=6161616161616161 (f: 1633771904.000000, d: 1.221764e+161)
[2025-01-11T17:50:46.227Z] Target=2_90_20250111_855 (Linux 6.4.0-150600.23.30-default)
[2025-01-11T17:50:46.227Z] CPU=ppc64le (4 logical CPUs) (0x1d5fc0000 RAM)
[2025-01-11T17:50:46.227Z] ----------- Stack Backtrace -----------
[2025-01-11T17:50:46.227Z] protectedBacktrace+0x24 (0x00007FFFBA2A2C84 [libj9prt29.so+0x32c84])
[2025-01-11T17:50:46.227Z] omrsig_protect+0x3e4 (0x00007FFFBA2A9CF4 [libj9prt29.so+0x39cf4])
[2025-01-11T17:50:46.227Z] omrintrospect_backtrace_thread_raw+0xe4 (0x00007FFFBA2A32A4 [libj9prt29.so+0x332a4])
[2025-01-11T17:50:46.227Z] protectedIntrospectBacktraceThread+0x24 (0x00007FFFBA2A28C4 [libj9prt29.so+0x328c4])
[2025-01-11T17:50:46.227Z] omrsig_protect+0x3e4 (0x00007FFFBA2A9CF4 [libj9prt29.so+0x39cf4])
[2025-01-11T17:50:46.227Z] omrintrospect_backtrace_thread+0x94 (0x00007FFFBA2A29D4 [libj9prt29.so+0x329d4])
[2025-01-11T17:50:46.227Z] generateDiagnosticFiles+0xd4 (0x00007FFFBA3830F4 [libj9vm29.so+0x430f4])
[2025-01-11T17:50:46.227Z] omrsig_protect+0x3e4 (0x00007FFFBA2A9CF4 [libj9prt29.so+0x39cf4])
[2025-01-11T17:50:46.227Z] structuredSignalHandler+0x194 (0x00007FFFBA383454 [libj9vm29.so+0x43454])
[2025-01-11T17:50:46.227Z] mainSynchSignalHandler+0x298 (0x00007FFFBA2A8AD8 [libj9prt29.so+0x38ad8])
[2025-01-11T17:50:46.227Z] __kernel_sigtramp_rt64+0x0 (0x00007FFFBAF80444 [linux-vdso64.so.1+0x444])
[2025-01-11T17:50:46.227Z] ---------------------------------------
[2025-01-11T17:50:46.227Z] JVMDUMP039I Processing dump event "gpf", detail "" at 2025/01/11 09:50:35 - please wait.

[2025-01-11T17:51:56.214Z] [INFO] Results:
[2025-01-11T17:51:56.215Z] [INFO] 
[2025-01-11T17:51:56.215Z] [ERROR] Failures: 
[2025-01-11T17:51:56.215Z] [ERROR]   TestRSAPSSInterop>BaseTestRSAPSSInterop.testRSASignatureWithPSSMultiByteSize_IBM2BC2:265
[2025-01-11T17:51:56.215Z] [INFO] 
[2025-01-11T17:51:56.215Z] [ERROR] Tests run: 2377, Failures: 1, Errors: 0, Skipped: 0
[2025-01-11T17:51:56.215Z] [INFO] 
[2025-01-11T17:51:56.215Z] [INFO] ------------------------------------------------------------------------
[2025-01-11T17:51:56.215Z] [INFO] BUILD FAILURE
[2025-01-11T17:51:56.215Z] [INFO] ------------------------------------------------------------------------
[2025-01-11T17:51:56.215Z] [INFO] Total time:  01:20 h
[2025-01-11T17:51:56.215Z] [INFO] Finished at: 2025-01-11T09:51:45-08:00
[2025-01-11T17:51:56.215Z] [INFO] ------------------------------------------------------------------------
[2025-01-11T17:51:56.215Z] [ERROR] Failed to execute goal org.apache.maven.plugins:maven-surefire-plugin:3.3.0:test (default-cli) on project OpenJCEPlus: 
[2025-01-11T17:51:56.215Z] [ERROR] 
[2025-01-11T17:51:56.215Z] [ERROR] Please refer to /home/jenkins/workspace/Test_openjdk17_j9_extended.functional_ppc64le_linux_fips140_3_openjceplusfips/aqa-tests/TKG/../TKG/output_17366130755911/openJcePlusTests_0/target/surefire-reports for the individual test results.
[2025-01-11T17:51:56.215Z] [ERROR] Please refer to dump files (if any exist) [date].dump, [date]-jvmRun[N].dump and [date].dumpstream.
[2025-01-11T17:51:56.215Z] [ERROR] The forked VM terminated without properly saying goodbye. VM crash or System.exit called?
[2025-01-11T17:51:56.215Z] [ERROR] Command was /bin/sh -c cd '/home/jenkins/workspace/Test_openjdk17_j9_extended.functional_ppc64le_linux_fips140_3_openjceplusfips/aqa-tests/TKG/../TKG/output_17366130755911/openJcePlusTests_0' && '/home/jenkins/workspace/Test_openjdk17_j9_extended.functional_ppc64le_linux_fips140_3_openjceplusfips/jdkbinary/j2sdk-image/bin/java' '--add-exports' 'openjceplus/com.ibm.crypto.plus.provider.ock=ALL-UNNAMED' '--add-exports' 'java.base/sun.security.util=ALL-UNNAMED' '-jar' '/home/jenkins/workspace/Test_openjdk17_j9_extended.functional_ppc64le_linux_fips140_3_openjceplusfips/aqa-tests/TKG/output_17366130755911/openJcePlusTests_0/target/surefire/surefirebooter-20250111083201576_3.jar' '/home/jenkins/workspace/Test_openjdk17_j9_extended.functional_ppc64le_linux_fips140_3_openjceplusfips/aqa-tests/TKG/output_17366130755911/openJcePlusTests_0/target/surefire' '2025-01-11T08-32-00_996-jvmRun1' 'surefire-20250111083201576_1tmp' 'surefire_0-20250111083201576_2tmp'
[2025-01-11T17:51:56.215Z] [ERROR] Error occurred in starting fork, check output in log
[2025-01-11T17:51:56.215Z] [ERROR] Process Exit Code: 255
[2025-01-11T17:51:56.215Z] [ERROR] Crashed tests:
[2025-01-11T17:51:56.215Z] [ERROR] ibm.jceplus.junit.TestMultithreadFIPS
[2025-01-11T17:51:56.215Z] [ERROR] org.apache.maven.surefire.booter.SurefireBooterForkException: The forked VM terminated without properly saying goodbye. VM crash or System.exit called?
[2025-01-11T17:51:56.215Z] [ERROR] Command was /bin/sh -c cd '/home/jenkins/workspace/Test_openjdk17_j9_extended.functional_ppc64le_linux_fips140_3_openjceplusfips/aqa-tests/TKG/../TKG/output_17366130755911/openJcePlusTests_0' && '/home/jenkins/workspace/Test_openjdk17_j9_extended.functional_ppc64le_linux_fips140_3_openjceplusfips/jdkbinary/j2sdk-image/bin/java' '--add-exports' 'openjceplus/com.ibm.crypto.plus.provider.ock=ALL-UNNAMED' '--add-exports' 'java.base/sun.security.util=ALL-UNNAMED' '-jar' '/home/jenkins/workspace/Test_openjdk17_j9_extended.functional_ppc64le_linux_fips140_3_openjceplusfips/aqa-tests/TKG/output_17366130755911/openJcePlusTests_0/target/surefire/surefirebooter-20250111083201576_3.jar' '/home/jenkins/workspace/Test_openjdk17_j9_extended.functional_ppc64le_linux_fips140_3_openjceplusfips/aqa-tests/TKG/output_17366130755911/openJcePlusTests_0/target/surefire' '2025-01-11T08-32-00_996-jvmRun1' 'surefire-20250111083201576_1tmp' 'surefire_0-20250111083201576_2tmp'
[2025-01-11T17:51:56.215Z] [ERROR] Error occurred in starting fork, check output in log
[2025-01-11T17:51:56.215Z] [ERROR] Process Exit Code: 255
[2025-01-11T17:51:56.215Z] [ERROR] Crashed tests:
[2025-01-11T17:51:56.215Z] [ERROR] ibm.jceplus.junit.TestMultithreadFIPS
[2025-01-11T17:51:56.215Z] [ERROR] 	at org.apache.maven.plugin.surefire.booterclient.ForkStarter.fork(ForkStarter.java:643)
[2025-01-11T17:51:56.215Z] [ERROR] 	at org.apache.maven.plugin.surefire.booterclient.ForkStarter.run(ForkStarter.java:285)
[2025-01-11T17:51:56.215Z] [ERROR] 	at org.apache.maven.plugin.surefire.booterclient.ForkStarter.run(ForkStarter.java:250)
[2025-01-11T17:51:56.215Z] [ERROR] 	at org.apache.maven.plugin.surefire.AbstractSurefireMojo.executeProvider(AbstractSurefireMojo.java:1224)
[2025-01-11T17:51:56.215Z] [ERROR] 	at org.apache.maven.plugin.surefire.AbstractSurefireMojo.executeAfterPreconditionsChecked(AbstractSurefireMojo.java:1073)
[2025-01-11T17:51:56.215Z] [ERROR] 	at org.apache.maven.plugin.surefire.AbstractSurefireMojo.execute(AbstractSurefireMojo.java:907)
[2025-01-11T17:51:56.215Z] [ERROR] 	at org.apache.maven.plugin.DefaultBuildPluginManager.executeMojo(DefaultBuildPluginManager.java:126)
[2025-01-11T17:51:56.215Z] [ERROR] 	at org.apache.maven.lifecycle.internal.MojoExecutor.doExecute2(MojoExecutor.java:328)
[2025-01-11T17:51:56.215Z] [ERROR] 	at org.apache.maven.lifecycle.internal.MojoExecutor.doExecute(MojoExecutor.java:316)
[2025-01-11T17:51:56.215Z] [ERROR] 	at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:212)
[2025-01-11T17:51:56.215Z] [ERROR] 	at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:174)
[2025-01-11T17:51:56.215Z] [ERROR] 	at org.apache.maven.lifecycle.internal.MojoExecutor.access$000(MojoExecutor.java:75)
[2025-01-11T17:51:56.215Z] [ERROR] 	at org.apache.maven.lifecycle.internal.MojoExecutor$1.run(MojoExecutor.java:162)
[2025-01-11T17:51:56.215Z] [ERROR] 	at org.apache.maven.plugin.DefaultMojosExecutionStrategy.execute(DefaultMojosExecutionStrategy.java:39)
[2025-01-11T17:51:56.215Z] [ERROR] 	at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:159)
[2025-01-11T17:51:56.215Z] [ERROR] 	at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject(LifecycleModuleBuilder.java:105)
[2025-01-11T17:51:56.215Z] [ERROR] 	at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject(LifecycleModuleBuilder.java:73)
[2025-01-11T17:51:56.215Z] [ERROR] 	at org.apache.maven.lifecycle.internal.builder.singlethreaded.SingleThreadedBuilder.build(SingleThreadedBuilder.java:53)
[2025-01-11T17:51:56.215Z] [ERROR] 	at org.apache.maven.lifecycle.internal.LifecycleStarter.execute(LifecycleStarter.java:118)
[2025-01-11T17:51:56.215Z] [ERROR] 	at org.apache.maven.DefaultMaven.doExecute(DefaultMaven.java:261)
[2025-01-11T17:51:56.215Z] [ERROR] 	at org.apache.maven.DefaultMaven.doExecute(DefaultMaven.java:173)
[2025-01-11T17:51:56.215Z] [ERROR] 	at org.apache.maven.DefaultMaven.execute(DefaultMaven.java:101)
[2025-01-11T17:51:56.215Z] [ERROR] 	at org.apache.maven.cli.MavenCli.execute(MavenCli.java:903)
[2025-01-11T17:51:56.215Z] [ERROR] 	at org.apache.maven.cli.MavenCli.doMain(MavenCli.java:280)
[2025-01-11T17:51:56.215Z] [ERROR] 	at org.apache.maven.cli.MavenCli.main(MavenCli.java:203)
[2025-01-11T17:51:56.215Z] [ERROR] 	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
[2025-01-11T17:51:56.215Z] [ERROR] 	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77)
[2025-01-11T17:51:56.215Z] [ERROR] 	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
[2025-01-11T17:51:56.215Z] [ERROR] 	at java.base/java.lang.reflect.Method.invoke(Method.java:575)
[2025-01-11T17:51:56.215Z] [ERROR] 	at org.codehaus.plexus.classworlds.launcher.Launcher.launchEnhanced(Launcher.java:255)
[2025-01-11T17:51:56.215Z] [ERROR] 	at org.codehaus.plexus.classworlds.launcher.Launcher.launch(Launcher.java:201)
[2025-01-11T17:51:56.215Z] [ERROR] 	at org.codehaus.plexus.classworlds.launcher.Launcher.mainWithExitCode(Launcher.java:361)
[2025-01-11T17:51:56.215Z] [ERROR] 	at org.codehaus.plexus.classworlds.launcher.Launcher.main(Launcher.java:314)
[2025-01-11T17:51:56.215Z] [ERROR] 
[2025-01-11T17:51:56.215Z] [ERROR] -> [Help 1]
[2025-01-11T17:51:56.215Z] [ERROR] 
[2025-01-11T17:51:56.215Z] [ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
[2025-01-11T17:51:56.215Z] [ERROR] Re-run Maven using the -X switch to enable full debug logging.
[2025-01-11T17:51:56.215Z] [ERROR] 
[2025-01-11T17:51:56.215Z] [ERROR] For more information about the errors and possible solutions, please read the following articles:
[2025-01-11T17:51:56.215Z] [ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoExecutionException
[2025-01-11T17:51:56.215Z] -----------------------------------
[2025-01-11T17:51:56.215Z] openJcePlusTests_0_FAILED

50x internal Grinder - there were timeouts, but no failure was reproduced.

FYI @jasonkatonica
@dmitripivkine
Copy link
Contributor

dmitripivkine commented Jan 16, 2025 •
edited
Loading

Crash in GC code during object allocation. I am going to triage it.

@dmitripivkine
Copy link
Contributor

crash stack:

#13 <signal handler called>
#14 0x0009600200003728 in ?? ()
#15 0x00007fffb99afecc in omrGcDebugAssertionOutput () from /team/dmitri/20947/jdk-17.0.14+6/lib/default/libj9gc29.so
#16 0x00007fffb99cb17c in MM_LargeObjectAllocateStats::getSizeClassIndex(unsigned long) ()
   from /team/dmitri/20947/jdk-17.0.14+6/lib/default/libj9gc29.so
#17 0x00007fffb99cf1f8 in MM_LargeObjectAllocateStats::decrementFreeEntrySizeClassStats(unsigned long, MM_FreeEntrySizeClassStats*, unsigned long) () from /team/dmitri/20947/jdk-17.0.14+6/lib/default/libj9gc29.so
#18 0x00007fffb99cf4e0 in MM_LargeObjectAllocateStats::decrementFreeEntrySizeClassStats(unsigned long) ()
   from /team/dmitri/20947/jdk-17.0.14+6/lib/default/libj9gc29.so
#19 0x00007fffb999cee8 in MM_MemoryPoolAddressOrderedList::allocateObject(MM_EnvironmentBase*, MM_AllocateDescription*)
    () from /team/dmitri/20947/jdk-17.0.14+6/lib/default/libj9gc29.so
#20 0x00007fffb9a6d36c in MM_MemorySubSpaceGeneric::allocateObject(MM_EnvironmentBase*, MM_AllocateDescription*, MM_MemorySubSpace*, MM_MemorySubSpace*, bool) () from /team/dmitri/20947/jdk-17.0.14+6/lib/default/libj9gc29.so
#21 0x00007fffb99be058 in MM_TLHAllocationInterface::allocateArrayletSpine(MM_EnvironmentBase*, MM_AllocateDescription*, MM_MemorySpace*, bool) () from /team/dmitri/20947/jdk-17.0.14+6/lib/default/libj9gc29.so
#22 0x00007fffb99c6990 in OMR_GC_AllocateObject(OMR_VMThread*, MM_AllocateInitialization*) ()
   from /team/dmitri/20947/jdk-17.0.14+6/lib/default/libj9gc29.so
#23 0x00007fffb9895094 in J9AllocateIndexableObjectNoGC () from /team/dmitri/20947/jdk-17.0.14+6/lib/default/libj9gc29.so
#24 0x00007fffb2c7f038 in fast_jitNewArray () from /team/dmitri/20947/jdk-17.0.14+6/lib/default/libj9jit29.so
#25 0x00007fffb2c94d44 in jitNewArray () from /team/dmitri/20947/jdk-17.0.14+6/lib/default/libj9jit29.so

This crash occur because of an attempt to allocate byte array 0x187F (6271) elements (see !MM_IndexableObjectAllocationModel 0x7FFF94F3B500 for reference. This array is irrelevant to the crash).

During memory search in the !mm_memorypooladdressorderedlist 0x7FFFB40D6E80 starting at !mm_heaplinkedfreeheader 0xE399E5E8 list walk reached damaged list element eventually:

> !mm_heaplinkedfreeheader 0x00000000E39D6FE8
MM_HeapLinkedFreeHeader at 0xe39d6fe8 {
  Fields for MM_HeapLinkedFreeHeader:
	0x0: U64 _next = 0x00001A55000B7700 (28952375293696)
	0x8: U64 _size = 0x6161616161616161 (7016996765293437281)
}

The garbage in _size field is a reason for crash in GC code.

Actually there is the object written at the place where Linked Free List element expected to be:

> !j9object 0xE39D6FE8
!J9IndexableObject 0x00000000E39D6FE8 {
    struct J9Class* clazz = !j9arrayclass 0xB7700   // [B
    Object flags = 0x00000000;
    U_32 size = 0x00001A55;
	[0] =  97, 0x61
	[1] =  97, 0x61
	[2] =  97, 0x61
...
To print entire range: !j9indexableobject 0x00000000E39D6FE8 0 6741
}

This object is complete (with header). It means it is allocated either by GC itself out-of-line (hard to believe - this is core Memory Pool functionality has not been changed for years and, if broken, should manifest itself much more often) or, alternatively by JIT as inline allocated object. The scenario in this case might be: this object was about to be allocated in TLH by JIT and proposed allocation address was stored somewhere in the native but Global GC occur. It triggered flush TLH for the thread and adding TLH reminder to free memory linked list. However internally stored memory address survived and JIT continued allocation process over newly written Linked Free Header and corrupted it.

!j9object 0xE39D6FE8 currently is referred from slots (including O-slot) from !stackslots 0xaf9400, for example from

<af9400> JIT frame: bp = 0x0000000000B6B4F8, pc = 0x00007FFF97E7E150, unwindSP = 0x0000000000B6B130, cp = 0x0000000000448B00, arg0EA = 0x0000000000B6B530, jitInfo = 0x00007FFF94B3C038
<af9400> 	Method: ibm/jceplus/junit/base/BaseTestAESGCMUpdate.doTestWithMultipleDataUpdate(ILjavax/crypto/SecretKey;[B[BLjavax/crypto/spec/GCMParameterSpec;I)[B !j9method 0x0000000000449CF0

@hzongaro Would you please take a look?

@dmitripivkine
Copy link
Contributor

dmitripivkine commented Jan 16, 2025 •
edited
Loading

I do see another hypothetical scenario for this failure. If there is code copied heap content somewhere and returned it back, and it operates with significantly larger chunk as expected it can create this pattern. We can not detect it for alive objects (still be at it's own place even rewritten) but newly created Linked Free Element can be overwritten by object use to be on this place before. I noticed that for array written over there are first 208 elements filled by 0x61 with zeroes in remaining part. I looked to examples of similar arrays in the heap, for example !j9indexableobject 0x00000000E39D3EA0 0 6240. This array is filled with 0x61 fully. Not sure can it be prove of "resurrection" scenario. I don't see any confirmation for it, so it is pure speculation.
(correction)
I think there is very low possibility of this scenario. Problematic object is alive currently, but it should be discovered dead for Linked Free Header to be created at its place. So two problem should occur at the same time.

@dmitripivkine
Copy link
Contributor

I guess there are more scenarios if, for example, native security code violates fundamental rules, like respect to VM Access etc.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
comp:crypto comp:gc comp:jit test failure
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@JasonFengJ9 @dmitripivkine