Bug ID: JDK-8372588 [asan] serviceability/sa/TestJmapCore.java and TestJmapCoreMetaspace.java fail after recent improvements

Type: Bug
Component: hotspot
Sub-Component: svc
Affected Version: 26

Priority: P4
Status: New
Resolution: Unresolved
OS: linux
CPU: generic

Submitted: 2025-11-26
Updated: 2025-11-29

In the 2 core related tests TestJmapCore.java and TestJmapCoreMetaspace.java we run into the errors below when using asan-enabled binaries. This is most likely because we do not write cores any more with asan; seems to be related to JDK-8368365 .

#  Internal Error (debug.cpp:282), pid=25655, tid=25658
#  fatal error: OutOfMemory encountered: Java heap space
#
# JRE version: OpenJDK Runtime Environment (26.0) (build 26-internal-adhoc.sapmachine.jdk)
# Java VM: OpenJDK 64-Bit Server VM (26-internal-adhoc.sapmachine.jdk, mixed mode, sharing, tiered, compressed oops, compact obj headers, g1 gc, linux-amd64)
# CreateCoredumpOnCrash turned off, no core file dumped
#
# An error report file with more information is saved as:
# /data/jvmtests/output_openjdk26_weekly_optU_linuxx86_64/jtreg_hotspot_tier2_work/JTwork/scratch/7/hs_err_pid25655.log
]
----------System.err:(15/877)----------
java.lang.RuntimeException: Output doesn't contain the location of core file.: expected true, was false
                at jdk.test.lib.Asserts.fail(Asserts.java:715)
                at jdk.test.lib.Asserts.assertTrue(Asserts.java:545)
                at jdk.test.lib.util.CoreUtils.parseCoreFileLocationFromOutput(CoreUtils.java:211)
                at jdk.test.lib.util.CoreUtils.getCoreFileLocation(CoreUtils.java:108)
                at TestJmapCore.test(TestJmapCore.java:86)
                at TestJmapCore.main(TestJmapCore.java:70)
                at java.base/jdk.internal.reflect.DirectMethodHandleAccessor.invoke(DirectMethodHandleAccessor.java:104)
                at java.base/java.lang.reflect.Method.invoke(Method.java:565)
                at com.sun.javatest.regtest.agent.MainActionHelper$AgentVMRunnable.run(MainActionHelper.java:335)
                at java.base/java.lang.Thread.run(Thread.java:1516)

and

#
#  Internal Error (debug.cpp:282), pid=25694, tid=25699
#  fatal error: OutOfMemory encountered: Metaspace
#
# JRE version: OpenJDK Runtime Environment (26.0) (build 26-internal-adhoc.sapmachine.jdk)
# Java VM: OpenJDK 64-Bit Server VM (26-internal-adhoc.sapmachine.jdk, mixed mode, sharing, tiered, compressed oops, compact obj headers, g1 gc, linux-amd64)
# CreateCoredumpOnCrash turned off, no core file dumped
#
# An error report file with more information is saved as:
# /data/jvmtests/output_openjdk26_weekly_optU_linuxx86_64/jtreg_hotspot_tier2_work/JTwork/scratch/9/hs_err_pid25694.log
]
----------System.err:(15/877)----------
java.lang.RuntimeException: Output doesn't contain the location of core file.: expected true, was false
                at jdk.test.lib.Asserts.fail(Asserts.java:715)
                at jdk.test.lib.Asserts.assertTrue(Asserts.java:545)
                at jdk.test.lib.util.CoreUtils.parseCoreFileLocationFromOutput(CoreUtils.java:211)
                at jdk.test.lib.util.CoreUtils.getCoreFileLocation(CoreUtils.java:108)
                at TestJmapCore.test(TestJmapCore.java:86)
                at TestJmapCore.main(TestJmapCore.java:70)
                at java.base/jdk.internal.reflect.DirectMethodHandleAccessor.invoke(DirectMethodHandleAccessor.java:104)
                at java.base/java.lang.reflect.Method.invoke(Method.java:565)
                at com.sun.javatest.regtest.agent.MainActionHelper$AgentVMRunnable.run(MainActionHelper.java:335)
                at java.base/java.lang.Thread.run(Thread.java:1516)

These test first check for the presence of the core file, and will fail if not present. Then they use SA tools to process the core file. That almost certainly will result in errors if any part of the core file is missing.
29-11-2025
A pull request was submitted for review. Branch: master URL: https://git.openjdk.org/jdk/pull/28552 Date: 2025-11-28 10:23:36 +0000
28-11-2025
Hi Thomas, the TestJmapCore uses jhsdb to work with the core : https://github.com/openjdk/jdk/blob/6901c05c9d23cde41e2af510e7d610af66e40770/test/hotspot/jtreg/serviceability/sa/TestJmapCore.java#L89 So it is 'unhappy' without the core file. I do not know what jhsdb does with the core, probably a tiny core would work too for the test, at least I do not see detailed checks in the test.
27-11-2025
[~mbaesken] If you want a quick solution, sure, just disable the tests for ASAN. I don't understand, though, how these tests succeeded before: by default, ASAN disables cores by enforcing a near-zero softlimit on core file size. Do these tests just test the existence of core files, or do they also parse the core files? If the former, the tests may have shown false negatives before (as in: "all ok" when the core files - being really tiny - were there but not really useful).
27-11-2025