JDK-8213250 : CDS archive creation aborts due to metaspace object allocation failure
  • Type: Bug
  • Component: hotspot
  • Sub-Component: runtime
  • Affected Version: 11,12
  • Priority: P3
  • Status: Resolved
  • Resolution: Fixed
  • Submitted: 2018-11-01
  • Updated: 2020-04-15
  • Resolved: 2018-11-09
The Version table provides details related to the release that this issue/RFE will be addressed.

Unresolved : Release in which this issue/RFE will be addressed.
Resolved: Release in which this issue/RFE has been resolved.
Fixed : Release in which this issue/RFE has been fixed. The release containing this fix may be available for download as an Early Access Release or a General Availability Release.

To download the current JDK release, click here.
JDK 11 JDK 12
11.0.8-oracleFixed 12 b20Fixed
Related Reports
Relates :  
JDK-8200078 - [Graal] runtime/appcds/GraalWithLimitedMetaspace.java crashes in visit_all_interfaces
Relates :  
JDK-8211956 - AppCDS crashes for some uses with JRuby
Relates :  
JDK-8213331 - Add an assert that Metaspace::allocate should not be called in VM thread
Relates :  
JDK-8213346 - Re-implement shared dictionary using CompactHashtable
Description
[This bug was discovered during investigation of JDK-8211956]

When creating a CDS archive for jruby 9.2.0.0, the following error happens once every few times. When the failure happens, The reported request size that triggered the failure may be different.

Loading classes to share: done.
Rewriting and linking classes ...
Rewriting and linking classes: done
Number of classes 7323
    instance classes   =  7244
    obj array classes  =    71
    type array classes =     8
Updating ConstMethods ... done. 
Removing unshareable information ... done. 
Failed allocating metaspace object type TypeArrayU8 of size 24. CDS dump aborted.
Please increase MaxMetaspaceSize (currently 18446744073709547520 bytes).

============
Steps to reproduce:
1. Download https://repo1.maven.org/maven2/org/jruby/jruby-dist/9.2.0.0/jruby-dist-9.2.0.0-bin.tar.gz (or from attachment in this bug report)
2. Download jruby-startup.tar.gz from attachment in this bug report
3. Download JDK 11 -- OpenJDK Runtime Environment 18.9 (build 11+28)
4. Extract the above files to

$HOME/jruby/jruby-9.2.0.0/bin
$HOME/jruby/jruby-startup
$HOME/jruby/jdk-11

5. Run the following scripts to set up your environment

unalias java
unalias jruby
unset JAVA_HOME
export PATH=$HOME/jruby/jruby-9.2.0.0/bin:$PATH
export PATH=$HOME/jruby/jdk-11/bin:$PATH
java -version
jruby -J-version
type jruby

It should report the following versions:
$ java -version
openjdk version "11" 2018-09-25
OpenJDK Runtime Environment 18.9 (build 11+28)
OpenJDK 64-Bit Server VM 18.9 (build 11+28, mixed mode)
$ jruby -J-version
openjdk version "11" 2018-09-25
OpenJDK Runtime Environment 18.9 (build 11+28)
OpenJDK 64-Bit Server VM 18.9 (build 11+28, mixed mode)
$ type jruby
jruby is hashed (/home/iklam/jruby/jruby-9.2.0.0/bin/jruby)

6. Run the following command to reproduce the failure. You may need to run it several times for it to fail:

cd ~/jruby/jruby-startup
JAVA_OPTS="-Xmx256m" jruby -Ilib bin/generate-appcds "-S gem list"

7. Once the failure has happened, you can run the following command to reproduce it more quickly (without doing a trial run)

VERIFY_JRUBY=1 JAVA_OPTS="-Xshare:dump -XX:+UnlockDiagnosticVMOptions -XX:SharedClassListFile=$HOME/jruby/jruby-9.2.0.0/lib/jruby.list -XX:SharedArchiveFile=$HOME/jruby/jruby-9.2.0.0/lib/jruby.jsa" $HOME/jruby/jruby-9.2.0.0/bin/jruby -S gem list

the above command actually exec's the following, which can be used to debug the problem in gdb

java -Xshare:dump -XX:+UnlockDiagnosticVMOptions \
    -XX:SharedClassListFile=${HOME}/jruby/jruby-9.2.0.0/lib/jruby.list \
    -XX:SharedArchiveFile=${HOME}/jruby/jruby-9.2.0.0/lib/jruby.jsa \
    -Xss2048k \
    -Djffi.boot.library.path=${HOME}/jruby/jruby-9.2.0.0/lib/jni \
    -Djava.security.egd=file:/dev/urandom \
    -classpath \
    ${HOME}/jruby/jruby-9.2.0.0/lib/jruby.jar:: \
    -Djruby.home=${HOME}/jruby/jruby-9.2.0.0 \
    -Djruby.lib=${HOME}/jruby/jruby-9.2.0.0/lib \
    -Djruby.script=jruby \
    -Djruby.shell=/bin/sh
Comments
Fix request (11u) I would like to downport this for parity with 11.0.8-oracle. I had to do a row of adaptions to get it working: http://mail.openjdk.java.net/pipermail/jdk-updates-dev/2020-March/002774.html
15-03-2020
URL: http://hg.openjdk.java.net/jdk/jdk/rev/e0fd97beab7e User: iklam Date: 2018-11-09 05:04:18 +0000
09-11-2018
The check for !(DumpSharedSpaces && THREAD->is_VM_thread()) was first introduced in JDK-8200078
02-11-2018
With JDK 11, a work-around is to run -Xshare:dump with a large (initial) MetaspaceSize. E.g., java -Xshare:dump -XX:MetaspaceSize=128m .....
02-11-2018
With JDK 12, I can easily reproduce with product build (once every 2~3 runs), but cannot reproduce with fastdebug or slowdebug builds, even when running in a loop. But I can reproduce in a product build that has debug information (-O0 -g), so good.
01-11-2018
We also saw a similar failure with the internal "spring boot" benchmark with latest JDK 12 builds: Number of classes 6012 instance classes = 5943 obj array classes = 61 type array classes = 8 Updating ConstMethods ... done. Removing unshareable information ... done. Failed allocating metaspace object type TypeArrayU8 of size 24. CDS dump aborted. Please increase MaxMetaspaceSize (currently 18446744073709547520 bytes). The cmd line options are: + /xxxxxx/spring-greeting-1.13/jlopt/bin/java -XX:-UsePerfData -XX:CICompilerCount=2 -XX:TieredStopAtLevel=1 -XX:+UseParallelGC -XX:ParallelGCThreads=1 -Xmx128m -DautoQuit=true -XX:+UnlockDiagnosticVMOptions -XX:-UseAOT -Xshare:off -XX:+IgnoreEmptyClassPaths -Xshare:dump -XX:+IgnoreEmptyClassPaths -Xlog:class+path=info -XX:+IgnoreUnverifiableClassesDuringDump -XX:SharedArchiveFile=App.jsa -XX:SharedClassListFile=App.classlist -XX:SharedReadWriteSize=75M -jar ./spring-greeting-1.13.jar
01-11-2018