JDK-6911204 : generated adapters with large signatures can fill up the code cache
The Version table provides details related to the release that this issue/RFE will be addressed.
Unresolved : Release in which this issue/RFE will be addressed. Resolved: Release in which this issue/RFE has been resolved. Fixed : Release in which this issue/RFE has been fixed. The release containing this fix may be available for download as an Early Access Release or a General Availability Release.
A customer reports that repeatedly loading and unloading their test class ultimately result in a OutOfMemory error. I attached the test case which shows the problem as unload.java.
6911204: generated adapters with large signatures can fill up the code cache
Reviewed-by: kvn, jrose
The adapter generation logic relies on the uint64_t fingerprint in the
methodOop to keep track of adapters and to reuse them for multiple
methods. Since the uint64_t can only hold a limited number of
arguments methods that don't fit aren't tracked. This means that
loading classes with lots of methods which aren't tracked can
eventually fill up the code cache. The fix is to acurately track all
signatures. I chose to use the representation of the calling
convention as the tracked information, since this will allow sharing
of adapters that are treated the same even though their signature in
basic types if diferent. For example, bytes, chars and shorts are all
passed as int so their calling convention is the same. This results
in fewer adapters being generated. Bootstrapping the JVM after this
changes takes about 1/4 as many adapters. The signatures are based on
VMRegPairs which are somewhat large so I went to a little extra
trouble to compact them.
At a minimum we also want to add a detail message in methodOopDesc when this occurs explaining what went wrong. I think we might want to change this to some other kind of exception since it's not a Java heap issue at all. Maybe a LinkageError?
We should accurately track the signatures for all the adapters instead of only tracking ones with small signatures. We might also consider unloading unused signatures by computing reference counts on the AdapterHandlerEntrys during GC.