Presently the VM is started with a fixed limit on the size of
generated code. When classes gets unloaded, the corresponding
nmethods gets flushed from the code cache.
If the code cache limit is reached, no more compilation occurs,
even is class unloading occurs at a future point. For most
programs this is a non-issue, but this failure has been observed
for the Cloudscape big app, which compiles SQL->Java on the fly
without ever unloading code.
A reasonable solution would be to write the currect GC invocation
count in an nmethod whenever it is seen on the stack (already done
for non-reentrant methods). If the code cache fills up, we can
force deoptimization and flushing of the nmethods with the lowest
An improved version of the scheme above would be to force frequent
safepoints when the code cache get close to the limit. At each
safepoint we could write the current GC invocation count in all
nmethods seen on stack. This would greate reduce the risk of
flushing a hot non-allocating nmethod.