JDK-8342570 : gc.*.gcUntilOld() assumes that full gc always promotes objects to old generation in a generational heap
  • Type: Sub-task
  • Component: hotspot
  • Sub-Component: gc
  • Affected Version: repo-shenandoah
  • Priority: P4
  • Status: Open
  • Resolution: Unresolved
  • Submitted: 2024-10-17
  • Updated: 2025-03-10
Related Reports
Relates :  
JDK-8342734 - GenShen: Test failure gc/shenandoah/TestReferenceRefersToShenandoah.java#generational
Description
gcUntilOld() in the following classes makes a very strong assumption that full gc's will eventually promote an arbitrary object to the old generation. This assumption may be too strong for some collectors, and should not therefore be needed in all of our tests. Specific collectors that would like to test for this property should continue to rely on that assumption, but other collectors that do not need that property to hold should not need to satisfy that assumption.

The following test classes are affected:

gc.TestReferenceRefersTo
gc.TestRerefenceRefersToDuringConcMark
gc.TestJNIWeak.TestJNIWeak

gc.shenandoah.TestReferenceRefersToShenandoah
Comments
A pull request was submitted for review. Branch: master URL: https://git.openjdk.org/shenandoah/pull/521 Date: 2024-10-21 15:42:07 +0000
26-10-2024
Will re-examine once the regression fixed in the linked ticket JDK-8342734 is landed.
24-10-2024
[~kbarrett], [~eosterlund], [~tschatzl] : all good points, thanks! Let me look more closely at the tests relying on the eventual promotion guarantee, and the Generational Shenandoah implementation details wrt reference processing behaviour, and we'll proceed based on what we find there.
23-10-2024
The issue is a bit deeper than I'd previously assumed; ignore the draft PR.
23-10-2024
The reason WhiteBox.fullGC was introduced was to provide a reliable way to get references and their referents into the old generation, to avoid having cross-generation issues interfering with tests. Maybe Shenandoah needs a more "interesting" implementation of that operation?
21-10-2024
[~ysr] the "yes" answer is correct for Generational ZGC. I wonder... the spurious inability to age objects... does it mean that you will throw OOME before clearing soft references (in the old generation) with a referent in the young generation that never gets promoted? That does sound a bit problematic. If that is the case, I wonder if we are looking at the tip of an iceberg here. Would you mind describing a bit during which conditions GenShen is unable to age objects? Is it related to the awkwardness of displaced mark words making our markWord age bits unstable? Or is it dense clusters of young objects that never trigger relocation, preventing aging? Or is it something else entirely?
21-10-2024
An alternative to adding a new API to CollectedHeap is just making WhiteBox.fullGC() guaranteeing this behavior. Particularly given all other collectors implement it this way, even if unintentionally. This seems a less intrusive option than having everyone being able to answer this question, and keeps compatibility with other tests assuming that (a single WhiteBox.fullGC promotes everything to old). There are quite a few tests assuming that when constructing their environment. Just like fullgc is assumed to guarantee enqueuing of unreachable finalizable objects - I just "fixed" a test a few days ago in that way (but it already assumed that before my change, but spammed WhiteBox.fullGC for no reason). It would be a lot more work (and fluff) if every test needed to be looked through, and modified accordingly. And then you couldn't test this behavior any more with GenShen.
19-10-2024
Hi Kim, it's a problem because with GenShen we may decide never to promote. Are there tests that rely on "eventual promotion" behavior? For the cases that I tested with the short-circuit (so we don't insist on the promotion as a guaranteed feature), the tests passed for GenShen where we do a GC then get out of the loop because we aren't guaranteed promotion. I'll publish a PR later today after more complete testing occurs. I did think that some reference processing tests which rely on references being cleared as a result of algorithms that look at "local reachability scope" might fail in these cases (as referent and referee might forever continue to be in different generations, e.g.), but I haven't encountered any such failures in testing so far. Note that as of today, GenShen is the only heap type that answers "no" to "does a full gc promote objects to old". Not sure if the "yes" answer is correct for Generational ZGC's. Thanks for your thoughts/suggestions.
18-10-2024
Some tests implement gcUntilOld as a loop. Others assume on full GC will suffice. The latter might not be a great assumption. But why is the former a problem?
18-10-2024
Once finalized, this will be independently landed upstream in tip first, if it passes muster in review.
18-10-2024