JDK-8151410 : CDS limits cds_total size + compressed_class_space_size to less than 4G during dump time
  • Type: Bug
  • Component: hotspot
  • Sub-Component: runtime
  • Affected Version: 9
  • Priority: P3
  • Status: Closed
  • Resolution: Duplicate
  • OS: generic
  • CPU: generic
  • Submitted: 2016-03-07
  • Updated: 2024-11-06
  • Resolved: 2017-04-04
The Version table provides details related to the release that this issue/RFE will be addressed.

Unresolved : Release in which this issue/RFE will be addressed.
Resolved: Release in which this issue/RFE has been resolved.
Fixed : Release in which this issue/RFE has been fixed. The release containing this fix may be available for download as an Early Access Release or a General Availability Release.

To download the current JDK release, click here.
JDK 10
10Resolved
Related Reports
Duplicate :  
Relates :  
Relates :  
Description
The following code in Meatspace::global_initialize() limits cdc_total size + compressed_class_space_size to less than 4G during dump time (on 64 platforms with compressed class pointer enabled). Thus, the narrow_klass_shift is always 0 when CDS is enabled. That is not necessary. 

void Metaspace::global_initialize() {
...
#ifdef _LP64
    if (cds_total + compressed_class_space_size() > UnscaledClassSpaceMax) {
      vm_exit_during_initialization("Unable to dump shared archive.",
          err_msg("Size of archive (" SIZE_FORMAT ") + compressed class space ("
                  SIZE_FORMAT ") == total (" SIZE_FORMAT ") is larger than compressed "
                  "klass limit: " UINT64_FORMAT, cds_total, compressed_class_space_size(),
                  cds_total + compressed_class_space_size(), UnscaledClassSpaceMax));
    }

    // Set the compressed klass pointer base so that decoding of these pointers works
    // properly when creating the shared archive.
    assert(UseCompressedOops && UseCompressedClassPointers,
      "UseCompressedOops and UseCompressedClassPointers must be set");
    Universe::set_narrow_klass_base((address)_space_list->current_virtual_space()->bottom());
    log_develop_trace(gc, metaspace)("Setting_narrow_klass_base to Address: " PTR_FORMAT,
                                     p2i(_space_list->current_virtual_space()->bottom()));

    Universe::set_narrow_klass_shift(0);
#endif // _LP64

Metaspace::set_narrow_klass_base_and_shift() takes both CDS shared space and compressed class space into account when setting the narrow klass base and shift at runtime (when shift is enabled). CDS archives dump time narrow klass base and shift. If the narrow klass base and shift are different and runtime, CDS invalidates the archived narrow klass pointers (currently only from shared strings by skipping the shared string data). The dump time limit can be removed.

void Metaspace::set_narrow_klass_base_and_shift(address metaspace_base, address cds_base) { 
  // Figure out the narrow_klass_base and the narrow_klass_shift. The 
  // narrow_klass_base is the lower of the metaspace base and the cds base 
  // (if cds is enabled). The narrow_klass_shift depends on the distance 
  // between the lower base and higher address. 
  address lower_base; 
  address higher_address; 
#if INCLUDE_CDS 
  if (UseSharedSpaces) { 
    higher_address = MAX2((address)(cds_base + FileMapInfo::shared_spaces_size()), 
                          (address)(metaspace_base + compressed_class_space_size())); 
    lower_base = MIN2(metaspace_base, cds_base); 
  } else 
#endif 
  { 
    higher_address = metaspace_base + compressed_class_space_size(); 
    lower_base = metaspace_base; 

    uint64_t klass_encoding_max = UnscaledClassSpaceMax << LogKlassAlignmentInBytes; 
    // If compressed class space fits in lower 32G, we don't need a base. 
    if (higher_address <= (address)klass_encoding_max) { 
      lower_base = 0; // Effectively lower base is zero. 
    } 
  } 

  Universe::set_narrow_klass_base(lower_base); 

  if ((uint64_t)(higher_address - lower_base) <= UnscaledClassSpaceMax) { 
    Universe::set_narrow_klass_shift(0); 
  } else { 
    assert(!UseSharedSpaces, "Cannot shift with UseSharedSpaces"); 
    Universe::set_narrow_klass_shift(LogKlassAlignmentInBytes); 
  } 
} 

Comments
This issue will be addressed by part of the change for JDK-8072061.
04-04-2017

Ioi, the first 'if' check above should also include the size of the shared class data, because the runtime class metadata includes data in both compressed class space and the shared class space. So, the runtime narrow_klass_shift is determined by CompressedClassSpaceSize+shared_class_size. However, the narrow_klass_shift needs to be determined early, which is before we know the total size of the shared class metadata during dump time. For dump time, we could just use the above and only look at the CompressedClassSpaceSize.
16-02-2017

Jiangli, I think the main problem raised by this bug is: "CDS archive is always dump with narrow_klass_shift==0". This problem is also related to JDK-8174986 - CDS shared string region is not compatible with AOT. I think the problem with narrow_klass_shift can be address as part of JDK-8072061 - Eliminate the need to manually specify CDS region sizes. With my JDK-8072061 fix, SharedReadWriteSize, SharedReadOnlySize, SharedMiscDataSize and SharedMiscCodeSize are ignored. Instead, during dump time, I reserve the space [SharedAddressBase, .... SharedAddressBase+CompressedClassSpaceSize], and allocate the Klasses within this space. I can add something like this in the metaspace initialization code during dump time: if (UseAOT || CompressedClassSpaceSize > UnscaledClassSpaceMax) { Universe::set_narrow_klass_shift(LogKlassAlignmentInBytes); } else { Universe::set_narrow_klass_shift(0); } Will this address your concern?
16-02-2017

Jiangli, thank you for analysis. I will file a new bug for this issue.
09-08-2016

Patch is attached.
15-06-2016