JDK-8164793 : new ArrayDeque(2**N) allocates backing array of size 2**(N+1)
  • Type: Bug
  • Component: core-libs
  • Sub-Component: java.util:collections
  • Affected Version: 6,7,8,9
  • Priority: P4
  • Status: Resolved
  • Resolution: Fixed
  • OS: generic
  • CPU: generic
  • Submitted: 2016-08-22
  • Updated: 2017-05-31
  • Resolved: 2016-11-29
The Version table provides details related to the release that this issue/RFE will be addressed.

Unresolved : Release in which this issue/RFE will be addressed.
Resolved: Release in which this issue/RFE has been resolved.
Fixed : Release in which this issue/RFE has been fixed. The release containing this fix may be available for download as an Early Access Release or a General Availability Release.

To download the current JDK release, click here.
JDK 9
9 b148Fixed
Related Reports
Relates :  
Relates :  
Description
FULL PRODUCT VERSION :
java version "1.6.0_45"
Java(TM) SE Runtime Environment (build 1.6.0_45-b06)
Java HotSpot(TM) Client VM (build 20.45-b01, mixed mode, sharing)

ADDITIONAL OS VERSION INFORMATION :
All OS

A DESCRIPTION OF THE PROBLEM :
The problem is on the method allocateElements.
When the paramenter is a number power of 2 (8, 16, 32, etc.) the calculated capacity is the double (16, 32, 64, etc.).
The problem is on the strategy to add the bits at the rigth, because the bits are allays added even the passed argument have a valid size (number power 2).
One way to resolve, is:
1 - on the if convert "<=" on "<", this will solve the problem for minimum capacity (8);
2 - initially inside the if, assign the passed argument -1 to the initialCapacity variable: "initialCapacity = numElements - 1;" ; (this will make the algoritm start on the previous number that wil be anulated by "initialCapacity++;" at the end;

problematic method with my annotations:

    private void allocateElements(int numElements) {
        int initialCapacity = MIN_INITIAL_CAPACITY;
        // Find the best power of two to hold elements.
        // Tests "<=" because arrays aren't kept full.
        if (numElements >= initialCapacity) { // change to: if (numElements = initialCapacity) {
            initialCapacity = numElements; // change to: initialCapacity = numElements - 1;
            initialCapacity |= (initialCapacity >>>  1);
            initialCapacity |= (initialCapacity >>>  2);
            initialCapacity |= (initialCapacity >>>  4);
            initialCapacity |= (initialCapacity >>>  8);
            initialCapacity |= (initialCapacity >>> 16);
            initialCapacity++;

            if (initialCapacity < 0)   // Too many elements, must back off
                initialCapacity >>>= 1;// Good luck allocating 2 ^ 30 elements
        }
        elements = (E[]) new Object[initialCapacity];
    }


STEPS TO FOLLOW TO REPRODUCE THE PROBLEM :
unit test code:

    @Test
    public void testArrayDequeAllocateElements() throws Exception {
        Object[] elementsArr = getelementsArrFromArrayDeque(new ArrayDeque<String>(8));
        assertEquals(8, getelementsArrFromArrayDeque(new ArrayDeque<String>(8)).length);// faill
        assertEquals(16, getelementsArrFromArrayDeque(new ArrayDeque<String>(9)).length);
        assertEquals(16, getelementsArrFromArrayDeque(new ArrayDeque<String>(10)).length);
        assertEquals(16, getelementsArrFromArrayDeque(new ArrayDeque<String>(11)).length);
        assertEquals(16, getelementsArrFromArrayDeque(new ArrayDeque<String>(12)).length);
        assertEquals(16, getelementsArrFromArrayDeque(new ArrayDeque<String>(13)).length);
        assertEquals(16, getelementsArrFromArrayDeque(new ArrayDeque<String>(14)).length);
        assertEquals(16, getelementsArrFromArrayDeque(new ArrayDeque<String>(15)).length);
        assertEquals(16, getelementsArrFromArrayDeque(new ArrayDeque<String>(16)).length);// faill
        assertEquals(32, getelementsArrFromArrayDeque(new ArrayDeque<String>(17)).length);
    }

    private Object[] getelementsArrFromArrayDeque(ArrayDeque<String> ad) throws NoSuchFieldException, IllegalAccessException {
        Object[] elementsArr = new Object[0];
        Field elementsField = ArrayDeque.class.getDeclaredField("elements");
        elementsField.setAccessible(true);
        elementsArr = (Object[]) elementsField.get(ad);
        return elementsArr;
    }


EXPECTED VERSUS ACTUAL BEHAVIOR :
EXPECTED -
The capacity should be the passed argument (if power of 2) ou the next power of 2 value
ACTUAL -
when power of 2 value passed (8, 16, 32, etc.) the capacity is the the double (16, 32, 64, etc.)

REPRODUCIBILITY :
This bug can be reproduced always.


Comments
Yes, almost all the details of the implementation would need to change, but the changes are straightforward. I would not normally backport such a change.
29-08-2016

The beauty of tbd_minor and tbd_major are that they AREN'T precisely defined. :-) I've gone ahead and marked this as tbd_minor. It doesn't require an API change, so it conceivably could be done in a minor release. However, changing the array size from 2**N to essentially an arbitrary size requires fairly widespread changes. I'd recommend it be done early in development of a major release.
29-08-2016

Jeff, I don't know where tbd_minor or tbd_major are precisely defined. It would be nice to fix in a major release, but a minor one is also OK. If anyone's going to fix it, it will probably be me.
29-08-2016

@Martin - would this be reasonable to set fixVersion to tbd_minor or tbd_major?
29-08-2016

The problem is that the internal algorithm requires both that the internal array is a power of two, and that one of the array elements is always null. With those assumptions, new ArrayDeque(2**N) sadly has no choice but to allocate a (pessimal) array of size 2**(N+1). ArrayDeque should be changed to completely redo the internal organization, saving one array element, and removing this allocation inefficiency. Instead of having head and tail fields, it should have head and size fields. Growing by 3/2 instead of 2 would probably also be progress. Workaround: new ArrayDeque(2**N - 1)
25-08-2016

To reproduce, run the attached test case with assertions enabled. The output matches what the submitter claims. JDK 6u45 - fail JDK 7u80 - Fail JDK 8u112 ea -Fail JDK 9+128 - fail Output: Exception in thread "main" java.lang.AssertionError at JI9043094.main(JI9043094.java:8)
25-08-2016