JDK-8079136 : Accessing a nested sublist leads to StackOverflowError
  • Type: Enhancement
  • Component: core-libs
  • Sub-Component: java.util:collections
  • Affected Version: 7u80
  • Priority: P4
  • Status: Closed
  • Resolution: Fixed
  • Submitted: 2015-04-29
  • Updated: 2019-11-21
  • Resolved: 2016-03-31
The Version table provides details related to the release that this issue/RFE will be addressed.

Unresolved : Release in which this issue/RFE will be addressed.
Resolved: Release in which this issue/RFE has been resolved.
Fixed : Release in which this issue/RFE has been fixed. The release containing this fix may be available for download as an Early Access Release or a General Availability Release.

To download the current JDK release, click here.
JDK 9
9 b113Fixed
Related Reports
Relates :  
Relates :  
Relates :  
Description
FULL PRODUCT VERSION :
java version "1.7.0_71"
Java(TM) SE Runtime Environment (build 1.7.0_71-b14)
Java HotSpot(TM) 64-Bit Server VM (build 24.71-b01, mixed mode)


ADDITIONAL OS VERSION INFORMATION :
Darwin MacBook-Pro.local 14.1.0 Darwin Kernel Version 14.1.0: Thu Feb 26 19:26:47 PST 2015; root:xnu-2782.10.73~1/RELEASE_X86_64 x86_64

But this is an OS agnostic bug: in Java code.

A DESCRIPTION OF THE PROBLEM :
Invoking List.subList(..) on a typical List implementation in java.util, returns a view onto the parent list. If you then invoke subList on that returned view, what you get is a view of view onto the parent list. Logically, this is the correct behavior. Implementation-wise, however, this "second-order" view is a [n unnecessary] wrapper on top of a wrapper of the parent list. Unfortunately, this small inefficiency can quickly snowball if an algorithm repeatedly applies subList starting with a good sized list (say 500k items). In the extreme cases, you end up with list instances which blow up (StackOverflowError) on later invoking List.get(int). 

STEPS TO FOLLOW TO REPRODUCE THE PROBLEM :
See attached unit test

EXPECTED VERSUS ACTUAL BEHAVIOR :
EXPECTED -
Unit test should pass.
ACTUAL -
Unit test fails:

[java.util.Arrays$ArrayList, size:50,000] 2,704 ms

java.lang.AssertionError: [java.util.Arrays$ArrayList, size:50,000]: error on 29,590th sub-list - java.lang.StackOverflowError
	at org.junit.Assert.fail(Assert.java:88)
	at com.gnahraf.util.bf.SubListBugReport.assertSubListTraversal(SubListBugReport.java:65)
	at com.gnahraf.util.bf.SubListBugReport.assertSubListTraversal(SubListBugReport.java:43)
	at com.gnahraf.util.bf.SubListBugReport.showIt(SubListBugReport.java:35)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)
	at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
	at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47)
	at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
	at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:325)
	at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:78)
	at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:57)
	at org.junit.runners.ParentRunner$3.run(ParentRunner.java:290)
	at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71)
	at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288)
	at org.junit.runners.ParentRunner.access$000(ParentRunner.java:58)
	at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:268)
	at org.junit.runners.ParentRunner.run(ParentRunner.java:363)
	at org.junit.runner.JUnitCore.run(JUnitCore.java:137)
	at com.intellij.junit4.JUnit4IdeaTestRunner.startRunnerWithArgs(JUnit4IdeaTestRunner.java:74)
	at com.intellij.rt.execution.junit.JUnitStarter.prepareStreamsAndStart(JUnitStarter.java:211)
	at com.intellij.rt.execution.junit.JUnitStarter.main(JUnitStarter.java:67)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
	at com.intellij.rt.execution.application.AppMain.main(AppMain.java:134)

[java.util.Arrays$ArrayList, size:500] 0 ms

REPRODUCIBILITY :
This bug can be reproduced always.

---------- BEGIN SOURCE ----------
/*
 * Copyright (c) 2015 Babak Farhang
 */

package com.gnahraf.util.bf;


import org.junit.Test;


import java.text.DecimalFormat;
import java.util.Arrays;
import java.util.List;

import static org.junit.Assert.*;

/**
 * Created by babak on 4/29/15.
 */
public class SubListBugReport {

    DecimalFormat formatter = new DecimalFormat("#,###");


    @Test
    public void worksForSmall() {
        assertSubListTraversal(500);
    }


    @Test
    public void showIt() {
        assertSubListTraversal(50000);
    }


    void assertSubListTraversal(int count) {
        Integer[] array = new Integer[count];
        for (int i = 0; i < count; ++i)
            array[i] = i;
        assertSubListTraversal(Arrays.asList(array));
    }


    void assertSubListTraversal(List<?> list) {
        int i = 1;
        StackOverflowError error = null;
        long tick = System.currentTimeMillis();
        try {
            List<?> sub = list;
            for (; i < list.size(); ) {
                sub = sub.subList(1, sub.size());
                assertEquals(list.get(i++), sub.get(0));
            }
            assertEquals(1, sub.size());
        } catch (StackOverflowError soe) {
            error = soe;
        }

        long lap = System.currentTimeMillis() - tick;
        System.out.println(label(list) + " " + formatter.format(lap) + " ms");
        if (error != null) {
            fail(label(list) + ": error on " + formatter.format(i) + "th sub-list - " + error);
            throw error;
        }
    }

    private void printLap(String label, long tick) {
        long lap = System.currentTimeMillis() - tick;
        System.out.println("[" + label + "] " + formatter.format(lap) + " ms");
    }

    private String label(List<?> list) {
        return "[" + list.getClass().getName() + ", size:" + formatter.format(list.size()) + "]";
    }
}

---------- END SOURCE ----------

CUSTOMER SUBMITTED WORKAROUND :
/*
 * Copyright (c) 2015 Babak Farhang.
 * Licensed under the Apache License, Version 2.0
 */

package com.gnahraf.util.bf;

import java.util.AbstractList;
import java.util.Collections;
import java.util.List;

/**
 * Wrap instances with this class as a work-around. 
 * Created by babak on 4/29/15.
 */
public class SubList<T> extends AbstractList<T> {

    private final static SubList<?> EMPTY = new SubList<Object>(Collections.EMPTY_LIST);

    public static <T> SubList<T> empty() {
        return (SubList<T>) EMPTY;
    }

    private final List<T> base;
    private final int offset;
    private final int size;

    public SubList(List<T> base) {
        this(base, 0, base.size(), false);
    }

    public SubList(List<T> base, int offset, int size) {
        this(base, offset, size, false);
        if (base == null)
            throw new IllegalArgumentException("null base list");
        if (offset < 0)
            throw new IllegalArgumentException("offset: " + offset);
        if (size < 0)
            throw new IllegalArgumentException("size: " + offset);
        if (offset + size > base.size())
            throw new IllegalArgumentException(
                    "offset: " + offset + ", size: " + size + "; base.size(): " + base.size());
    }


    private SubList(List<T> base, int offset, int size, boolean ignore) {
        this.base = base;
        this.offset = offset;
        this.size = size;
    }

    @Override
    public T get(int location) {
        return base.get(location + offset);
    }

    @Override
    public int size() {
        return size;
    }

    @Override
    public List<T> subList(int start, int end) {
        if (end == size && start == 0)
            return this;

        if (start < 0 || end < start || end > size)
            throw new IllegalArgumentException(
                    "start: " + start + ", end: " + end + ", size: " + size);

        int subSize = end - start;
        if (subSize == 0)
            return empty();
        else
            return new SubList<T>(base, offset + start, subSize, false);
    }
}



Comments
Yes, I was just going to respin the discussion at the core-libs-dev mail group.
28-01-2016

My apologies for completing forgetting about recent attempts to fix.
27-01-2016

Previous discussion on core-libs-dev@openjdk: 1) http://mail.openjdk.java.net/pipermail/core-libs-dev/2011-July/007222.html 2) http://mail.openjdk.java.net/pipermail/core-libs-dev/2015-May/033159.html In the second thread Ivan G posted a couple rounds of webrevs. There are certainly some subtle issues here. I'm not sure what the resolution was. In any case it was "In Progress" at some point in the past; I'm not sure if that reflects actual current status.
27-01-2016

This is a "well-known" design problem with nested sublist creation. Each sublist only knows about its parent, causing deeply nested layers of wrappers. This falls out naturally from the design. Fixing it compatibly may be hard. There's an expectation that ConcurrentModificationException is detected relative to the parent, and that would change if the link to the parent were no longer there. I seem to recall efforts in the distant past to fix it, but they went nowhere. I can't find an existing bug for this. I don't know why this bug is in state "Fix Understood". Does anyone have plans to seriously tackle this? It's more important to fix in a modern world where fork/join on ArrayLists may cause the deeply nested sublist effect, typically affecting performance but not causing StackOverflowError.
27-01-2016

Attaching updated test case by the submitter. Checked this for JDK 8u45, 8u60 ea b21, and 9 ea b71 and could reproduce as mentioned.
07-07-2015

Adding comments from the submitter: ----------------------------------------------------------------------------------------------------------------------------- On 7/7/2015 9:50 AM, .......... wrote: > In case it's hard to imagine how one could hit this bug, my use case involved a multi-pass computation over a largish (2M) list of objects, and I was representing progress--or, rather the remaining work to be done, with List.subList(x, List.size()) after processing say x items (x being much smaller than List.size()). After a few tens of thousands of that call, I hit the bug. I could've, of course, represented this "frontier" of work to be done with say another object containing both the list and an index into that list, but representing it simply with a sub list was much, much cleaner. > > Any way, I hope you push to get this fixed. The Java Collections is a foundational library which everyone trusts will be kept squeaky clean and little, easy-to-fix warts like this one besmirch that well deserved reputation. > > Best, > ..... -------------------------------------------------------------------------------------------------------------------------------
07-07-2015