Given the following test suite class:

---%<---
import java.lang.reflect.Method;
import java.util.Arrays;
import junit.framework.TestCase;
public class OrderedTest extends TestCase {
    static {
        try {
            Method m = Class.class.getDeclaredMethod("getDeclaredMethods0", boolean.class);
            m.setAccessible(true);
            Method[] ms = (Method[]) m.invoke(OrderedTest.class, false);
            System.err.println(Arrays.toString(ms));
            ms = (Method[]) m.invoke(OrderedTest.class, true);
            System.err.println(Arrays.toString(ms));
        } catch (Exception x) {
            x.printStackTrace();
        }
    }
    private static int counter;
    public OrderedTest(String name) {
        super(name);
    }
    public void testZero() {
        assertEquals(0, counter);
        counter++;
    }
    public void testOne() {
        assertEquals(1, counter);
        counter++;
    }
}
---%<---

Compile with JUnit 4.8.2 in the classpath, using this command to run:

java -cp ... junit.textui.TestRunner OrderedTest

Using JDK 5 or 6, the test always passes:

---%<---
[public void OrderedTest.testZero(), public void OrderedTest.testOne()]
[public void OrderedTest.testZero(), public void OrderedTest.testOne()]
..
Time: 0.001

OK (2 tests)

---%<---

Using JDK 7 (b147 on Ubuntu Lucid x86), the test usually passes too. But sometimes it fails:

---%<---
[public void OrderedTest.testOne(), public void OrderedTest.testZero()]
[public void OrderedTest.testOne(), public void OrderedTest.testZero()]
.F.
Time: 0.001
There was 1 failure:
1) testOne(OrderedTest)junit.framework.AssertionFailedError: expected:<1> but was:<0>
	at OrderedTest.testOne(OrderedTest.java:26)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)

FAILURES!!!
Tests run: 2,  Failures: 1,  Errors: 0

---%<---

Compiling using earlier versions of javac, or using -source 5 -target 5, has no effect; and JDK 7 javap prints the methods in source order. So the reordering seems to be occurring in HotSpot.

Now of course the Javadoc for Class.getDeclaredMethods states that the order of results in unspecified, but JUnit uses this order unchanged when running test cases, and in any large body of tests there will be some that unintentionally relied on the order of execution of test cases. Such tests may have run reliably for years using earlier JDKs but will suddenly break - sometimes - when run on JDK 7. The reason for the failure may also not be immediately apparent.