JDK-8314515 : java/util/concurrent/SynchronousQueue/Fairness.java failed with "Error: fair=false i=8 j=0"
  • Type: Bug
  • Component: core-libs
  • Sub-Component: java.util.concurrent
  • Affected Version: 17,21,22,23
  • Priority: P4
  • Status: Resolved
  • Resolution: Fixed
  • OS: generic
  • CPU: x86_64
  • Submitted: 2023-08-17
  • Updated: 2024-09-22
  • Resolved: 2024-01-11
The Version table provides details related to the release that this issue/RFE will be addressed.

Unresolved : Release in which this issue/RFE will be addressed.
Resolved: Release in which this issue/RFE has been resolved.
Fixed : Release in which this issue/RFE has been fixed. The release containing this fix may be available for download as an Early Access Release or a General Availability Release.

To download the current JDK release, click here.
JDK 21 JDK 23
21.0.5Fixed 23 b06Fixed
Related Reports
Relates :  
JDK-8300663 - java/util/concurrent/SynchronousQueue/Fairness.java failed with "Error: fair=true i=0 j=1"
Description
The following test failed in the JDK22 CI:

java/util/concurrent/SynchronousQueue/Fairness.java

Here's a snippet from the log file:

#section:main
----------messages:(7/220)----------
command: main Fairness
reason: Assumed action based on file name: run main Fairness 
started: Wed Aug 16 19:43:08 UTC 2023
Mode: agentvm
Agent id: 50
finished: Wed Aug 16 19:43:55 UTC 2023
elapsed time (seconds): 47.581
----------configuration:(12/1456)----------

<snip>

----------System.err:(12/533)----------
java.lang.Error: fair=false i=8 j=0

	at Fairness.testFairness(Fairness.java:64)
	at Fairness.main(Fairness.java:71)
	at java.base/jdk.internal.reflect.DirectMethodHandleAccessor.invoke(DirectMethodHandleAccessor.java:103)
	at java.base/java.lang.reflect.Method.invoke(Method.java:580)
	at com.sun.javatest.regtest.agent.MainActionHelper$AgentVMRunnable.run(MainActionHelper.java:333)
	at java.base/java.lang.Thread.run(Thread.java:1570)

JavaTest Message: Test threw exception: java.lang.Error
JavaTest Message: shutting down test

result: Failed. Execution failed: `main' threw exception: java.lang.Error: fair=false i=8 j=0


This failure mode reminds me of:

JDK-8300663 java/util/concurrent/SynchronousQueue/Fairness.java failed with "Error: fair=true i=0 j=1"

which was fixed in jdk-22+8 on 2023.07.22.
Comments
[jdk21u-fix-request] Approval Request from Elif Aslan “Clean backport of fixing java/util/concurrent/SynchronousQueue/Fairness.java . Low risk. GHA tested”
18-06-2024
A pull request was submitted for review. URL: https://git.openjdk.org/jdk21u-dev/pull/745 Date: 2024-06-18 18:01:01 +0000
18-06-2024
[~dholmes] I would hope so, since it would reduce noise. But I want to first make sure that the fix is watertight first.
15-01-2024
Are we backporting this to 22?
15-01-2024
Not sure which JDKs this affects, but adding at least JDK 21 to match similarly looking JDK-8300663.
12-01-2024
[~vklang] - Will do!
11-01-2024
[~dcubed] Keep your fingers crossed! :)
11-01-2024
The fix for this bug is integrated in jdk-23+5-299.
11-01-2024
Changeset: 35e96627 Author: Viktor Klang <vklang@openjdk.org> Date: 2024-01-11 08:16:28 +0000 URL: https://git.openjdk.org/jdk/commit/35e9662767cc0a1dea9b5afa2a6d61a85297253c
11-01-2024
If the current theory around the cause of this issue was correct, then this should now be resolved. Given the problem of repeatability of this issue, it remains to be proven. Worth noting is that the fix was aimed at the test, and not the implementation, of the feature tested.
11-01-2024
Given that this test only seems to fail with an expected difference of 1 (in the fair=false case it tests inverse order of elements to fair=true) I am hopeful that https://github.com/openjdk/jdk/pull/17082 might be able to close any potential window of opportunity where thread status is updated before parking is in effect.
10-01-2024
Here's a log file snippet from the jdk-23+5-288-tier1 sighting: java/util/concurrent/SynchronousQueue/Fairness.java #section:main ----------messages:(7/217)---------- command: main Fairness reason: Assumed action based on file name: run main Fairness started: Wed Jan 10 18:32:17 GMT 2024 Mode: agentvm Agent id: 6 finished: Wed Jan 10 18:32:17 GMT 2024 elapsed time (seconds): 0.01 ----------configuration:(12/1559)---------- <snip> ----------System.err:(12/532)---------- java.lang.Error: fair=true i=1 j=2 at Fairness.testFairness(Fairness.java:64) at Fairness.main(Fairness.java:73) at java.base/jdk.internal.reflect.DirectMethodHandleAccessor.invoke(DirectMethodHandleAccessor.java:103) at java.base/java.lang.reflect.Method.invoke(Method.java:580) at com.sun.javatest.regtest.agent.MainActionHelper$AgentVMRunnable.run(MainActionHelper.java:333) at java.base/java.lang.Thread.run(Thread.java:1575) JavaTest Message: Test threw exception: java.lang.Error JavaTest Message: shutting down test result: Failed. Execution failed: `main' threw exception: java.lang.Error: fair=true i=1 j=2
10-01-2024
A pull request was submitted for review. URL: https://git.openjdk.org/jdk/pull/17082 Date: 2023-12-12 15:13:01 +0000
05-01-2024
Here's a log file snippet from the jdk-23+4-170-tier8 sighting: java/util/concurrent/SynchronousQueue/Fairness.java #section:main ----------messages:(7/228)---------- command: main Fairness reason: Assumed action based on file name: run main Fairness started: Thu Jan 04 06:10:52 UTC 2024 Mode: agentvm Agent id: 131 finished: Thu Jan 04 06:11:48 UTC 2024 elapsed time (seconds): 56.629 ----------configuration:(12/1167)*---------- Boot Layer class path: C:\\ade\\mesos\\work_dir\\jib-master\\install\\jtreg\\7.3.1\\1\\bundles\\jtreg-7.3.1+1.zip\\jtreg\\lib\\jtreg.jar C:\\ade\\mesos\\work_dir\\jib-master\\install\\jtreg\\7.3.1\\1\\bundles\\jtreg-7.3.1+1.zip\\jtreg\\lib\\junit-platform-console-standalone-1.9.2.jar C:\\ade\\mesos\\work_dir\\jib-master\\install\\jtreg\\7.3.1\\1\\bundles\\jtreg-7.3.1+1.zip\\jtreg\\lib\\testng-7.3.0.jar C:\\ade\\mesos\\work_dir\\jib-master\\install\\jtreg\\7.3.1\\1\\bundles\\jtreg-7.3.1+1.zip\\jtreg\\lib\\jcommander-1.82.jar C:\\ade\\mesos\\work_dir\\jib-master\\install\\jtreg\\7.3.1\\1\\bundles\\jtreg-7.3.1+1.zip\\jtreg\\lib\\guice-5.1.0.jar patch: java.base C:\\sb\\prod\\1704345123\\testoutput\\test-support\\jtreg_open_test_jdk_jdk_util\\patches\\java.base Test Layer class path: C:\\sb\\prod\\1704345123\\testoutput\\test-support\\jtreg_open_test_jdk_jdk_util\\classes\\3\\java\\util\\concurrent\\SynchronousQueue\\Fairness.d C:\\ade\\mesos\\work_dir\\jib-master\\install\\jdk-23+4-170\\src.full\\open\\test\\jdk\\java\\util\\concurrent\\SynchronousQueue ----------rerun:(35/4950)*---------- <snip> ----------System.err:(12/545)---------- java.lang.Error: fair=false i=8 j=0 at Fairness.testFairness(Fairness.java:64) at Fairness.main(Fairness.java:71) at java.base/jdk.internal.reflect.DirectMethodHandleAccessor.invoke(DirectMethodHandleAccessor.java:103) at java.base/java.lang.reflect.Method.invoke(Method.java:580) at com.sun.javatest.regtest.agent.MainActionHelper$AgentVMRunnable.run(MainActionHelper.java:333) at java.base/java.lang.Thread.run(Thread.java:1570) JavaTest Message: Test threw exception: java.lang.Error JavaTest Message: shutting down test result: Failed. Execution failed: `main' threw exception: java.lang.Error: fair=false i=8 j=0
04-01-2024
[~martin] I think that's a fair point. Currently we only test for platform threads (arguably it should also be tested with VTs?), and also given spinwait-backoff-park-loops it is difficult to test under all conditions since there's no way of determining the fairness aspect in this case besides statistically (since you'd need to formally define a total order AND have a bullet-proof way of telling whether a thread is considered blocked). All of the above of course is only as good as my (current) understanding, so I stand corrected if not.
23-08-2023
This test was flake-free for many years. The fact that has changed demands an explanation. OTOH I can't get it to fail myself, despite experimenting with 1000x repeat loops. The FIFO order of fair queues is in the spec and we shouldn't just stop testing that. (In fact, we should add missing tests for the dual case of multiple consumer threads queued up.) A last resort would be to weaken the test so that FIFO/LIFO is observed "most" of the time. Other synchronizers have inspection methods like getQueuedThreads. We could add such methods here as well, if only private ones for use by tests. Such a method could be implemented by traversing the linked list in SynchronousQueue's Transferer.
23-08-2023
Personally, I think that we either need a new test design to supersede this test, or delete it. My rationale is that I don't seem to be able to convince myself that the current test is robust enough to more than stochastically prove correctness, which means that we'll have to accept spurious failures, which means that we don't necessarily know if it is failing because it sometimes does, or because the implementation has a bug.
21-08-2023
The fix being discussed in JDK-8300663 has not yet been applied, so this is likely another symptom of the same issue. But I'm now leaning toward just deleting this test, as almost suggested by Martin. Unlike other queues, we can't usefully test per-producer fifo/lifo here. And checking fifo/lifo across threads requires the use of mechanics to check that threads are blocked in a specific way (here, on put()) that are at best fragile, depend on unspecified JVM properties, and unrepresentative of any actual usage. So, any objections to deleting the test?
21-08-2023
There's a good chance that there's an actual bug in SynchronousQueue's recent implementation changes that is being exposed by the rare test failures. Perhaps a spurious wakeup from park is causing rare queue reorderings? Perthaps checking Thread.State is insufficient to detect quiescence in q.put? The concept of ordering of producer threads in SynchronousQueue is in any case fuzzy, since there's no obvious way to tell whether one call to q.put should be ordered before another. There's no happens-before relationship, since neither thread has returned from q.put. I'm slightly surprised (or have I forgotten?) that LinkedTransferQueue's implementation is extended in SynchromousQueue to allow LIFO removal.
20-08-2023
A pull request was submitted for review. URL: https://git.openjdk.org/jdk/pull/15337 Date: 2023-08-17 22:35:39 +0000
17-08-2023
As usual, I can't reproduce this, but I'll try to fix it anyways by applying the traditional martinizing.
17-08-2023