ADDITIONAL SYSTEM INFORMATION :
It is OS independent as far as I know.

A DESCRIPTION OF THE PROBLEM :
When calling execute on a ForkJoinPool from a Runnable which is running inside the pool it is submitted to a local queue. The behavior changed in java 17, where locally submitted work items can starve global work items completely, and prevent them from ever running when all threads are busy. An example where this happens is when implementing actors which submit new messages to their executor, an actor which submits new requests as soon as receiving a reply can prevent all other actors from ever running by taking over all available threads.

As far as I could trace in java 16 there was a fairness bound on the amount of work processed from a local queue:

https://github.com/openjdk/jdk16u/blame/master/src/java.base/share/classes/java/util/concurrent/ForkJoinPool.java#L1012

However in java 17 there is no bound, and as long as there's one more task in the local queue thread spins without ever checking other queues:

https://github.com/openjdk/jdk17u/blame/master/src/java.base/share/classes/java/util/concurrent/ForkJoinPool.java#L1179

I have checked the latest available java 21 early access builds and this bug is still present. The problem does not reproduce when using a fixed thread pool, only with different variants of ForkJoinPool.

REGRESSION : Last worked in version 16

STEPS TO FOLLOW TO REPRODUCE THE PROBLEM :
1. Compile the provided MadSquirrel.java.
2. Run java MadSquirrel --squirrels 16 (where 16 is e.g. twice as much as the number of cores), you may also add --work-stealing-pool 8 arguments to specify the fixed number of threads when your cpu has too many cores
3. Observe the run stats printed every second, where each value after "Runs per second" corresponds to the number of times a particular squirrel executed that second.

EXPECTED VERSUS ACTUAL BEHAVIOR :
EXPECTED -
All squirrels should run periodically, there should be columns of "0".
ACTUAL -
Starting from java 17 I only ever observe at most the number of cores of squirrels running, others are starved and never given a chance to run after submission. For example:

% JAVA_HOME=~/tmp/jdk-17.0.9+5/Contents/Home java MadSquirrel --squirrels 16 --work-stealing-pool 8
Runs per second: 21288834, 28629451, 14856813, 15717559, 17508366, 25000050, 25665850, 26995270, 0, 0, 0, 0, 0, 0, 0, 0, total = 175662193
Runs per second: 19274901, 44231828, 16977416, 16935325, 6395690, 45072388, 46884795, 25108702, 0, 0, 0, 0, 0, 0, 0, 0, total = 220881045
Runs per second: 16874261, 42383196, 16365358, 16377313, 6774456, 41708803, 41877243, 24265580, 0, 0, 0, 0, 0, 0, 0, 0, total = 206626210
...

The same behavior with the latest java 21:

% JAVA_HOME=~/tmp/jdk-21+35/Contents/Home java MadSquirrel --squirrels 16 --work-stealing-pool 8
Runs per second: 27476990, 34773738, 30294322, 16072980, 15731359, 31293077, 30014221, 34929121, 0, 0, 0, 0, 0, 0, 0, 0, total = 220585808
Runs per second: 44260204, 45125972, 45126419, 12301768, 12364036, 43841939, 44674170, 45869310, 0, 0, 0, 0, 0, 0, 0, 0, total = 293563818
Runs per second: 43018632, 42239571, 43919551, 12150496, 12267957, 42320011, 41783814, 43268962, 0, 0, 0, 0, 0, 0, 0, 0, total = 280968994
...

---------- BEGIN SOURCE ----------
import java.util.ArrayList;
import java.util.concurrent.Executor;
import java.util.concurrent.Executors;
import java.util.concurrent.ForkJoinPool;
import java.util.concurrent.atomic.AtomicInteger;

public class MadSquirrel implements Runnable {
    private final Executor executor;
    private final AtomicInteger counter = new AtomicInteger(0);

    public MadSquirrel(Executor executor) {
        this.executor = executor;
    }

    public int getCount() {
        return counter.getAndSet(0);
    }

    @Override
    public void run() {
        counter.incrementAndGet();
        executor.execute(this);
    }

    public static void main(String[] args) throws InterruptedException {
        int nSquirrels = 1;
        Executor executor = ForkJoinPool.commonPool();
        for (int i = 0; i < args.length; ++i) {
            switch (args[i]) {
                case "--work-stealing-pool": {
                    int threads = Integer.parseInt(args[++i]);
                    executor = Executors.newWorkStealingPool(threads);
                    break;
                }
                case "--fixed-thread-pool": {
                    int threads = Integer.parseInt(args[++i]);
                    executor = Executors.newFixedThreadPool(threads);
                    break;
                }
                case "--squirrels": {
                    nSquirrels = Integer.parseInt(args[++i]);
                    break;
                }
                default: {
                    System.err.printf("Unsupported argument: %s\n", args[i]);
                    System.exit(1);
                }
            }
        }

        var squirrels = new ArrayList<MadSquirrel>();
        for (int i = 0; i < nSquirrels; ++i) {
            var squirrel = new MadSquirrel(executor);
            squirrels.add(squirrel);
            executor.execute(squirrel);
        }

        while (true) {
            Thread.sleep(1000);
            var sb = new StringBuilder();
            int total = 0;
            for (int i = 0; i < squirrels.size(); ++i) {
                if (i != 0) {
                    sb.append(", ");
                }
                int count = squirrels.get(i).getCount();
                sb.append(count);
                total += count;
            }
            System.out.printf("Runs per second: %s, total = %d\n", sb.toString(), total);
        }
    }
}

---------- END SOURCE ----------

FREQUENCY : always