Deterministic Concurrent Testing Using Fray

Mon, 17 Nov 2025 16:03:00 +0000
Testing, JVM, Jox, Concurrent, Fray

Testing concurrent code is hard. If you run a multi-threaded test and it passes, you never know if the code is correct or if it's just luck. The only way to be certain is through formal verification; however, despite significant progress in this area, writing and maintaining formally verified code remains very expensive and time-consuming. Hence, it's only applicable to a narrow spectrum of use cases.

That's where deterministic concurrent testing comes in. It doesn't give you a guarantee that your multi-threaded code is correct, but it also provides much more than unit or stress tests (that is, when you run your code in a loop, hoping that enough thread interleavings will occur to find any problems).

Fray is one such library, which enables writing concurrent tests for the JVM, deterministically simulating various thread interleaving, and if needed, replaying runs that failed, using a standard Java debugger.

Let's take a closer look at how Fray can be used and how it works.

What kind of bugs Fray detects

Before we begin, to reiterate: when writing a concurrent test using Fray, you won't get a guarantee that every concurrency bug will be found. Usually, the search space (the number of possible thread interleavings) is too large, as it grows exponentially with each synchronization point (such as acquiring a lock, invoking blocking I/O, or reading an atomic integer).

However, Fray explores the search space in a "probabilistically intelligent" way, implementing the latest research on the subject, and thus maximizing the probability that if a bug exists, it will be found.

On a more technical level, the test code is first instrumented at bytecode load-time. Then, Fray runs it in multiple iterations. It integrates well with build systems such as Maven or Gradle, testing frameworks such as JUnit, and any JVM-based languages. A concurrent test fails if there's a deadlock, or if any thread started during the test ends with an exception.

Here's a simple Fray-enabled test case, using JUnit:

@ExtendWith(FrayTestExtension.class)
public class FrayDemoTest {
    // Similar examples might include: rate limiters, CPU credits, ...
    class Bank {
        volatile int balance = 100;

        void withdraw(int amount) {
            if (balance >= amount) {
                balance = balance—amount;
            }

            assert(balance >= 0);
        }
    }

    @ConcurrencyTest
    public void bankTest() throws InterruptedException {
        var bank = new Bank();
        Thread t1 = new Thread(() -> {
            bank.withdraw(75);
        });
        Thread t2 = new Thread(() -> {
            bank.withdraw(75);
        });
        t1.start();
        t2.start();
        t1.join();
        t2.join();
    }
}

Despite using volatile to protect memory from concurrent access, this code is not correct, as there exists a thread interleaving where the assertion fails, and we end up with a negative account balance:

time | t1 | t2 |
1 | if (balance >= amount) | | true
2 | | if (balance >= amount) | still true!
3 | balance = balance - amount | | balance is 25
4 | | balance = balance - amount | balance is -50!

The problem is that if followed by an assignment is not an atomic operation. In fact, there are three separate operations (read in if, read old value, write...