Max Bernstein's Blog

Sorry for marking all the posts as unread

Wed, 31 Jan 2024 00:00:00 +0000

I noticed that the URLs were all a little off (had two slashes instead of one) and went in and fixed it. I did not think everyone's RSS software was going to freak out the way it did. PS: this is a special RSS-only post that is not visible on the site. Enjoy.

A quick look at zero-knowledge proofs

Fri, 07 Aug 2026 00:00:00 +0000

NB: This isn’t about crypto. I don’t care about crypto.

Chris messaged me the other week asking if I wanted to implement zero-knowledge proofs. I initially was not interested, but then he said:

What if I told you there’s a version of them that has nothing to do with cryptocurrencies? What if I told you it involves graph theory? What if I told you there’s a 30 line implementation?

Now that was interesting.

The idea of a zero-knowledge proof (ZKP) is that there are two parties: the prover and the verifier. The prover asserts that it has a solution to a (generally NP-complete) problem. The prover can convince the verifier of this without sharing the actual solution to the problem.

The canonical example is 3-coloring a graph. That is, the prover asserts that, for a given (shared) graph, it has a valid 3-coloring. It wants to convince the verifier of this without revealing the actual color assignment.

As a quick recap, graph coloring is the problem where given a graph, we find a way to assign each node a color such that no two adjacent nodes have the same color. 3-coloring is coloring with at most 3 colors.

How do you do this? Assorted blog posts and fancy-looking demonstrations were interesting but did not help us understand much.

Chris and I went around in circles for a bit until we decided to take a look at one of the original papers (PDF) by Goldreich, Micali, and Widgerson. We only really read page 23 (labeled page 713 in the PDF) but that was enough to get things going.

The paper’s protocol

Protocol 4 from the paper describes an interactive 3-color proof session between the prover (P, with numbered steps) and the verifier (V, with numbered steps), reproduced here:

common input A graph G(V, E) (n = |V|, m = |E|).

The following four steps are executed m² times, each time using independent coin tosses.

(P1) The prover chooses at random an assignment of three colors to the three independent sets induced by φ, colors the graph using this 3-coloring, and places these colors in n locked boxes each bearing the number of the corresponding vertex. More specifically, the prover chooses a permutation π ∈R S₃, places π(φ(i)) in a box marked i (∀ i ∈ V), locks all boxes and sends them (without the keys) to the verifier.

(V1) The verifier chooses at random an edge e ∈R E and sends it to the prover. (Intuitively, the verifier asks to examine the colors of the endpoints of e ∈ E.)

(P2) If e = (u, v) ∈ E, then the prover reveals the colors of u and v by sending the verifier the keys to boxes u and v. Otherwise, the prover does nothing.

(V2) The verifier opens boxes u and v using the keys received and checks whether they contain two different elements of {1, 2, 3}. If the keys do not match the boxes, or the contents violate the condition then the verifier rejects and stops. Otherwise, the verifier continues to the next iteration.

If the verifier has completed all m² iterations then it accepts.

We’ll come back to the number of iterations. For now let’s try to just do one iteration. For each step, I’ll annotate the code with “Only prover” or “Only verifier” so that it’s clear who can see what data.

One iteration

We’ll start by sketching out what it means to have a graph. For the example graphviz graph above, we have the following edge list data structure:

# Shared between prover, verifier
edges = [(0, 1), (1, 2), (2, 3), (3, 4), (4, 0), (0, 2)]

Each tuple in the list represents a connection between two numbered nodes. Fancy stuff. Because it’s an undirected graph, (0, 1) means the same as (1, 0) so we don’t have to include both. We can also color it:

# Only prover
coloring = {0: "navy", 1: "darkgreen", 2: "crimson", 3: "navy", 4: "darkgreen"}

Each key is a node number and each value is a color.

Though finding a 3-coloring of a graph is slow, verifying one is fast—linear in the number of edges. Let’s verify that we have a valid sample coloring:

# For the reader
# Check each edge to make sure no edge has the same color on each node
assert all(coloring[u] != coloring[v] for u, v in edges)
# Check that the total number of colors used is 3
assert len(set(coloring.values())) <= 3

We’ll now go through the paper’s steps one by one, writing some code to accompany each step.

Tips and tricks

If you are building alongside the blog post, I recommend using random.seed(0) so your randomness doesn’t change between runs of your program. I also recommend setting the environment variable PYTHONHASHSEED to 0 if you are using hash for the same stability reasons.

Step P1

The first thing we need to do is permute the coloring we have. That is, we should swap around the color values while maintaining the 3-color property.

Thankfully, this is easier than it might sound: the names of colors are meaningless to the 3-coloring; they need only be different along the edges. So if we do a bijective (A maps to one B, and B came from one A) mapping from old to new name, this will hold.

I came up with this function that shuffles the colors, lines them up side-by-side, makes a table, and then uses that to make a new coloring:

import random

# Only prover
def permute_three_coloring(coloring):
    all_colors = list(set(coloring.values()))
    new_colors = random.sample(all_colors, len(all_colors))
    permutation = {old: new for old, new in zip(all_colors, new_colors)}
    return {node: permutation[color] for node, color in coloring.items()}
# For example,
# {0: "crimson", 1: "navy", 2: "darkgreen", 3: "crimson", 4: "navy"}

Then we have to place the colors in “locked boxes”. One way to proverbially lock a box is to apply a one-way function to it: for example, a hash function. If we hash each color and then pass only the hashes to the verifier, the verifier cannot open them.

This example uses the Python standard library hash function for brevity but it might be better to use a cryptographic hash function like hashlib.sha256:

# Only prover. Wrong!
def hash_coloring_wrong(coloring):
    return {node: hash(color) for node, color in coloring.items()}
# For example:
# {0: -6789624683659967261, 1: 7846608853949633950, 2: 6009240650600289446,
#  3: -6789624683659967261, 4: 7846608853949633950}

There’s just one problem with handing these locked boxes to the verifier: two boxes locked with the same colors will have the same hashes. The verifier would know the coloring. Even if the exact colors are now hidden, it’s really the structure of the coloring for which we want to give up zero knowledge.

To get around this, we can add what’s called a nonce to each node and its coloring. That is, each node gets a little bit of random data packed into the hash so that different nodes’ "darkgreen" hash values look different.

# Only prover
def nonce():
    return random.randrange(100)

def box_coloring(coloring):
    return {node: (color, nonce()) for (node, color) in coloring.items()}

def hash_values(coloring):
    return {k: hash(v) for (k, v) in coloring.items()}

permuted_coloring = permute_three_coloring(coloring)
# For example:
# {0: "crimson", 1: "navy", 2: "darkgreen", 3: "crimson", 4: "navy"}
boxed_coloring = box_coloring(permuted_coloring)
# For example:
# {0: ("crimson", 33), 1: ("navy", 65), 2: ("darkgreen", 62),
#  3: ("crimson", 51), 4: ("navy", 38)}
hashed_coloring = hash_values(boxed_coloring)
# For example:
# {0: -2275004828450249492, 1: 2227921633151400991, 2: -5024343381376265886,
#  3: -5381005702768533635, 4: 1164729608819214729}

Again, you probably don’t want to use the standard library random number generator for your nonces. You should consider something like secrets.token_hex() from the secrets module (Python 3.6+). Maybe even consider using the hmac module.

Finally, we can send the hashed_coloring to the verifier and begin step V1.

Step V1

Since the verifier knows the graph (but not its colors), it can pick an arbitrary edge to inspect. It wants to verify that the arbitrary edge it picked satisfies the 3-color conditions. It sends off a request for an arbitrary edge e:

# Only verifier
e = random.choice(edges)
revealed = prover_please_reveal_colors(e)

This is implicitly relying on some global state (the prover knowing what “session” is active with the verifier). If you have multiple verifiers or concurrent sessions or something, you may need to thread through some context identifier in the communication.

Step P2

The prover, having received this request, sends over the colors and nonces for each of the nodes in the edge.

# Only prover
def prover_please_reveal_colors(edge):
    u, v = edge
    return {u: boxed_coloring[u], v: boxed_coloring[v]}
# For example:
# {3: ('crimson', 51), 4: ('navy', 38)}

You may be suspicious at this point because we’re leaking some information about the coloring.

Note that it’s ok for the prover to reveal the color for one edge, because 1) the colors have been shuffled once per round and 2) we’re going to apply our box locking protocol each time we reveal an edge (also once per round), so the verifier accumulates no information about our colors between iterations.

Step V2

The verifier can check that the color+nonce hashes to the hash value given for each node in step P1. This ensures that the prover is not changing colors around mid-round. This relies on the verifier and the prover using the same hash function (and the same hash seed if using hash).

# Only verifier
for (node, (color, nonce)) in revealed.items():
    assert hashed_coloring[node] == hash((color, nonce)), f"Hash mismatch!"

The verifier can then inspect that the two color values are different. This gives a small amount of credence (1/|E| because you know something about one edge now) that the graph is 3-colored appropriately because the prover had no way of knowing which edge the verifier would want to inspect.

If either of these two conditions doesn’t check out, the verifier rejects.

Probabilities

You have to do at least a couple of rounds of this for the verifier to believe the prover about the 3-coloring.

The paper goes on to assert that “the probability that the verifier will accept (i.e., complete all the m² rounds without detecting that “something is wrong”) is bounded above by (1 - m⁻¹)^(m²)” (where m = |E|). Which is pretty good. For a large graph (say, 1000 edges), this falls off reasonably quickly:

m = 1000
for i in range(1, 4600):
    print("(1 - m⁻¹)^round = ", (1 - m**-1)**i)

At 4600 rounds, you’re at 1% possibility of “cheating”. At 10,000 rounds, you’re at 0.0045% possibility of “cheating”. At m² = 1,000,000 rounds, it’s very low.

A networked demo

Writing Python code that runs in a single process with comments denoting “prover” and “verifier” is not very satisfying. It does not preclude accidental data leaking in the slightest. It would be much more satisfying if there was some sort of barrier, like a process barrier or a network barrier, between the prover and the verifier.

For this reason, Chris and I have prepared a server (prover) and client (verifier) demo. You can click “Run Round” to run a round (and show the permuted colors). You can visit the docs page to see the API docs and build a client yourself!

(If you don't see a graph right below this message, please wait a moment for the server to wake up.)

Encoding other NP-complete problems

At this point, we’ve shown how you can use a zero-knowledge interactive proof to verify that someone has a valid 3-coloring of a graph without learning any information about the 3-coloring. So what? Is there anything else we can prove with zero knowledge? Is the interactive proof of 3-coloring just a contrived party trick without real applications?

Well…

Sudoku

Sudoku puzzles are another example of a problem that’s hard to solve and easy to verify. It is algorithmically hard to fill in 81 squares to satisfy the constraints of all rows, columns, and boxes containing the digits 1-9, but the verifier is extremely quick.

Let’s say we wanted to prove we’ve finished a Sudoku, but we don’t want to give up a morsel of information about the solution. We can execute an interactive proof very analogous to 3-coloring! Instead of shuffling colors, we shuffle digits. Instead of revealing edges, we reveal rows, columns, and boxes.

Next time your fellow bus rider leans over asking to see your Sudoku solution, just ask them to go through 90,000 easy steps first!

Reduction

Let’s say we have a hard problem and we’ve computed a solution it, but we don’t have an obvious algorithm on hand to execute an interactive proof for it. Thanks to the authors of the paper above, we know that if the problem is NP-complete there’s an interactive proof for it!

We use the power of a polynomial time reduction. We (somehow) convert our solution to a graph and its 3-coloring, then just follow the steps and code above! The “somehow” is the tricky part, but much research exists on converting between different NP-complete problems.

For example, you may want to create a zero-knowledge proof that you know the prime factors for a very large composite number. Unfortunately, for only double digit numbers your graph is thousands of nodes, so reducing to 3-coloring has its limits in practice and you’re better off with more sophisticated proof techniques than a reduction to 3-coloring.

Wrapping up

After doing a bit of research, we decided that the most common real world use cases of zero-knowledge proofs (age verification, crypto, etc) aren’t particularly interesting to us. We enjoyed the graphs and theory of computation and networked computing though. We hope you had fun playing around with interactive proofs too.

The inliner is yielding benefits for ZJIT

Tue, 28 Jul 2026 00:00:00 +0000

Originally published on Rails At Scale.

We recently enabled a really cool feature in ZJIT that makes it feel like a Real Compiler™: the inliner! We’ll write more about it soon. In this post, we’ll talk about one excellent concrete benefit we are already seeing and how it optimizes blocks in pretty much every Ruby program.

I’ll start off with a refresher on how blocks work in the Ruby interpreter, then show you how ZJIT understands and optimizes that bytecode, and then show you the impact of the inliner.

Ruby: a refresher

In the beginning, there were loops.

arr = [1, 2, 3]
i = 0
sum = 0
while i < arr.length
  sum += arr[i]
  i += 1
end

People used them to navigate and manipulate variable-length structures, like arrays and strings. This was fine.

Then, in the 1970s, a small group of computer scientists at Palo Alto Research Center invented a programming language called Smalltalk. One of the core features of Smalltalk was that everything was an object and computation was done by sending messages to objects.

This meant that iteration wouldn’t do at all. Instead, we would have to send the do: message to the array object and pass it a block object.

#(1 2 3) do: [:a | sum := sum + a].

Then, in the 1990s, Matz, inspired by Smalltalk and Perl, created Ruby. We still have “normal” loops but we also have a very Smalltalk-y way of doing it, too:

arr = [1, 2, 3]
sum = 0
arr.each do |a|
  sum += a
end

When this program gets compiled to Ruby bytecode, it ends up looking like a mostly normal method call to each except that we pass a special kind of argument to it: a block argument.

To see how this works inside CRuby, we’re going to look at a listing of YARV bytecode—CRuby bytecode. For more on YARV, I recommend Kevin Newton’s excellent Advent of YARV.

Ignore most of the bytecode dump below except for the instruction at 0009, the send instruction. We are send-ing (see? a message!) each with the block argument block in <main> (passed a different way than “normal” arguments, hence the argc:0).

$ ruby --dump=insns /tmp/arrayeach.rb
== disasm: #<ISeq:<main>@/tmp/arrayeach.rb:1 (1,0)-(5,3)>
local table (...)
[ 2] arr@0      [ 1] sum@1
0000 duparray                   [1, 2, 3]                 (   1)[Li]
0002 setlocal_WC_0              arr@0
0004 putobject_INT2FIX_0_                                 (   2)[Li]
0005 setlocal_WC_0              sum@1
0007 getlocal_WC_0              arr@0                     (   3)[Li]
0009 send                       <calldata!mid:each, argc:0>, block in <main>
0012 leave
...continued below...

This block in <main> from the bytecode listing above is the generated name of the instruction sequence (bytecode) corresponding to the block we passed to each. Its code, shown below, is the next thing in the bytecode dump. You can see the usage of local variables sum and a via getlocal and setlocal variants and also opt_plus, which represents addition (+). Again, the details are not terribly important:

...continued from above...
== disasm: #<ISeq:block in <main>@/tmp/arrayeach.rb:3 (3,9)-(5,3)>
local table (...)
[ 1] a@0<AmbiguousArg>
0000 getlocal_WC_1              sum@1                     (   4)[LiBc]
0002 getlocal_WC_0              a@0
0004 opt_plus                   <calldata!mid:+, argc:1, ARGS_SIMPLE>[CcCr]
0006 dup
0007 setlocal_WC_1              sum@1
0009 leave                                                (   5)[Br]
$

To make this work, Array has an each method. This each method takes in its optional block argument and calls it once for every element in the array.

As with most of the core data structures, Array’s methods tend to be written in C and each is no different. Here is its nice and short definition with comments added by me:

VALUE
rb_ary_each(VALUE ary)
{
    long i;
    // Some debug-mode-only checks
    ary_verify(ary);
    // If we didn't pass a block in, return an enumerator object
    RETURN_SIZED_ENUMERATOR(ary, 0, 0, ary_enum_length);
    // We passed a block in, so call it for every element
    for (i=0; i<RARRAY_LEN(ary); i++) {
        rb_yield(RARRAY_AREF(ary, i));
    }
    return ary;
}

You can’t really see the block parameter to the C code because it’s passed in a special location on the Ruby VM’s own stack called the block handler. All you need to know is that rb_yield knows where to find that and how to call it.

There’s just one more thing, which is… hang on, weren’t we building a JIT to optimize Ruby code? How are we going to optimize this C code?

JITs: a refresher

Rewriting code from C to Ruby means that the JIT compiler gets a chance to introspect the run-time behavior and code. This means that, over time, as the compiler and the runtime system grow together, more and more code might get rewritten in Ruby. This started happening a couple of years ago with YJIT.

YJIT precipitated some interesting changes to Ruby VM internals. For example, in 2022, Array#each being written in C started to hurt: it was an opaque blob that the JIT couldn’t reason about. So Kokubun submitted a PR to rewrite it in Ruby. After some back and forth, in 2024, Kokubun landed a different PR that everyone was happy with.

class Array
  def each # :nodoc:
    Primitive.attr! :inline_block, :c_trace, :without_interrupts

    unless defined?(yield)
      return Primitive.cexpr! 'SIZED_ENUMERATOR(self, 0, 0, ary_enum_length)'
    end
    i = 0
    until Primitive.rb_jit_ary_at_end(i)
      yield Primitive.rb_jit_ary_at(i)
      i = Primitive.rb_jit_fixnum_inc(i)
    end
    self
  end
end

Ah, finally. A version that JITs can reason about.

I keep saying “reason about” and what that means concretely here is a) that it’s written in a format that the JIT can ingest and optimize: Ruby, and b) mostly a brief rehashing of the key lessons in the venerable Smalltalk (!) paper Efficient Implementation of the Smalltalk-80 System (PDF):

Though systems may offer very dynamic behavior, people don’t frequently make use of wild features all over the place
- Most people pass fewer than 4 types of objects through a given method
- As a corollary, even if there are many classes in a system, code locality is super important, and we can take advantage of that
- Also, most people do not define, re-define, and otherwise continuously modify method definitions

(And if you don’t believe me that the lessons still apply, check out the excellent paper Who You Gonna Call (PDF) by Sophie Kaleba, Octave Larose, Richard Jones, and Stefan Marr.)

So JITs like to watch what types of objects flow through methods before compiling them. And JITs like to, since they know the types of objects, cache method lookups and specialize method invocations on those objects.

For example, take a look at this code:

def next_capacity(x)
  x.length * 2 + 1
end

x could be anything. There is no way of knowing its type by looking at the code. This means that the length method could be anything. Furthermore, there is no way of knowing what the return type of x.length is, so we can’t specialize the method lookups or invocations of * or + either.

But! Per our lessons above, likely only a few types flow through this code. Say the JIT’s profiler notices that x has historically been an Array. Then we might reasonably assume that it will continue to be an Array, so when we compile the method, we add a run-time type check: if the type is no longer an Array, jump back into the interpreter.

Let’s see what optimized code ZJIT can construct by combining profiling information with the above bytecode. ZJIT operates on its own high-level intermediate representation called, uncreatively, HIR. In the following HIR snippet, we can see this very run-time type check (“guard”) for the Array class:

fn next_capacity@file.rb:8:
...
bb3(v9:BasicObject, v10:BasicObject):  # v9 is self and v10 is x
  PatchPoint NoSingletonClass(Array@0xdecaf1)
  PatchPoint MethodRedefined(Array@0xdecaf2, length@0xdecaf3, cme:0xdecaf4)
  v35:ArrayExact = GuardType v10, ArrayExact recompile
  v36:CInt64 = ArrayLength v35
  v37:Fixnum = BoxFixnum v36
  # ...

(with real pointers replaced by fake ones for readability)

Because classes are, among other things, collections of methods, this type information tells us what the call target of x.length is: it’s Array#length! We have a special fast code snippet to read an array’s length so we do that instead of a method call.

And in case the methods ever get changed out from underneath us, we leave behind these markers called PatchPoint that invalidate the code.

Finally, because we know that the result of Array#length is always a small integer (Fixnum), we can special case the method lookups for * and + as well.

fn next_capacity@file.rb:8:
...
bb3(v9:BasicObject, v10:BasicObject):  # v9 is self and v10 is x
  PatchPoint NoSingletonClass(Array@0xdecaf)
  PatchPoint MethodRedefined(Array@0xdecaf, length@0xdecaf, cme:0xdecaf)
  v35:ArrayExact = GuardType v10, ArrayExact recompile
  v36:CInt64 = ArrayLength v35
  v37:Fixnum = BoxFixnum v36
  v18:Fixnum[2] = Const Value(2)
  PatchPoint MethodRedefined(Integer@0xdecaf, *@0xdecaf, cme:0xdecaf)
  v41:Fixnum = FixnumMult v37, v18
  v23:Fixnum[1] = Const Value(1)
  PatchPoint MethodRedefined(Integer@0xdecaf, +@0xdecaf, cme:0xdecaf)
  v45:Fixnum = FixnumAdd v41, v23
  CheckInterrupts
  Return v45

There you have it. This is how JITs work: observe, assume, specialize. So why am I telling you all this? How does this circle back to blocks?

Well, blocks work not quite the same way, but similarly. Instead of having an object that we call a method on, we just have the target instruction sequence¹. But if we observe that the block argument, in its special location, has been consistently one object, we can specialize the call to it.

This is more or less fine for some cases of code. For example, in the following code snippet, we only have one caller to custom_iterator, so its profiled block will be monomorphic (one observed shape).

def custom_iterator
  yield 1
  yield 2
  yield 3
end

custom_iterator do |x|
  puts x
end

This reinforces what we know and love about the Smalltalk-80 paper: code locality wins! Yes!

But unfortunately this falls apart when we start thinking about all of the core library methods (and potentially the methods and classes you have stashed away in the grab-bag utils.rb in your application). Those methods are probably megamorphic (many many observed shapes). They probably see all sorts of stuff because they are general-purpose utilities that everybody needs, all the time.

One such example in the Ruby core library is the venerable Array#each that we saw earlier. Because there are a million different call sites to Array#each across your application and each probably passes a totally different block, we’re in an unhappy situation. How can we possibly optimize for so many different blocks? What happened to our code locality? How do we fix this?

The code locality is in another tower

It’s okay. Code locality still rules. Look at all the various callers of Array#each. They all pass a different, but constant², block at the call-site:

def method_a(some_array)
  some_array.each do |x|
    puts x
  end
end

def method_b(some_array)
  some_array.each do |x|
    puts x*2
  end
end

def method_c(some_array)
  some_array.each do |x|
    puts x*4
  end
end

So the code locality that we need in Array#each to optimize the code is one level up at the caller. If we can use that call context, we can specialize the code.

YJIT accomplishes this by splitting, which is a very natural transformation for basic block versioning and tracing. Such compilers are good at following code paths as they would be executed and putting together context across method calls. However, YJIT’s heuristic for splitting blocks is based on manual annotations: it will only kick in for certain Ruby library functions specially annotated with Primitive.attr! :inline_block (and the name is a bit of a misnomer).

The team that builds ZJIT, a method JIT, decided not to add splitting facilities. We could split methods (and blocks), but we have an easier time reasoning about larger code units than YJIT does because we optimize an entire method at once. So instead, ZJIT chooses to get call context by inlining.

Inlining

Method inlining refers to copying the body of the callee into the caller. In the above example, it means copying the body of Array#each into each of method_a, method_b, and method_c. I don’t mean the Ruby code and I don’t mean the bytecode: I mean the HIR. ZJIT does this by building the HIR of the callee (Array#each) into the existing HIR of the caller (method_a, …).

The illustrious Kevin Menard wrote ZJIT’s inliner and he’ll write a post about all the details soon. It’s pretty interesting stuff.

For now, we can take a look at the (lightly edited) result of Array#each being inlined into method_a. The details don’t matter, but there are two things I want to call out:

What used to be a dynamic invokeblock is now what we call InvokeBlockIseqDirect because we know from the call context what block method_a is passing to Array#each.
What used to be a method call is now a loop: the condition check is in bb10, the body is in bb9, and the stuff after the loop is bb13/bb4. If you’re interested, try to find the iteration variable and where it gets incremented. See the footnote³ for the answer.

This is a massive improvement over the previous very generic operations.

fn method_a@/tmp/arrayeach.rb:2:
# ...
bb3(v9:BasicObject, v10:BasicObject):
  # ...
  v55:Fixnum[0] = Const Value(0)
  Jump bb10(v25, v55)
bb10(v68:ArrayExact, v69:Fixnum):
  v73:CInt64 = ArrayLength v68
  v74:Fixnum = BoxFixnum v73
  v75:BoolExact = FixnumGe v69, v74
  v77:CBool = Test v75
  CondBranch v77, bb13(), bb9(v68, v69)
bb13():
  CheckInterrupts
  Jump bb4(v68)
bb9(v90:ArrayExact, v91:Fixnum):
  v96:CInt64 = UnboxFixnum v91
  v97:BasicObject = ArrayAref v90, v96
  v99:CPtr = GetEP 0
  v100:CInt64 = LoadField v99, :VM_ENV_DATA_INDEX_SPECVAL@-0x8
  v101:CInt64[-4] = Const CInt64(-4)
  v102:CInt64 = IntAnd v100, v101
  v103:BasicObject = InvokeBlockIseqDirect (0x16f3d5968), v102, v97
  v107:Fixnum[1] = Const Value(1)
  v108:Fixnum = FixnumAdd v91, v107
  PatchPoint NoEPEscape(each)
  Jump bb10(v90, v108)
bb4(v114:BasicObject):
  PopInlineFrame
  PatchPoint NoEPEscape(method_a)
  CheckInterrupts
  Return v114

You may notice that there is still one call in the loop: the call to the block (InvokeBlockIseqDirect). That’s next on our list to tackle. We are optimistic that we will soon also be able to inline block calls. Then the whole thing will really be just a loop!

The code that enables us to reason about which block got passed into the inlined callee (Array#each) only landed a couple of days ago (July 10, 2026), written by Luke Gruber. This was one of Luke’s first changes to ZJIT.

So what does performance look like?

Well, for starters, the microbenchmarks that we use as “performance unit tests” for specific aspects of Ruby went wild. Some benchmarks that were using block-based looping got much faster; they were previously bounded by ZJIT’s block invocation performance.

Take a look at the cfunc_itself benchmark, which tests that we can fold away the call to Ruby’s built-in itself method. ZJIT ends up optimizing the times method to invoke the do block directly, and the do block gets optimized to nothing but a guard on the self’s class to make sure it hasn’t changed.

run_benchmark(500) do
  500000.times do |i|
    itself
    itself
    itself
    itself
    itself
    itself
    itself
    itself
    itself
    itself
  end
end

Because we had previously optimized the body away, the result of the direct block invocation is a massive speedup:

The yellow line represents ZJIT speedup over the interpreter. Higher is better. 35x!

Other benchmarks also kind of stop making sense because of the amount of inlining. Our bmethod benchmark, which benchmarked how fast we can call methods defined with define_method, also stopped measuring anything of use. We’re going to have to rework the benchmark to be more fair…

The yellow line represents ZJIT speedup over the interpreter. Higher is better. 139x!

As expected, larger Rails benchmarks don’t see a ton of change; they exercise a diffuse set of features so optimizing any one feature bumps the big benchmarks only a little bit.

I am excited to see what happens when we can fully turn each and friends into call-less loops!

Wrapping up

Code locality rules. The inliner helps inject more of it and reason across method calls. ZJIT, a little over one year old, is growing up! :’)

We’re still tuning the inliner knobs. Some of the code in this post required tweaking --zjit-inline-threshold to convince the compiler to inline Array#each into method_a because of Array#each’s size. It will take some time for the ZJIT developers to figure out reasonable defaults.

Try out our HIR explorer at tryzjit.fly.dev. Try out ZJIT in your application by adding the --zjit flag to a Ruby over 4.0. Thanks for reading and see you next time.

Mostly. I am glossing over nil, procs, ifuncs, etc. But the common path is by and away iseq blocks. ↩
It’s not always the case that these methods are called with a constant block iseq. Sometimes they are called with the &:symbol form, for example. Or with the &some_block_variable form. In that case, we can use an inline cache to (with a guard) make it constant once more. But we have not implemented that yet, because it is rarer. ↩
The iteration variable is v91! You can see it get used as an array index when it gets unboxed as v96 and passed to ArrayAref. You can also see it get incremented with FixnumAdd and the constant 1. In bb10 it has a different name, v69, which gets checked against the array length. ↩

Travel notes: PLDI Boulder

Sun, 05 Jul 2026 00:00:00 +0000

I had another excellent PLDI this past June. It was my fourth¹. I continued to meet new people and learn new things!

Overall: I got to meet a lot of new people, which was exciting. I had some good chats about research. I asked a question at a talk! I got to show Aaron and Jacob PLDI and see them enjoy it. I missed hanging out with CF Bolz-Tereick and Chris Fallin, the usual suspects at conferences I attend. I’m looking forward to next year.

This post is more about the conference than the town of Boulder (unlike the last PLDI post about Seoul) because I didn’t do much Boulder exploring.

Sunday

I got in late on Sunday. Then I had to take a long car ride from Denver airport to Boulder. I don’t think I had ever flown into Denver with intent to go to Boulder before so it was a bit of a surprise.

Jacob offered to have a late dinner with me so we had a tasty meal at Gaia Masala and Burger. Shout out to Harry, our server.

Monday

Monday was a workshop day. I signed up for EGRAPHS and mostly stayed in that workshop. People kept throwing around the term “Knuth-Bendix”, as they have been for several years, and only in one of these workshop talks did someone explain it in a way that made any sense at all. It seems kind of like equality saturation but for the rewrite rules themselves—no actual expression graphs involved. I DMed Phil this sketchy explanation during a talk to get his response and I got to watch him cock his head and think about it in real time.

At lunch I met Qiantan Hong and we got to talking about Common Lisp and its object system, CLOS. Seems like a combination of ahead-of-time compilation and multiple dispatch is really tricky.

I had dinner with Aaron at Postino and then wandered into a bunch of people staying at the conference hotel chatting in the lobby. Ben Titzer said “fix my subtyping bug”, which I interpreted as him saying hi. I ended up just planting myself at the table as a bunch of interesting people cycled through: Jared Roesch, Mae Milano, Hila Peleg, Russel Arbore. It was a late evening.

Tuesday

Back to the workshops! But late because of aforementioned late evening. I saw Vadym’s talk about Remora. I only understood about 40% of it but it was good to catch up. I hadn’t seen him since leaving Northeastern.

Around a break time I joined a little cluster of people talking about e-graphs and I guessed asked enough basic questions that Pavel convinced Max Willsey to run a “BYOEG” (build your own e-graph) tutorial. The structure was as follows: Max would instruct Jacob as to what kind of thing to build next but not be prescriptive about exactly how to build it and not look at Jacob’s screen. The rest of us would sit around a table and try to follow along as best we could. I hear Pavel has a blog post about this experience coming soon…

I saw Slava Pestov walking around and introduced myself because we keep liking one others’ bad jokes on Mastodon. We ended up getting dinner with Aaron and Jacob that night at Leaf. We learned a lot about monoids, Knuth-Bendix (!), Factor, and Swift. Slava volunteered to do a similar follow-along “BYOKBC” (build your own Knuth-Bendix completion) tutorial the next day.

Wednesday

First day of the conference! I was walking into the hotel in the morning and I had made it about three feet onto the property when Alexa VanHattum, who was going the opposite direction, convinced me to instead get coffee elsewhere. We had a nice catch up and I got to hear about what teaching is like these days.

Lunch was fun. I got to do another round of “ambush person whose research I admire” and plopped down with Ben and Christian Wimmer. I’d spent a lot of time struggling with Christian’s papers on linear scan, then convinced him to chat about register allocation with us on a video call a couple of months ago. We continued some of that at lunch but then I (kind of accidentally kind of on purpose) got Ben started talking about Sea of Nodes and how it is and is not different between Java and (for example stand-in for dynamic languages) JavaScript. Apparently he is thinking about a similar thing that he is calling Sea of Variables.

We talked about inlining challenges and how to infuse profiles with call context, which can be a challenge. I feel more inspired to get type-based alias analysis working in ZJIT.

I tracked down Christian later in the courtyard and got to hear about what he’s working on these days. I know very little about ML compilers and ML hardware and things like that so hearing about the challenges was neat. Yannis Smaragdakis joined our little standing table chat and we got to learn about Datalog. Because I had previously written about linear scan register allocation and about liveness analysis with Datalog, I goaded him into pairing with me on writing a full linear scan implementation in Datalog.

This ended up taking the rest of the evening and several beers and then a lot of the next day! And after the first bit of code I did not manage to contribute very much at all.

Thursday

I met Hannah Gommerstadt and we got to chatting about bikes and formal methods (separately). Slava walked by and I got to introduce them. Then Jacob too.

I continued pairing with Yannis but remained really lost. The only thing I think I contributed was some familiarity with the core algorithm, which he had only really seen in passing before. Eventually he got it fully working, but it needed some deep trickery. More on this soon in its own post.

I saw a talk about versioned e-graphs and that got me wondering if their implementation can be used as a persistent e-graph or even just persistent union-find. Sometimes you want to do backtracking, or have undo-redo in your compiler.

Then I went to a talk about streaming byte-pair encoding (BPE). BPE is hard to do streaming because it definitionally requires looking over the whole input string. They did some neat trickery to find boundaries in the string that demarcate regions that don’t interfere with one another and thereby tokenize on-the-fly. I didn’t understand it fully but I asked my only question of the conference, which was if this could also be used to implement BPE in parallel. Seems the answer is “maybe” so I should probably reach out and ask further.

Slava started showing me and Jacob and Aaron how to implement Knuth-Bendix completion for strings. I had a lot of tiny little bugs which slowed progress. Such is life. The banquet and awards ceremony started so we called it a night and went off to eat dinner. They had good lentils.

Friday

I ran into Thalia Archibald and John Regehr and we talked about (really, they talked about and I tried to learn something) what it might mean to either port Alive2 to another compiler than LLVM, or build “Alive3”, or “Mini-Alive” for some other IR. John suggested fuzzing the hell out of the thing first, then doing something more formal later… especially if it’s a dynamic language IR where a lot of the opcodes end up being “function call that can do anything”.

I had a nice chat with Steven Holtzen and Zach Tatlock about research and grad student life. I got some good advice. I meant to talk to Zach about this thing we keep occasionally chatting about that I call “the big e-graph in the sky”. I talked a bit about it to Max Willsey and he had some good probing questions about what would be slow, challenging, or somehow undefined given my problem statement.

I continued struggling to implement Knuth-Bendix with significant assistance from Slava and I think eventually got something working. I had a really nasty bug due to string slicing semantics² in Ruby.

Aaron went off to learn about deep immutability in Python and then got to chatting with the authors of the paper. We got to compare notes about language and language implementation challenges. It’s been a long time since I was in Python-land.

Aaron and Slava and I got enchiladas for dinner.

Saturday

I had no reading material for the flight so I went downtown, intending to buy one book, but got too many books. They barely fit into my bag for the flight home. I started reading Anathem for the second time. It holds up. It’s a damn good book.

Actually, it might be my fifth. I just remembered that I attended PLMW in 2020 and also watched a few online talks at wild hours from my living room. ↩
The semantics in Ruby are probably globally reasonable but did not fit the thing I was trying to do: if we have two strings a and b, we want to find the index at which they start to overlap, overlap. Then we want to grab the bit of a that is to the left of overlap. I had initially written that as a[..overlap-1].

However, if a and b overlap at the start of a, overlap is 0. This generates the range ..-1, which means we’ll slice until the end of a.
```
irb(main):001> "abc"[..-1]
=> "abc"
irb(main):002>
```
Not what we want. Instead, the fix in the commit shows how I had to add a special slice function called string_upto that handles the 0 case. ↩

A survey of inlining heuristics

Wed, 03 Jun 2026 00:00:00 +0000

Compilers, especially method just-in-time compilers, operate on one function at a time. It is a natural code unit size, especially for a dynamic language JIT: at a given point in time, what more information can you gather about other parts of a running, changing system?

I don’t have any data to back this up—maybe I should go gather some—but on average, methods are small. Especially in languages such as Ruby that use method dispatch for everything, even instance variable (attribute, field, …) lookups, they are small. And everywhere.

This makes the compiler sad. If we are to continue to anthropomorphize them, compilers like having more context so they can optimize better. Consider the following silly-looking example that is actually representative of a surprising amount of real-world code:

class Point
  attr_reader :x, :y

  def initialize(x, y)
    @x = x
    @y = y
  end

  def distance(other)
    Math.sqrt((@x - other.x)**2 + (@y - other.y)**2)
  end
end

def distance_from_origin(x, y)
  Point.new(x, y).distance(Point.new(0, 0))
end

Right now, in the distance_from_origin method, I count 8 different method calls:

Point.new
Point#initialize
Point.new
Point#initialize
Point#distance
Float#**
Float#**
Math.sqrt

(Technically more, but the ivar lookups (including attr_reader!), addition, and subtraction are generally specialized and don’t push a frame, even in the interpreter.)

Furthermore, there are at least two heap allocations: one for each Point instance.

Last, there is a bunch of memory traffic to and from Point instances.

This all is a huge bummer! What should be a simple math operation is now overwhelmed with a bunch of other stuff. Point is certainly not a zero-cost abstraction.

Even if we had a bunch of other optimizations such as load-store elimination or escape analysis, they would not be able to do much: pretty much everything escapes and is effectful. That is, unless we inline. Inlining is the lever that enables a bunch of other optimization passes to kick in.

Inlining: the “easy” part

I wrote about the design and implementation of Cinder’s inliner (FB link, personal blog link) a couple of years ago. I wrote about arguably the simplest part, which is copying the callee body into the caller. It took me at least a week to get working. Probably closer to months if you consider all the plumbing through the rest of the JIT. In February during a small hackathon, I watched my colleague k0kubun prototype that bit of the inliner inside ZJIT in about 30 minutes.

There is more to do when pretty much every part of the VM is observable from the guest language: both Python and Ruby allow inspecting the state of the locals, the call stack, etc from user code. Sampling profilers also expect some amount of breadcrumbs to work with to inspect the stack. So there’s some more machinery still required to pretend like the callee function was not inlined. I talk about this a little bit in the Cinder blog post.

Even so, all of that can probably be designed and wired together in a couple of months. Then you will find yourself tuning the inliner for the next 10 years. This is much harder.

When: the harder part

The thing that makes inlining difficult, especially in a method JIT, is that you are trying to make an entire (dynamic!) system faster but you are only looking through a microscope and only capable of local reasoning¹. Whereas other optimizations such as strength reduction, inline caches, and value numbering are an un-alloyed good for the generated code, inlining can have negative effects. It is also perhaps the first optimization people add that has non-local impact.

If you inline wrong, your code size might blow up. This might thrash your CPU’s caches. Bummer, but happens to the best of us.

But also, if you inline wrong, you might get in the way of other helpful optimizations: if you hit some size limit after inlining method A, you might never get to inline B, which is the key to unlocking the performance of the method you are trying to optimize.

Last, inlining might hurt compile time. In situations where latency is paramount (think: interactive client JavaScript), adding tons more code into the fray might add noticeable hiccups, even if the long-term throughput improves. As always, in-band compilation is a trade-off because any time you spend compiling, you are not executing code.

You have to write your compiler to reason about all of this stuff. So you have heuristics. For example, here is Michael Pollan’s inliner heuristic:

Inline methods. Mostly small. Not too many.

I did a survey of a bunch of compilers, mostly JIT compilers, to see what their inlining heuristics look like. I also read (skimmed) some papers to see what those folks had to say. I wonder if they agree.

This post was a long time coming. I started working on it about five years ago but then when I quit working at Facebook I accidentally left behind all of the inliner research I did for Cinder’s inliner. So then I kind of just thought about it aimlessly for a while before redoing it this year. Anyway, here’s wonderwall.

The heuristics

Spoiler alert: all in all, people tend to look at:

Profiles of call target
Cumulative caller size (increasing as callees get inlined)
Callee size
Inline depth
Number of inlined calls at a certain depth
If recursion is present
Callee/caller call count ratio (if callee only called less than K% of calls to caller, don’t inline callee)
Callee stack usage
Polymorphism in callee
What mode the compiler is in (baseline vs more aggressive)
If the callee looks like it always raises/throws

And also have different interesting ways to pipe in profile information.

Last, some newer papers do some wild stuff:

Train neural networks to make inlining decisions
Let inlining drive the entire optimization pipeline, treating it as a search heuristic over a BFS walk of the call graph
Use AOT-gathered information to aid in JIT heuristics

Another thing to consider in inlining is how you gather and interpret profiles.

Call context and profiles: the other harder part

When you compile a function, you tend to specialize it based on the input it has historically been given. For a monomorphic input, maybe you guard that the type is still the same and otherwise jump into the interpreter. For a polymorphic input, maybe you check the top K (~4) common cases and otherwise jump into the interpreter. Fine.

But sometimes you can be compiling a polymorphic method bar that is actually monomorphic in its caller foo. That is, foo might only ever pass one kind of input to bar, but other callers pass all kinds of stuff. Here is a bit of a silly example to show what I mean:

class HashWithIndifferentAccess
  def initialize
    @hash = {}
  end

  # Allow reading from the Hash with either a String or a Symbol
  def [](key) = @hash[key.to_sym]

  # ...
end

# some method...
some_hash = HashWithIndifferentAccess.new
# ...
some_hash["abc"]

# some other method...
another_hash = HashWithIndifferentAccess.new
# ...
another_hash[:xyz]

Just kidding, not so silly at all. It’s a super common pattern in Rails. It makes key polymorphic in HashWithIndifferentAccess#[] even though for many of its callers, it may well be monomorphic (or even a constant).

In order to plumb this information through to the compiler, you have to figure out this call context relationship. There are a couple of common ways to do it.

Splitting

YJIT, for example, though it does not inline, splits methods based on the types of the arguments going in. This means that it clones the compiled code, generating a new version for each context. This does not give call context (“A calls B”) but gives type context (“B is called with integers, B’ is called with strings”).

A compiler could do type-based splitting in the interpreter or a baseline tier.

Profile splitting

If you don’t fancy duplicating the code, you can instead duplicate the profiles. You could either do this using type context (as above) or using call context. SpiderMonkey, for example, does “trial inlining” that allows callers to pass down a bit of memory for potential inline candidate callees to record their inline caches. Instead of each function holding its own ICScript, the caller allocates a unique ICScript for that potential-inline call-site. This gives each callee function (at least?) one level of call context.

Later, when inlining the callee into the caller, we don’t have other callers’ type information polluting the IR builder (or whatever reads the profiles).

Bytecode inlining

JavaScriptCore handles this by inlining bytecode into other bytecode. This is a gnarly transformation but gives the interpreter, even (!) access to call context. On tier-up to the compiler, all the inlining decisions have been made already.

EDIT: Upon reflection, I think think this is wrong and based on a misremembered conversation.

Early tier with counters

HotSpot handles this with multiple tiers. The interpreter tiers up to the client compiler, C1. C1 profiles branch and call targets in compiled code. C1 may eventually recompile based on this new information. C1 may eventually tier up to C2, which copies C1 inlining decisions. This way, we get call context in profiles via inlining.

Inline and analyze and hope

One last thing you could do is just trust your type inference and branch folding in the optimizer. You could inline and do polymorphic specialization in the callee when building the IR, then hope that your branch pruning monomorphizes the inlined callee. It’s a little wasteful because the polymorphic code is built “for nothing”, but it might work fine?

Okay, onto the collected notes and half-baked commentary. Here’s a survey of a bunch of JIT compilers and how they reason about inlining heuristics.

Thanks

But before we get into that, thanks to Iain Ireland, CF Bolz-Tereick, and Ian Rogers for feedback on this blog post!

The survey: bits and bobbles

What follows is mostly a “bits and bobbles” section a la Phil Zucker.

We’ll start with Cinder, because when I wrote Cinder’s inliner I added only the simplest heuristics, mostly “don’t inline” signals. Over time, after I left, people tuned more.

Cinder: new

Update: Shortly after writing this post, and completely unrelated to my writing this post, Alex Malyshev reworked the inlining heuristincs in Cinder. I left the old section below as Cinder: old and this section covers the newer design as of July 2026.

The two PRs: the first and the second.

The first PR ranks potential callees to inline by call count and avoids inlining callees that have 5% or less of the call count of the caller (inliner_cold_call_threshold). It also orders the callees by perceived importance using this metric: hotter callees get inlined first.

The second PR starts inlining transitively. That is, if we inline function B into function A, the calls in function B themselves become considered inlining candidates. Furthermore, it begins ranking call-sites by cost (measured by bytecode count) and using a worklist to drive the inlining decision making. It prohibits recursive inlines, preventing an inline if the code is already found in the compile-time inline stack. Last, it adds a depth limit for the recursion (inliner_depth_limit).

The old explicit sorting goes away; the new implementation uses std::priority_queue to order by cost and tie-break by the order in which a call was discovered.

Cinder: old

The inliner starts from the caller CFG, walking it to find suitable inlining candidates. Inlining candidates are only for call targets that are known—in Cinder’s case, only for monomorphic call targets—and pass some checks. The callee is only known by it’s function object, which includes its bytecode. There is no IR available for the callee until we decide to inline.

Most of the “can’t handle this” checks are related to argument handling. Python has a pretty complex calling convention, so if the caller/callee have not agreed on how the arguments should be passed through, the inliner doesn’t care to try and figure it out on its own. That is the responsibility of other parts of the compiler. Things in this canInline function could be considered “TODO”.

bool canInline(Function& caller, AbstractCall* call_instr) {
  // ...
  BorrowedRef<PyFunctionObject> func = call_instr->func;
  auto fail = [&](InlineFailureType failure_type) {
    dlogAndCollectFailureStats(caller, call_instr, failure_type);
    return false;
  };

  if (func->func_kwdefaults != nullptr) {
    return fail(InlineFailureType::kHasKwdefaults);
  }

  BorrowedRef<PyCodeObject> code{func->func_code};
  JIT_CHECK(PyCode_Check(code), "Expected PyCodeObject");

  if (code->co_kwonlyargcount > 0) {
    return fail(InlineFailureType::kHasKwOnlyArgs);
  }
  // ...
}

Failures are logged so they can be analyzed. If the Cinder team determines that there is some very frequent case they should handle, they will find out from the logs.

The inliner collects all candidate call instructions in one pass over the CFG. It loads the configurable “cost limit” from the options struct. Then it does one pass over the inlining candidates vector, inlining until it (maybe) hits the cost limit.

// ...
size_t cost_limit = getConfig().inliner_cost_limit;
size_t cost = codeCost(irfunc.code);

// Inline as many calls as possible, starting from the top of the function and
// working down.
for (auto& call : to_inline) {
  BorrowedRef<PyCodeObject> call_code{call.func->func_code};
  size_t new_cost = cost + codeCost(call_code);
  if (new_cost > cost_limit) {
    LOG_INLINER(
        "Inliner reached cost limit of {} when trying to inline {} into {}, "
        "inlining stopping early",
        new_cost,
        funcFullname(call.func),
        irfunc.fullname);
    break;
  }
  cost = new_cost;

  inlineFunctionCall(irfunc, &call);

  // We need to reflow types after every inline to propagate new type
  // information from the callee.
  reflowTypes(irfunc);
}
// ...

It does some graph maintenance work after inlining these calls, but that’s it.

This approach gets a surprising amount of utility for being so simple: it inlines constants (quite a few methods look like def foo(): return 5), small methods, and (at least, as far as I can remember) shrinks the compiled code size. All for very little compile time overhead.

There’s one other “standalone” Python JIT out there, PyPy. So we should look at that too.

PyPy

There are two inliners in PyPy. One is inside the RPython to C translation pipeline, which acts more like an ahead-of-time compiler². Then there is the tracing JIT bit, which has its own optimizer and heuristics. We’re going to look at the latter.

I talked to CF Bolz-Tereick about the inliner and their comment was that PyPy’s inlining heuristic is “yes”. There are a couple of exceptions, such as not inlining recursive functions or functions with loops. But the basic idea of tracing includes tracing through call instructions, which naturally means that you are “inlining”.

PyPy also does this neat thing where they treat frame pushes like normal allocation. Frame pushes, frame reads, and frame writes get written to the trace like normal object memory traffic and can get optimized away like other field reads and writes. This means that they can “just” use DCE to eliminate frame pushes and pops, whereas Cinder has some complicated mechanism to do it (which is my fault).

TODO get more details here

V8

V8 is a JS engine and it has over the years had many execution approaches. We’ll look at three of them since they all have or had their place in the history:

Hydrogen was the first real SSA IR and it looks very familiar to me, having worked on Cinder and now ZJIT. It is now defunct.
Turbofan was the replacement, going full Sea of Nodes. In the grand scheme of things it is a pretty fast compiler, but it does not hold back from doing some expensive rewrites. This was recently rewritten from Sea of Nodes to a mode traditional CFG and nicknamed Turboshaft.
Maglev is meant to coexist alongside Turbofan, preferring to speculate a little more eagerly and do fewer incremental rewrites in the name of compile time.³

They also each inline at different times in the pipeline, which made for a fun time trying to understand the different codebases.

V8 Hydrogen

Inlining happens during Hydrogen graph building

https://github.com/tekknolagi/v8/blob/a969ab67f8e1e7475d9b26468225c3a772890c64/src/crankshaft/hydrogen.cc#L9236

Don’t store function bytecode of all functions; need to re-parse callee text source to inline

Heuristics https://github.com/tekknolagi/v8/blob/a969ab67f8e1e7475d9b26468225c3a772890c64/src/crankshaft/hydrogen.cc#L7807

something about native context
check callee AST size against configurable limit
check inlining depth against configurable limit
don’t inline recursive functions
check current cumulative method size (as tracked by AST node count) against configurable limit

V8 TurboFan

https://docs.google.com/document/d/1VoYBhpDhJC4VlqMXCKvae-8IGuheBGxy32EOgC2LnT8/edit

https://github.com/v8/v8/blob/036842f4841326130a40adfcff38f85a9b4cd30a/src/compiler/js-inlining-heuristic.h#L14

Find candidates https://github.com/v8/v8/blob/036842f4841326130a40adfcff38f85a9b4cd30a/src/compiler/js-inlining-heuristic.cc#L134
Can inline https://github.com/v8/v8/blob/036842f4841326130a40adfcff38f85a9b4cd30a/src/compiler/js-inlining-heuristic.cc#L75
Force inline small functions https://github.com/v8/v8/blob/036842f4841326130a40adfcff38f85a9b4cd30a/src/compiler/js-inlining-heuristic.cc#L309
Loop over sorted (by comparator) list https://github.com/v8/v8/blob/036842f4841326130a40adfcff38f85a9b4cd30a/src/compiler/js-inlining-heuristic.cc#L847

V8 Maglev

When optimizing, add call instructions to the inline candidates list: https://github.com/v8/v8/blob/1a391f98cc7a9196369f2d6cab7df35ffbe92c08/src/maglev/maglev-graph-optimizer.cc#L1271

ProcessResult MaglevGraphOptimizer::VisitCall(Call* node,
                                              const ProcessingState& state) {
  // ...
  int bytecode_length = shared.GetBytecodeArray(broker()).length();
  float score =
      (call_frequency / bytecode_length) * (loop_depth_ > 0 ? 1.5 : 1.0);

  bool is_small_function =
      bytecode_length <
      reducer_.graph()->compilation_info()->flags().max_eager_inlined_bytecode;
  // ...
  MaglevCallSiteInfo* call_site = reducer_.zone()->New<MaglevCallSiteInfo>(
      MaglevCallerDetails{
          ...
          is_small_function, call_frequency,
          ...
      },
      score, bytecode_length);
  reducer_.PushInlineCandidate(call_site);
  // ...
}

https://github.com/v8/v8/blob/036842f4841326130a40adfcff38f85a9b4cd30a/src/maglev/maglev-inlining.h#L36

Unlike for example Cinder, Maglev looks like it does not have a lot of restrictions about what can get inlined into what, so its “can inline” signal is about budget. Actually two budgets: small budget and normal budget.

bool MaglevInliner::CanInlineCall() {
  // We stop inlining entirely if the small budget is exhausted.
  // Inlining decisions after that become bad if we stop inlining small
  // functions, but keep inlining large ones.
  return !graph_->inlineable_calls().empty() &&
         (graph_->total_inlined_bytecode_size() <
              max_inlined_bytecode_size_cumulative() ||
          graph_->total_inlined_bytecode_size_small() <
              max_inlined_bytecode_size_small_total());
}

Then its inlining loop is a greedy walk of the to-inline queue checking candidate sizes.

bool MaglevInliner::InlineCallSites() {
  DCHECK(CanInlineCall());
  while (!graph_->inlineable_calls().empty()) {
    // pop from inlineable_calls
    MaglevCallSiteInfo* call_site = ChooseNextCallSite();

    bool is_small_with_heapnum_input_outputs =
        IsSmallWithHeapNumberInputsOutputs(call_site);

    if (graph_->total_inlined_bytecode_size() >
        max_inlined_bytecode_size_cumulative()) {
      // We ran out of budget. Checking if this is a small-ish function that we
      // can still inline.
      if (graph_->total_inlined_bytecode_size_small() >
          max_inlined_bytecode_size_small_total()) {
        graph_->compilation_info()->set_could_not_inline_all_candidates();
        break;
      }

      if (!is_small_with_heapnum_input_outputs) {
        graph_->compilation_info()->set_could_not_inline_all_candidates();
        // Not that we don't break just rather just continue: next candidates
        // might be inlineable.
        continue;
      }
    }

    InliningResult result =
        BuildInlineFunction(call_site, is_small_with_heapnum_input_outputs);
    // ...
  }
  return true;
}

It runs this loop (which drains the queue) interleaved with the optimizer (which populates the queue).

bool MaglevInliner::Run() {
  if (graph_->inlineable_calls().empty()) return true;

  while (CanInlineCall()) {
    if (!InlineCallSites()) return false;
    RunOptimizer();
  }
  // ...
}

Confusingly, though, the optimizer also calls another function called CanInlineCall which checks if it legally can inline:

skip recursion
https://github.com/v8/v8/blob/1a391f98cc7a9196369f2d6cab7df35ffbe92c08/src/objects/shared-function-info-inl.h#L421
not called enough (min call frequency)
bytecode too big

bool MaglevGraphBuilder::ShouldEagerInlineCall( ~~appears unused? / dead declaration?~~ maybe src/maglev/maglev-graph-builder.cc is just not working on github search

MaybeReduceResult MaglevGraphBuilder::TryBuildCallKnownJSFunction( ~~also unused / dead declaration~~ same

JavaScriptCore

JavaScriptCore is funky! Unlike these other compilers that do inlining in their neat little SSA IRs, JSC inlines at the bytecode level⁴. This is their way of making sure that they get at least one level of call context into their interpreter inline caches, which will eventually give better information to the compiler.

JSC only inlines based on bytecode profile information, and only inlines bytecode??

TODO find better sources for bytecode inlining

SpiderMonkey

SpiderMonkey has another way of getting that call context without doing bytecode inlining: they add call context to their inline caches. Methods can pass down an ICScript to their callees where the callee writes its inline cache information. Then, when compiling, the callee is more likely to be monomorphized.

Wasm

https://github.com/mozilla-firefox/firefox/blob/438a3ce10eb77fb50d968463b7741117aec5bb4a/js/src/wasm/WasmHeuristics.h#L213

SpiderMonkey ICScript

Wasmtime and Cranelift

https://fitzgen.com/2025/11/19/inliner.html

HotSpot

Plan: run in interpreter; tier up to C1; profile call targets; inline in C1; profile branch counts; tier up to C2, which copies C1 inlining decisions in bytecode parser

HotSpot C2

https://github.com/openjdk/jdk/blob/a05d5d2514c835f2bfeaf7a8c7df0ac241f0177f/src/hotspot/share/opto/bytecodeInfo.cpp#L116

https://github.com/openjdk/jdk/blob/497dca2549a9829530670576115bf4b8fab386b3/src/hotspot/share/opto/bytecodeInfo.cpp#L197

https://github.com/openjdk/jdk/blob/497dca2549a9829530670576115bf4b8fab386b3/src/hotspot/share/opto/parse.hpp#L42

https://github.com/openjdk/jdk/blob/497dca2549a9829530670576115bf4b8fab386b3/src/hotspot/share/opto/doCall.cpp#L185

Not too small

Walk up the call stack to figure out what to compile

Handling the right thing to inline: def foo(a) = a.each {|x| x } want to compile foo, inline each, inline block, not compile block separately (probably)

HotSpot C1

https://bernsteinbear.com/assets/img/design-hotspot-client-compiler.pdf

https://github.com/openjdk/jdk/blob/d854a04231a437a6af36ae65780961f40f336343/src/hotspot/share/c1/c1_GraphBuilder.cpp#L755

https://github.com/openjdk/jdk/blob/d854a04231a437a6af36ae65780961f40f336343/src/hotspot/share/c1/c1_GraphBuilder.cpp#L3854

skip callees with exception handlers (unless explicitly allowed with a CLI flag)
skip synchronized callees (unless explicitly allowed with a CLI flag)
skip classes with unlinked callees
skip uninitialized classes
…

heuristics:

max inline level (default 9)
max recursive inline level (default 1)
callee bytecode size (max for top level is 35 bytecodes, but falls off by 10% per inline level)

callee stack usage (max of 10 slots)

      // Additional condition to limit stack usage for non-recursive calls.
      if ((callee_recursive_level == 0) &&
          (callee->max_stack() + callee->max_locals() - callee->size_of_parameters() > C1InlineStackLimit)) {
        INLINE_BAILOUT("callee uses too much stack");
      }

max total method size (default 8000 bytecodes)

TruffleRuby

TruffleRuby uses weighted compile queue

Graal https://ieeexplore.ieee.org/document/8661171

.NET

https://github.com/dotnet/runtime/blob/2d638dc1179164a08d9387cbe6354fe2b7e4d823/docs/design/coreclr/jit/inlining-plans.md

https://github.com/dotnet/runtime/blob/0b3f3ab1ecf4de06459e5f0e2b7cb3baf70ef981/src/coreclr/jit/inline.def#L94

https://github.com/dotnet/runtime/blob/0b3f3ab1ecf4de06459e5f0e2b7cb3baf70ef981/src/coreclr/jit/inlinepolicy.cpp

https://github.com/dotnet/runtime/blob/0b3f3ab1ecf4de06459e5f0e2b7cb3baf70ef981/docs/design/coreclr/jit/inline-size-estimates.md?plain=1#L5 https://github.com/dotnet/runtime/blob/0b3f3ab1ecf4de06459e5f0e2b7cb3baf70ef981/src/coreclr/jit/fginline.cpp

https://github.com/dotnet/runtime/issues/10303

https://github.com/AndyAyersMS/PerformanceExplorer/blob/master/notes/notes-aug-2016.md

Dart

https://github.com/dart-lang/sdk/blob/391212f3da8cc0790fc532d367549042216bd5ca/runtime/vm/compiler/backend/inliner.cc#L49

https://github.com/dart-lang/sdk/blob/391212f3da8cc0790fc532d367549042216bd5ca/runtime/vm/compiler/backend/inliner.cc#L1023

https://web.archive.org/web/20170830093403id_/https://link.springer.com/content/pdf/10.1007/978-3-540-78791-4_5.pdf

DEFINE_FLAG(int,
            deoptimization_counter_inlining_threshold,
            12,
            "How many times we allow deoptimization before we stop inlining.");
DEFINE_FLAG(bool, trace_inlining, false, "Trace inlining");
DEFINE_FLAG(charp, inlining_filter, nullptr, "Inline only in named function");

// Flags for inlining heuristics.
DEFINE_FLAG(int,
            inline_getters_setters_smaller_than,
            10,
            "Always inline getters and setters that have fewer instructions");
DEFINE_FLAG(int,
            inlining_depth_threshold,
            6,
            "Inline function calls up to threshold nesting depth");
DEFINE_FLAG(
    int,
    inlining_size_threshold,
    25,
    "Always inline functions that have threshold or fewer instructions");
DEFINE_FLAG(int,
            inlining_callee_call_sites_threshold,
            1,
            "Always inline functions containing threshold or fewer calls.");
DEFINE_FLAG(int,
            inlining_callee_size_threshold,
            160,
            "Do not inline callees larger than threshold");
DEFINE_FLAG(int,
            inlining_small_leaf_size_threshold,
            50,
            "Do not inline leaf callees larger than threshold");
DEFINE_FLAG(int,
            inlining_caller_size_threshold,
            50000,
            "Stop inlining once caller reaches the threshold.");
DEFINE_FLAG(int,
            inlining_hotness,
            10,
            "Inline only hotter calls, in percents (0 .. 100); "
            "default 10%: calls above-equal 10% of max-count are inlined.");
DEFINE_FLAG(int,
            inlining_recursion_depth_threshold,
            1,
            "Inline recursive function calls up to threshold recursion depth.");
DEFINE_FLAG(int,
            max_inlined_per_depth,
            500,
            "Max. number of inlined calls per depth");

An adaptive strategy for inline substitution (PDF)

  // Inlining heuristics based on Cooper et al. 2008.
  InliningDecision ShouldWeInline(const Function& callee,
                                  intptr_t instr_count,
                                  intptr_t call_site_count) {
    // Pragma or size heuristics.
    if (inliner_->AlwaysInline(callee)) {
      return InliningDecision::Yes("AlwaysInline");
    } else if (inlined_size_ > FLAG_inlining_caller_size_threshold) {
      // Prevent caller methods becoming humongous and thus slow to compile.
      return InliningDecision::No("--inlining-caller-size-threshold");
    } else if (instr_count > FLAG_inlining_callee_size_threshold) {
      // Prevent inlining of callee methods that exceed certain size.
      return InliningDecision::No("--inlining-callee-size-threshold");
    }
    // Inlining depth.
    const int callee_inlining_depth = callee.inlining_depth();
    if (callee_inlining_depth > 0 &&
        ((callee_inlining_depth + inlining_depth_) >
         FLAG_inlining_depth_threshold)) {
      return InliningDecision::No("--inlining-depth-threshold");
    }
    // Situation instr_count == 0 denotes no counts have been computed yet.
    // In that case, we say ok to the early heuristic and come back with the
    // late heuristic.
    if (instr_count == 0) {
      return InliningDecision::Yes("need to count first");
    } else if (instr_count <= FLAG_inlining_size_threshold) {
      return InliningDecision::Yes("--inlining-size-threshold");
    } else if (call_site_count <= FLAG_inlining_callee_call_sites_threshold) {
      return InliningDecision::Yes("--inlining-callee-call-sites-threshold");
    }
    return InliningDecision::No("default");
  }

HHVM

tracelet based

https://github.com/facebook/hhvm/blob/eeba7ad1ffa372a9b8cc9d1ec7f5295d45627009/hphp/runtime/vm/jit/inlining-decider.h#L89

  // Refuse if the cost exceeds our thresholds.
  // We measure the cost of inlining each callstack and stop when it exceeds a
  // certain threshold.  (Note that we do not measure the total cost of all the
  // inlined calls for a given caller---just the cost of each nested stack.)
  cost = costOfInlining(callerSk, callee, regionAndUnit, annotationsPtr);
  if (cost <= Cfg::HHIR::AlwaysInlineVasmCostLimit) {
    return accept(folly::sformat("cost={} within always-inline limit", cost));
  }

  if (region.instrSize() > irgs.budgetBCInstrs) {
    return refuse(folly::sformat(
      "exhausted bytecode budget: budgetBCInstrs={}, regionSize={}",
      irgs.budgetBCInstrs, region.instrSize()));
  }
  auto maxTotalCost = adjustedMaxVasmCost(irgs, region, inlineDepth(irgs));
  int maxCost = maxTotalCost;
  if (Cfg::HHIR::InliningUseStackedCost) {
    maxCost -= irgs.inlineState.cost;
  }
  const auto baseProfCount = s_baseProfCount.load();
  const auto callerProfCount = irgen::curProfCount(irgs);
  const auto calleeProfCount = irgen::calleeProfCount(irgs, region);
  if (cost > maxCost) {
    auto const depth = inlineDepth(irgs);
    return refuse(folly::sformat(
      "too expensive: cost={} : maxCost={} : "
      "baseProfCount={} : callerProfCount={} : calleeProfCount={} : depth={}",
      cost, maxCost, baseProfCount, callerProfCount, calleeProfCount, depth));
  }

  return accept(folly::sformat("small region with return: cost={} : "
                               "maxTotalCost={} : maxCost={} : baseProfCount={}"
                               " : callerProfCount={} : calleeProfCount={}",
                               cost, maxTotalCost, maxCost, baseProfCount,
                               callerProfCount, calleeProfCount));

ART

https://github.com/LineageOS/android_art/blob/8ce603e0c68899bdfbc9cd4c50dcc65bbf777982/compiler/optimizing/inliner.h

// Instruction limit to control memory.
static constexpr size_t kMaximumNumberOfTotalInstructions = 1024;

// Maximum number of instructions for considering a method small,
// which we will always try to inline if the other non-instruction limits
// are not reached.
static constexpr size_t kMaximumNumberOfInstructionsForSmallMethod = 3;

// Limit the number of dex registers that we accumulate while inlining
// to avoid creating large amount of nested environments.
static constexpr size_t kMaximumNumberOfCumulatedDexRegisters = 32;

// Limit recursive call inlining, which do not benefit from too
// much inlining compared to code locality.
static constexpr size_t kMaximumNumberOfRecursiveCalls = 4;

// Limit recursive polymorphic call inlining to prevent code bloat, since it can quickly get out of
// hand in the presence of multiple Wrapper classes. We set this to 0 to disallow polymorphic
// recursive calls at all.
static constexpr size_t kMaximumNumberOfPolymorphicRecursiveCalls = 0;

// Controls the use of inline caches in AOT mode.
static constexpr bool kUseAOTInlineCaches = true;

// Controls the use of inlining try catches.
static constexpr bool kInlineTryCatches = true;

void HInliner::UpdateInliningBudget() {
  if (total_number_of_instructions_ >= kMaximumNumberOfTotalInstructions) {
    // Always try to inline small methods.
    inlining_budget_ = kMaximumNumberOfInstructionsForSmallMethod;
  } else {
    inlining_budget_ = std::max(
        kMaximumNumberOfInstructionsForSmallMethod,
        kMaximumNumberOfTotalInstructions - total_number_of_instructions_);
  }
}

bool HInliner::IsInliningEncouraged(const HInvoke* invoke_instruction,
                                    ArtMethod* method,
                                    const CodeItemDataAccessor& accessor) const {
  if (CountRecursiveCallsOf(method) > kMaximumNumberOfRecursiveCalls) {
    LOG_FAIL(stats_, MethodCompilationStat::kNotInlinedRecursiveBudget)
        << "Method "
        << method->PrettyMethod()
        << " is not inlined because it has reached its recursive call budget.";
    return false;
  }

  size_t inline_max_code_units = codegen_->GetCompilerOptions().GetInlineMaxCodeUnits();
  if (accessor.InsnsSizeInCodeUnits() > inline_max_code_units) {
    LOG_FAIL(stats_, MethodCompilationStat::kNotInlinedCodeItem)
        << "Method " << method->PrettyMethod()
        << " is not inlined because its code item is too big: "
        << accessor.InsnsSizeInCodeUnits()
        << " > "
        << inline_max_code_units;
    return false;
  }

  if (graph_->IsCompilingBaseline() &&
      accessor.InsnsSizeInCodeUnits() > CompilerOptions::kBaselineInlineMaxCodeUnits) {
    LOG_FAIL_NO_STAT() << "Reached baseline maximum code unit for inlining  "
                       << method->PrettyMethod();
    outermost_graph_->SetUsefulOptimizing();
    return false;
  }

  if (invoke_instruction->GetBlock()->GetLastInstruction()->IsThrow()) {
    LOG_FAIL(stats_, MethodCompilationStat::kNotInlinedEndsWithThrow)
        << "Method " << method->PrettyMethod()
        << " is not inlined because its block ends with a throw";
    return false;
  }

  return true;
}

if (outermost_graph_->IsCompilingBaseline() &&
    (current->IsInvokeVirtual() || current->IsInvokeInterface()) &&
    ProfilingInfoBuilder::IsInlineCacheUseful(current->AsInvoke(), codegen_)) {
  uint32_t maximum_inlining_depth_for_baseline =
      InlineCache::MaxDexPcEncodingDepth(
          outermost_graph_->GetArtMethod(),
          codegen_->GetCompilerOptions().GetInlineMaxCodeUnits());
  if (depth_ + 1 > maximum_inlining_depth_for_baseline) {
    LOG_FAIL_NO_STAT() << "Reached maximum depth for inlining in baseline compilation: "
                       << depth_ << " for " << callee_graph->GetArtMethod()->PrettyMethod();
    outermost_graph_->SetUsefulOptimizing();
    return false;
  }
}

JikesRVM

https://github.com/JikesRVM/JikesRVM/blob/5072f19761115d987b6ee162f49a03522d36c697/rvm/src/org/jikesrvm/compilers/opt/inlining/DefaultInlineOracle.java#L55

Other/research

Partial inlining

Understanding and Exploiting Optimal Function Inlining (PDF)

machine learning

Automatic construction of inlining heuristics using machine learning

Machine-Learning-Based Optimization Heuristics in Dynamic Compilers (PDF)

Guiding Inlining Decisions Using Post-Inlining Transformations (PDF)

U Can’t Inline This! (PDF)

Towards better inlining decisions using inlining trials

RhizomeRuby inlining

An Optimization-Driven Incremental Inline Substitution Algorithm for Just-in-Time Compilers (PDF)

Automatic Tuning of Inlining Heuristics (PDF)

Inlining-Benefit Prediction with Interprocedural Partial Escape Analysis (PDF)

Inlining of Virtual Methods (PDF)

A Study of Type Analysis for Speculative Method Inlining in a JIT Environment (PDF)

A Comparative Study of Static and Profile-Based Heuristics for Inlining (PDF)

clusters from Custom benefit-driven inliner in Falcon JIT (PDF)

Graal

https://github.com/oracle/graal/blob/5dde777cba22a99ebe3f19745d03ddfbc35c563c/compiler/src/jdk.graal.compiler/src/jdk/graal/compiler/phases/common/inlining/policy/GreedyInliningPolicy.java

https://github.com/oracle/graal/blob/5dde777cba22a99ebe3f19745d03ddfbc35c563c/compiler/src/jdk.graal.compiler/src/jdk/graal/compiler/phases/common/inlining/InliningPhase.java

https://github.com/oracle/graal/blob/5dde777cba22a99ebe3f19745d03ddfbc35c563c/compiler/src/jdk.graal.compiler/src/jdk/graal/compiler/phases/common/inlining/info/elem/InlineableGraph.java#L148

There are some newer papers, especially in Java land, that try to do a lot of analysis ahead-of-time and bundle the resulting information in .class files. Then the JIT can read it and see more than local context.

Or, if you are an AOT compiler, you can probably do a lot more whole system reasoning—both for time budget reasons and also because you can see more functions at once. ↩
Check it out if you like. I stumbled across it by accident. ↩
See also “Turbolev”, which seems to merge Maglev (CFG) with Turbofan (Sea of Nodes)… somehow. ↩
Potentially a misunderstanding based on a private conversation. I’m working on tracking down the implementation… ↩

Checking assembly with Z3

Mon, 01 Jun 2026 00:00:00 +0000

Short post today. New ZJIT contributor dak2 submitted a PR to fix an overflow bug in fixnum division in ZJIT. We did the division fine, but lied about the type of the result in the case of dividing FIXNUM_MIN by -1. You can see how this is special-cased in CRuby:

static inline void
rb_fix_divmod_fix(VALUE a, VALUE b, VALUE *divp, VALUE *modp)
{
    // ...
    if (x == FIXNUM_MIN && y == -1) {
        if (divp) *divp = LONG2NUM(-FIXNUM_MIN);
        if (modp) *modp = LONG2FIX(0);
        return;
    }
    // ...
}

Since -FIXNUM_MIN (note the negative) does not fit in a fixnum, it gets promoted to a bignum. It’s one of two special cases in fixnum division that does not produce a fixnum, the other being dividing by zero (which produces an error).

$ irb
irb(main):001> LONG_MAX = 2**63 - 1
=> 9223372036854775807
irb(main):002> FIXNUM_MAX = LONG_MAX / 2
=> 4611686018427387903
irb(main):003> LONG_MIN = -LONG_MAX - 1
=> -9223372036854775808
irb(main):004> FIXNUM_MIN = LONG_MIN / 2
=> -4611686018427387904
irb(main):005> (-FIXNUM_MIN) < FIXNUM_MAX
=> false
irb(main):006>
$

This is due to the numbers being two’s complement and therefore having more negative numbers than positive numbers (because of zero).

dak2’s proposed patch included a branchless test for this left == FIXNUM_MIN and right == -1 case, making us leave JIT code and enter the interpreter rather than handle it inline. The patch encodes this branchless test as xor, xor, or, test, je in our platform-independent low-level IR (LIR):

// Side exit on FIXNUM_MIN / -1, which overflows to a Bignum, not a Fixnum.
// Branchless (left == FIXNUM_MIN && right == -1): (left ^ MIN) | (right ^ -1) == 0.
let left_diff = asm.xor(left, Opnd::from(VALUE::fixnum_from_isize(RUBY_FIXNUM_MIN)));
let right_diff = asm.xor(right, Opnd::from(VALUE::fixnum_from_isize(-1)));
let combined = asm.or(left_diff, right_diff);
asm.test(combined, combined);
asm.je(jit, side_exit(jit, state, FixnumDivOverflow));

I didn’t understand why those were equivalent. Rather than try and bang my head against it, I thought I’d let Z3 try. After all, I’ve been watching CF have fun with it for a couple years now.

Z3 is an SMT solver. The core trick to use Z3 as a “proof engine” that I learned from CF¹ is to make Z3 search for counter-examples by negating the condition:

#!/usr/bin/env python3
# /// script
# requires-python = ">=3.13"
# dependencies = [
#     "z3-solver",
# ]
# ///

import z3

WORD_SIZE = 64
LONG_MAX   = z3.BitVecVal(2**63 - 1, WORD_SIZE)
LONG_MIN   = -LONG_MAX - 1
FIXNUM_MIN = LONG_MIN / 2

def prove(cond):
    solver = z3.Solver()
    assert solver.check(z3.Not(cond)) == z3.unsat, solver.model()

left = z3.BitVec("left", WORD_SIZE)
right = z3.BitVec("right", WORD_SIZE)
# The original C condition
lhs = z3.And(left == FIXNUM_MIN, right == -1)
# The new branchless LIR
rhs = ((left ^ FIXNUM_MIN) | (right ^ -1)) == 0
prove(lhs == rhs)

This negation is required because the core model of Z3 is a machine that finds example values. This means that there is an implicit “exists” in front of your condition. To disprove this, Z3 needs to search all inputs. However, if you negate the condition, it becomes a “for all”. This means that in order to disprove the “for all”, Z3 only needs to find a single counterexample.

$ uv run prove_fixnum_min.py
$

Z3 did not complain, so the new code must be fine. Just as a quick check, in case assert is turned off or something, I like to see tests fail. So after modifying one of the constants from -1 to 0:

$ uv run prove_fixnum_min.py
Traceback (most recent call last):
  File "/path/prove_fixnum_min.py", line 26, in <module>
    prove(lhs == rhs)
    ~~~~~^^^^^^^^^^^^
  File "/path/prove_fixnum_min.py", line 18, in prove
    assert solver.check(z3.Not(cond)) == z3.unsat, solver.model()
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
AssertionError: [right = 18446744073709551615, left = 13835058055282163712]
$

Neat.

Thanks

Thanks to CF Bolz-Tereick for reading and giving feedback on this post.

Update

I tried running this with lean_py after seeing a post about it. I installed elan and got a Lean installation going and manage to successfully import lean_py.z3 as z3 and whatnot, but then I think compiling the Lean program failed somehow:

cedar% uv run prove_fixnum_min.py
Traceback (most recent call last):
  File "/home/max/Documents/code/bernsteinbear.com/prove_fixnum_min.py", line 28, in <module>
    z3.prove(lhs == rhs)
...
...
  File "/home/max/.cache/uv/environments-v2/prove-fixnum-min-67c8d67f8106c77d/lib/python3.13/site-packages/lean_py/utils.py", line 17, in run_command
    raise RuntimeError(f"Command {args} exited with {res.returncode}: {stderr}")
RuntimeError: Command ['lake', 'build'] exited with 1: error: build failed

If I have time, I will come back to this and try and figure out what the inscrutable error is. I’ve never used Lean before, though…

They note that this is a “standard technique” and they did not come up with it. ↩

Travel notes: RubyKaigi Hakodate

Mon, 18 May 2026 00:00:00 +0000

I just got back from a three and a half week trip to Japan. It was the longest trip I have ever been on (aside from studying abroad in Germany, which felt different). I made the following wild circuit with only a backpack and a duffel:

Tokyo
Toyama
Kanazawa
Nara ish
Ito
Hakodate
Nikko
Mashiko
Karuizawa
Tokyo

This trip was split into three parts: time with my immediate family, going to a conference, and then time with my partner. They were all great and also I am glad to be home.

I’ll post my abbreviated travel notes here, including activity and food recommendations.

Part one

We started in Tokyo but we were only there for about 40 hours. We focused our time mostly on arts and crafts: we did a kintsugi workshop, spent time at an artists cooperative, and then did a lot of walking around. This was a good intro to the trip, because everyone kept waking up at 4am and crashing at 7pm due to the jet lag. 4am wakeup makes for nice morning walks to 7-Eleven.

I brought my family to T’s Tantan in Tokyo Station because I’m vegetarian and it’s otherwise hard to find ramen that approaches kosher in Japan. It continues to be great and I really appreciate having a steady vegetarian option available. Many years ago when I visited Tokyo there was a place that served a delicious tomato-based vegetarian ramen, but I hear it has since permanently closed. Bummer.

We took the shinkansen to Kanazawa. I love the train. It’s fast. It’s quiet. You can eat your snacks on board and gaze out the window as the world whizzes by. It’s nice.

We toured a soy sauce factory (meh; they don’t let you in the room where the magic happens) and the old town (pretty!) before finally eventually ending up at our small hotel in Toyama: Satoyama Auberge Maki No Oto. I highly recommend this hotel. It is beautiful, the staff is lovely, the food was excellent, and they were very accomodating of me being vegetarian.

We continued on to Toyama, which is a port town. We got to talking with an older local guy who told us all about his favorite local spots. We learned after leaving that this guy has extraordinarily fancy taste and they were all either Michelin starred or at least Michelin rated and with a lead time of months. We opted to instead go to a local brewery, which had a ghost pepper beer (!) and pizza.

We then moved on via train to Osaka, where we transferred to a car to head (eventually) to our hotel in the hills near Nara. We toured the Daimon sake brewery. They explained every little thing about the process, which was especially interesting to me, as I’ve done some small amount of homebrewing and I bake. They sounded similar. We had a tasting and even got to talk to Daimon-san. I recommend going.

I also recommend the Akame 48 waterfalls walk/hike, which has some exquisite falls, and Murou Art Forest. They had some really wonderful installations.

My brother and I parted ways from the rest of my family in Osaka: they headed further west and we headed north to Itō on the Izu peninsula. We got a surprise perfectly clear view of Fuji along the way.

It’s beautiful there. They don’t seem to welcome foreigners in a lot of their restaurants (we were turned away several times) but one place had a guy who enthusiastically welcomed us in. We ended that evening enjoying a some food and a beer while also being stared at by a 300lb completely tattooed guy. It was a little unsettling but we left without incident.

My brother and I made our way to Tokyo for the day before his flight and before my train north to Hakodate for RubyKaigi. I once again did that thing where I walked around in humid 80F heat with a large backpack and pants and was extraordinarily warm toward the end of the day. After about a liter of Aquarius on the train north I felt better.

Part two

I stayed at Yunokawa Prince Hotel Nagisatei which I would like to especially call out for having an enormous, diverse, and very vegetarian friendly breakfast. Every morning I got to try new and tasty things and even feel full after. It was great.

Hakodate is beautiful in the spring. I arrived at peak cherry blossom season and Goryokaku, their star shaped fort, is absolutely decked out in cherry blossoms. It is also moderately swarmed by tourists (in this case, three cruise ships). It didn’t feel over-crowded though. I enjoyed eating at The Bear King which had a vegetarian friendly option.

The next day was the committer meeting. I don’t remember a ton from it other than people talking at length about the semantics of deep freezing an object (do you freeze its class? its class’s superclass? …?). I picked up my badge and also got to check out my colleague Chris Salzberg’s bar SOLENOID! It’s a neat spot. I headed out to go find some dinner.

This is about when I got a message on my phone that there was going to be an earthquake, so I walked back into the bar and said “hey, did you get this?” just before everything started shaking. It was the biggest earthquake I’ve experienced, but I was metaphorically not too shaken up. Then we got the tsunami warning.

Chris’s bar is already something like 8 meters above sea level and at the foot of Mt Hakodate. With the city sirens going off and the police directing traffic with batons, though, I decided my best bet was just to march directly up the mountain to get more elevation. Since the tsunami wasn’t scheduled to arrive for about 20 or 30 minutes and my hotel was across the sea-level part of town, I parked myself on a little concrete post. Chris found me eventually. Someone told us that there was a middle school offering refuge, so we went and hung out on the side of the gymnasium. They were really nice about it.

On Wednesday, the conference started. It was really well signed and organized. My usual complaint with conferences is that there’s nothing to eat for vegetarians (or that we get mashed with the gluten-free people and each group only gets a salad and bad bread) but that did not happen! They had really stellar vegetarian bento. They had a lot of leftovers toward the end of lunch so I even went and got a second. This was about when I started freaking out because my speaking slot was approaching and I wasn’t yet feeling my talk.

Normally when I give a talk, I get up in front of people and I pace and gesticulate and productively complain and throw in some fun anecdotes and the audience, one way or another, ends up learning about JITs at scale, or Scheme semantics, or something. It’s what I’d done for my little lunch talk at Brown two weeks prior. I even titled that talk One must imagine compiler engineers happy so there was plenty of room for educational complaining. But this RubyKaigi talk was in front of an enormous crowd and toward a more general audience than I was used to addressing. The slides did not feel like they were flowing until about twenty minutes before my talk.

In the end it went alright. I realized about 40 seconds in that I had way too much content so I ended up speaking rapidly for 30 minutes straight, completely unaware of the audience (which you can’t see anyway because of the lights). I only really noticed people when I made a dumb six-seven joke and Aaron laughed.

The rest of the conference I was able to relax and enjoy other people’s talks. I got some good hallway track in, too. I think there’s a good group of people who are interested in Ruby tracing (for example, Perfetto in ZJIT) so maybe we will make something happen.

We had a nice small dinner at Yasai Bar Miruya, which was vegan (!) and had some nice sake. The host was very friendly, too.

I nerd-sniped John and J into implementing a VM for the Universal Machine. This was a daunting homework assignment back in undergrad but it was a fun project later in life.

S joined toward the end of the conference. She’s also vegetarian so we got some really excellent vegetarian ramen at MAIDO Ramen.

Finally, S and I headed south on the shinkansen for Nikko.

Part three

Nikko is small, beautiful, and a tourist day-trip town. Dinner closes early. Shops close earlier. Since we were staying there we had to make sure to track down and visit the one or two vegetarian places before they shuttered.

S and I, along with J and J, took the bus up from Nikko, up the windiest switchbacks, to the Kegon Falls. We were going to take a boat across the lake, but the water level was too low for the dock on the other side, so we ended up half hiking and half taking a bus. Then we continued our hike through the Senjōgahara Marshland (beautiful), to the Yudaki Cascades (lovely), which also had a surprise restaurant and ice cream shop at the base! It’s called Yutaki Rest House. After some great (vegetarian friendly!! wow!!) udon, we marched up the waterfall and around Yuno Lake at the top to Yumoto Onsen. In order to make the last reasonable bus back to town, we just enjoyed putting our feet in the foot bath.

One day was rainy. In the evening, J and I thought it would be fun to continue our Universal Machine implementations. As Norman Ramsey would say, “my implementation is 90 lines long and runs sandmark in under six seconds.”

We also enjoyed doing a tour of the shrines right above Nikko. The shrines are resplendent against the backdrop of forest.

Pro bus tip: you can either pay by IC card or credit card. No need to grab a ticket if you do that.

S and I shipped our bags (thanks, Yamato) before continuing on to the small town of Moka, the staging area for our big pottery festival day. Unfortunately, there was no good way to get there: there was no reasonable series of trains and no taxi would take us. Ultimately we ended up taking the train to Utsonomiya and catching the long local bus to Moka. About twenty minutes into this ride, in the middle of nowhere, bus nearly empty, the bus driver pulled over and ran over to us looking kind of panicked. He asked where we were going and was visibly relieved when we said Moka. I suppose we are not the usual riders. Very nice of him.

Upon arrival, S introduced me to CoCo ICHIBANYA, which is also super vegetarian friendly. I loved it. We ate really well before walking to our tiny hotel.

We did not really know what to expect from the Mashiko pottery festival. The internet said it would be crowded and to arrive early, so we got up at 6:30am for estimated 7am departure on the tiny train from Moka to Mashiko. On most trains you can pay with an IC card but we were out in the sticks so we asked the only other guy on the platform how to pay for the train. He said he had no idea and that this was his first time here. When the train showed up completely packed to the gills and we had to (politely) push onto it, we started to realize that this was The Event and it was going to be mayhem.

Also, fun fact: the way the Moka train payment works is that you grab a little ticket from the train, and, upon arrival, wait in line to present your ticket to two very overwhelmed looking people at a table, who charge you, and you pay in cash.

Onto Mashiko: the festival was packed. There’s pottery everywhere the eye can see. There are tents and there are full buildings. It varies in quality and artistry from fine to jaw-droppingly spectacular. You could completely stock your kitchen from this fair alone and it would even be cost-effective. The main bummer for us is that we had to get pottery safely back home. We limited ourselves to a reasonable assortment but we really wanted to buy a beautiful painted 20 inch plate with a bird on a branch.

After a ton of walking around, we took another long long bus back to Utsonomiya and continued onto Karuizawa. We didn’t know what to expect from Karuizawa but, having been, I could probably concisely describe it as “Aspen for people from Tokyo”. It was… fine. We loved our hotel, Tsuruya Ryokan. The manager was very excited when we borrowed a Studio Ghibli DVD from their collection.

We continued on to Tokyo, our final stop. We our usual tour of stationery stores and bakeries—the bread was something to write home about (har har). We enjoyed a (vegetarian!! friendly!!) kaiseki meal at Hyoki Shabu-shabu Ginza before enjoying some live music at Rocky Top.

We also recommend Jikasei MENSHO for vegetarian ramen.

Bakery checklist:

BOUL’ANGE NIHONBASHI (check! good croissants)
Bricolage bread & co (check! good everything)
Brasserie Viron Marunouchi
Beaver Bread
Bricolage bread & co.
Bartizan Bread Factory
Gontran Cherrier Tokyo Aoyama Shop
Comme’N Tokyo
Shiomi Bakery
BRØD
The Little BAKERY <!–
https://www.jocjapantravel.com/kanto-tokyo-bakeries/ –>

We had an uneventful and reasonably easy trip home. Whew. Long post for a long trip. See you next year in Miyazaki!

Partial static single information form

Tue, 12 May 2026 00:00:00 +0000

In compilers, static single information form (SSI) is a common extension to static single assignment form (SSA). It was introduced by C. Scott Ananian in 1999 in his MS thesis (PDF) ¹.

SSI extends your existing SSA intermediate representation by discovering facts from your existing program and reifying them as path-dependent/flow-sensitive IR nodes. That might sound complicated, but at least the basic idea is pretty natural. I talk a little bit about it in What I talk about when I talk about IRs and I’ll rehash here in more depth, starting with some motivating examples. Consider this admittedly contrived example:

v0: Integer = ...
if v0 > 0:
    # ...
    v1: PositiveInteger = AbsoluteValue v0
    # ...

We should be able to learn from the comparison that in some branches in the IR, v0 is positive. In that region, we can add a new IR instruction v2 that attaches that knowledge right in the instruction’s type field (yay, sparseness!) and then rewrite uses of v0 to now use v2.

v0: Integer = ...
if v0 > 0:
    v2: PositiveInteger = RefineType v0, Positive
    # ...
    v1: PositiveInteger = AbsoluteValue v2
    # ...

Because we’ve done that, our (imaginary) optimization rule that gets rid of AbsoluteValue on known-positive integers can kick in, and we can delete the invocation of AbsoluteValue. Yay, optimization!

But a couple of questions remain, at least for me:

Where/when in the compiler pipeline do we insert and remove these type refinements?
Do we need to refine after every conditional?
Do we need to implement the whole into-SSI and out-of-SSI algorithms from all the complicated-looking papers?

We’ll go through them, starting with the compiler pipeline.

The original SSI paper starts with (I think?) SSA form and places some number of new refinement nodes based on conditionals. I have admittedly not tried very hard, but the into-SSI algorithms look complicated and kind of heavyweight. As a reward, you get “linear” into-SSI time complexity.

But I am a humble compiler engineer, and I don’t have the time to go through and load all of this into my head. Instead what I have seen done and have been doing is to take a shortcut: build partial SSI during SSA construction².

Most of the time this is from bytecode, but it could also be from some other non-SSA IR. In any case, this is an excellent shortcut for two reasons:

It lets me cleanly separate adding the type refinements (pretty straightforward) from the hard part of doing all of the operand rewriting and phi placement and marking and all manner of other nonsense.
In addition to separating the concerns, the hard part is already done by SSA construction. We can actually just skip it! SSA construction handles phi placement, operand rewriting, all of it. It probably fits neatly into a naive or a Braun-style (PDF) construction.

This is pretty compelling. We can learn from the bytecode with a very small amount of marginal new complexity. See my implementation in ZJIT, for example. All it really does is modify the abstract interpreter state when building SSA out of branchnil, branchif, and branchunless bytecode instructions to take into account the new refined values.

This is fine for branches that are already in the user’s source program but sometimes optimization, especially of dynamic languages, adds new branches that were not there before. And sometimes these branches get added much later, long after SSA construction. What then? Can we do something similar and rely on existing infrastructure?

During SSA optimization

Implicit in this “can we do it” is the assumption that your IR tracks data dependencies from use to corresponding def, but not from def to uses. Sea of Nodes (at least the Simple implementation), is an IR that tracks both directions all the time for easier rewriting. Many IRs do not do this, so we will continue assuming that there’s no “easy way out”.

JIT optimization of dynamic language compilers often adds synthetic Guard instructions to the IR that enforce pre-conditions. These guards allow optimizing happy/fast path cases in JIT code while leaving the interpreter as a fallback. For example, we might be able to optimize two back-to-back setinstancevariable instructions (a very dynamic operation in the world of ideas, but fast when concretely implemented using object shapes) from:

x = ...
setinstancevariable x, :@a, 1
setinstancevariable x, :@b, 2
# ... use x somewhere ...

which is very generic and involves calling into C code that might raise an exception, to something more like:

x = ...
v0 = GuardHeapObject x
v1 = GuardShape v0, 0xcafe
v2 = Const 1
StoreField v1, 0x8, v2
v3 = GuardHeapObject x
v4 = GuardShape v3, 0xcafe
v5 = Const 2
StoreField v4, 0x10, v5
# ... use x somewhere ...

which is much faster (assuming shape stability at run-time). There’s an irritating problem, though, which is that we have a bunch of duplicate instructions littered around the IR now because our optimizer worked on each instruction individually. Kind of a “template optimizer” situation. Now we need some pass to clean up the detritus.

Global value numbering (GVN) will do a good job of de-duplicating instructions. It should notice that we already have an instruction that looks like GuardHeapObject x called v0 and rewrite v3 into v3 = v0. That’s great because we have de-duplicated the guard. GVN may not get everything, though; if some instructions later use x, they will not get rewritten to instead use the output of these new guard instructions. To do that, we need to add some kind of canonicalize pass or augment GVN with some canonicalization feature. That canonicalization would handle rewriting operands to use the “latest version” of some value, so to speak. See the canonicalization section of Chris Fallin’s excellent aegraphs blog post for more (and of course the (currently block-local) implementation in ZJIT).

def canonicalize(bb)
  rewrite_map = {}
  bb.map do |i|
    i.map_operands! do |o|
      rewrite_map[o] || o
    end
    if i.opcode == :guardtype
      rewrite_map[i.operands[0]] = i
    end
    i
  end
end

Where I’m going with all of this, though, is that you may already have some dominance-based instruction rewriting mechanism in your compiler, either as part of GVN or separately! And you can use this to do a very low code into-partial-SSI in the middle of your optimizer.

This means you could very well get away with inserting RefineType instructions in successor blocks of conditionals and get the into-SSI “for free”.

After which conditionals do we refine?

That’s up to you. There’s a trade-off between compile-time and run-time, especially in JITs. Inserting more instructions and rewriting more times may slow down your compiler. It’s a cheap lunch, not a free one.

How does this compare to the complicated looking papers?

I don’t know. I don’t have a good grasp of how this “partial SSI” compares to the “full SSI”. I don’t plan on implementing full SSI in the near future.

I will note that this partial SSI approach doesn’t do two things:

It doesn’t split variables with a new sigma node, and it generally inserts the refine node within the target block rather than above the branch
(For canonicalize only) It doesn’t insert new phi nodes; it just leaves both IR nodes available and, instead of re-merging, drops them

I can’t tell what impact this has.

In other compilers

Like Simple, TruffleRuby is built on a Sea of Nodes IR (Graal). Chris Seaton has an excellent blog post about TruffleRuby’s use of “stamp nodes” (“Pi nodes”³). The replaceAtUsagesAndDelete function does a lot of heavy lifting, I think because Graal tracks uses.

Cinder mostly inserts RefineType instructions in the HIR builder, before into-SSA, and then lets the SSA construction take care of things. That’s where I learned this trick, actually. Here is one example of refining the type of the matched operand when building IR for pattern matching.

Luau is working on something like this, but for their type checker. Chatting with someone on their team is actually part of the reason I got motivated to write this post.

Android ART looks like it has HBoundType and inserts them in reference type propagation. This handles class checks, null checks, and instanceof checks.

Aside: logic for e.g. HeapObject upgrade

Last, I want to talk a little bit about some interesting reasoning you can do when you have two implementations of something that you can switch between. For example, JIT (+ interpreter), or aliasing and non-aliasing cases in C code, or the weirdo NULL-UB reasoning LLVM can do to C code, things like that.

In ZJIT, we currently insert RefineTypes opportunistically in “easy” cases when building our HIR from the interpreter bytecode.

For example, if in the bytecode there is a branch that compares some value x with nil, it will have two outgoing control-flow edges: one block where x is definitely nil, and one block where x is definitely not nil. In each of these control-flow edges, we can insert corresponding type refinement hints. That’s pretty standard. But we can also do weirder stuff.

CRuby has a notion of heap objects vs immediate objects. Many (most?) objects are heap objects. However, integer 5, for example is not allocated on the heap but instead represented by a tagged bit pattern that pretends to be an address: the whole value is encoded in the pointer itself.

We encode this knowledge in the HIR’s type system: “heapness” and “immediateness” each get a bit in the type lattice. We use this in the optimizer to reason about effects, among other things.

We can’t know a lot of the time what type a thing is, so we pessimistically type most objects flowing through bytecode as BasicObject. This type encapsulates the entire world of possible values that could go on the stack or in a local variable.

On most heap objects, with only a few exceptions, you can write instance variables (fields, attributes, whatever you want to call them). You can never write an instance variable to an immediate. This means that if we observe the following pattern in the bytecode:

x: BasicObject = ...
setinstancevariable x, :@abc, 1

Then after building and emitting HIR for the setinstancevariable opcode, we can upgrade the type of x from a BasicObject to a HeapBasicObject. We can do this because if it weren’t a heap-allocated object, we would have left the compiled code and entered the interpreter.

This is another SSI-type thing you can do in your compiler.

Conclusion

Uhh I guess the conclusion is that you don’t have to do full SSI and partial SSI is available and not too scary? Does your compiler do this? Reader, please write in.

…and optimized in 2002 (PDF), revisited in 2009 (PDF), implemented in LLVM in 2010 (PDF), investigated in 2017 for abstract compilation (PDF), and probably more. The 2009 paper by Boissinot, Brisk, Darte, and Rastello even shows that both Ananian and Singer’s papers have bugs, while perhaps unintentionally also making an excellent pun about the literature being “sparse”. ↩
This blog post is different than the what the LLVM paper (PDF) calls partial SSI. Partial for different reasons. Maybe it’s not even single information anymore. ↩
Today I learned that this terminology comes from the ABCD paper (PDF). ↩

Value numbering

Sat, 04 Apr 2026 00:00:00 +0000

Welcome back to compiler land. Today we’re going to talk about value numbering, which is like SSA, but more.

Static single assignment (SSA) gives names to values: every expression has a name, and each name corresponds to exactly one expression. It transforms programs like this:

x = 0
x = x + 1
x = x + 1

where the variable x is assigned more than once in the program text, into programs like this:

v0 = 0
v1 = v0 + 1
v2 = v1 + 1

where each assignment to x has been replaced with an assignment to a new fresh name.

It’s great because it makes clear the differences between the two x + 1 expressions. Though they textually look similar, they compute different values. The first computes 1 and the second computes 2. In this example, it is not possible to substitute in a variable and re-use the value of x + 1, because the xs are different.

But what if we see two “textually” identical instructions in SSA? That sounds much more promising than non-SSA because the transformation into SSA form has removed (much of) the statefulness of it all. When can we re-use the result?

Identifying instructions that are known at compile-time to always produce the same value at run-time is called value numbering.

Eliminating common subexpressions

To understand value numbering, let’s extend the above IR snippet with two more instructions, v3 and v4.

v0 = 0
v1 = v0 + 1
v2 = v1 + 1
v3 = v0 + 1  # new
v4 = do_something(v2, v3)  # new

In this new snippet, v3 looks the same as v1: adding v0 and 1. Assuming our addition operation is some ideal mathematical addition, we can absolutely re-use v1; no need to compute the addition again. We can rewrite the IR to something like:

v0 = 0
v1 = v0 + 1
v2 = v1 + 1
v3 = v1
v4 = do_something(v2, v3)

This is kind of similar to the destructive union-find representation that JavaScriptCore and a couple other compilers use, where the optimizer doesn’t eagerly re-write all uses but instead leaves a little breadcrumb Identity/Assign instruction¹.

We could then run our copy propagation pass (“union-find cleanup”?) and get:

v0 = 0
v1 = v0 + 1
v2 = v1 + 1
v4 = do_something(v2, v1)

Great. But how does this happen? How does an optimizer identify reusable instruction candidates that are “textually identical”? Generally, there is no actual text in the IR.

One popular solution is to compute a hash of each instruction. Then any instructions with the same hash (that also compare equal, in case of collisions) are considered equivalent. This is called hash-consing.

When trying to figure all this out, I read through a couple of different implementations. I particularly like the Maxine VM implementation. For example, here is the valueNumber (hashing) and valueEqual functions for most binary operations, slightly modified for clarity:

public abstract class Instruction extends Value { ... }

// The base class for binary operations
public abstract class Op2 extends Instruction {
    // Each binary operation has an opcode and two opearands
    public final int opcode;  // (IMUL, IADD, ...)
    Value x;
    Value y;

    @Override
    public int valueNumber() {
        // There are other fields but only opcode, and operands get hashed.
        // Always set at least one bit in case the hash wraps to zero.
        return 0x20000000
        | (opcode
           + 7  * System.identityHashCode(x)
           + 11 * System.identityHashCode(y));
    }

    @Override
    public boolean valueEqual(Instruction i) {
        if (i instanceof Op2) {
            Op2 o = (Op2) i;
            return opcode == o.opcode && x == o.x && y == o.y;
        }
        return false;
    }
}

The rest of the value numbering implementation assumes that if a valueNumber function returns 0, it does not wish to be considered for value numbering. Why might an instruction opt-out of value numbering?

Pure vs impure

An instruction might opt out of value numbering if it is not “pure”.

Some instructions are not pure. Purity is in the eye of the beholder, but in general it means that an instruction does not interact with the state of the outside world, except for trivial computation on its operands. (What does it mean to de-duplicate/cache/reuse printf?)

A load from an array object is also not a pure operation². The load operation implicitly relies on the state of the memory. Also, even if the array was known-constant, in some runtime systems, the load might raise an exception. Changing the source location where an exception is raised is generally frowned upon. Languages such as Java often have requirements about where exceptions are raised codified in their specifications.

We’ll work only on pure operations for now, but we’ll come back to this later. We do often want to optimize impure operations as well!

We’ll start off with the simplest form of value numbering, which operates only on linear sequences of instructions, like basic blocks or traces.

Local value numbering

Let’s build a small implementation of local value numbering (LVN). We’ll start with straight-line code—no branches or anything tricky.

Most compiler optimizations on control-flow graphs (CFGs) iterate over the instructions “top to bottom”³ and it seems like we can do the same thing here too.

From what we’ve seen so far optimizing our made-up IR snippet, we can do something like this:

initialize a map from instruction numbers to instruction pointers
for each instruction i
- if i wants to participate in value numbering
  - if i’s value number is already in the map, replace all pointers to i in the rest of the program with the corresponding value from the map
  - otherwise, add i to the map

The find-and-replace, remember, is not a literal find-and-replace, but instead something like:

instr.opcode = "Assign"
instr.operands[0] = replacement

instr.make_equal_to(replacement)

(if you have been following along with the toy optimizer series)

This several-line function (as long as you already have a hash map and a union-find available to you) is enough to build local value numbering! And real compilers are built this way, too.

If you don’t believe me, take a look at this slightly edited snippet from Maxine’s value numbering implementation. It has all of the components we just talked about: iterating over instructions, map lookup, and some substitution.

// Local value numbering
BlockBegin block = ...;
ValueMap currentMap = new ValueMap();
InstructionSubstituter subst = new InstructionSubstituter();

// visit all instructions of this block
for (Instruction instr = block.next(); instr != null; instr = instr.next()) {
    // attempt value numbering (uses valueNumber() and valueEqual())
    //
    // return a previous instruction if it exists in the map, or insert the
    // current instruction into the map and return it
    Instruction f = currentMap.findInsert(instr);
    if (f != instr) {
        // remember the replacement in the union-find
        subst.setSubst(instr, f);
    }
}

This alone will get you pretty far. Code generators of all shapes tend to leave messy repeated computations all over their generated code and this will make short work of them.

Sometimes, though, your computations are spread across control flow—over multiple basic blocks. What do you do then?

Global value numbering

Computing value numbers for an entire function is called global value numbering (GVN) and it requires dealing with control flow (if, loops, etc). I don’t just mean that for an entire function, we run local value numbering block-by-block. Global value numbering implies that expressions can be de-duplicated and shared across blocks.

Let’s tackle control flow case by case.

First is the simple case from above: one block. In this case, we can go top to bottom with our value numbering and do alright.

The second case is also reasonable to handle: one block flowing into another. In this case, we can still go top to bottom. We just have to find a way to iterate over the blocks.

If we’re not going to share value maps between blocks, the order doesn’t matter. But since the point of global value numbering is to share values, we have to iterate them in topological order (reverse post order (RPO)). This ensures that predecessors get visited before successors. If you have bb0 -> bb1, we have to visit first bb0 and then bb1.

Because of how SSA works and how CFGs work, the second block can “look up” into the first block and use the values from it. To get global value numbering working, we have to copy bb0’s value map before we start processing bb1 so we can re-use the instructions.

Maybe something like:

value_map = ValueMap()
for block in function.reverse_post_order():
    local_value_numbering(block, value_map)

Then the expressions can accrue across blocks. bb1 can re-use the already-computed Add v0, 1 from bb0 because it is still in the map.

…but this breaks as soon as you have control-flow splits. Consider the following shape graph:

We’re going to iterate over that graph in one of two orders: A B C or A C B. In either case, we’re going to be adding all this stuff into the value map from one block (say, B) that is not actually available to its sibling block (say, C).

When I say “not available”, I mean “would not have been computed before”. This is because we execute either A then B or A then C. There’s no world in which we execute B then C.

But alright, look at a third case where there is such a world: a control-flow join. In this diagram, we have two predecessor blocks B and C each flowing into D. In this diagram, B always flows into D and also C always flows into D. So the iterator order is fine, right?

Well, still no. We have the same sibling problem as before. B and C still can’t share value maps.

We also have a weird question when we enter D: where did we come from? If we came from B, we can re-use expressions from B. If we came from C, we can re-use expressions from C. But we cannot in general know which predecessor block we came from.

The only block we know for sure that we executed before D is A. This means we can re-use A’s value map in D because we can guarantee that all execution paths that enter D have previously gone through A.

This relationship is called a dominator relationship and this is the key to one style of global value numbering that we’re going to talk about in this post. A block can always use the value map from any other block that dominates it. For completeness’ sake, in the diamond diagram, A dominates each of B and C, too.

We can compute dominators a couple of ways⁴, but that’s a little bit out of scope for this blog post. If we assume that we have dominator information available in our CFG, we can use that for global value numbering. And that’s just what—you guessed it—Maxine VM does.

It iterates over all blocks in reverse post-order, doing local value numbering, threading through value maps from dominator blocks. In this case, their method dominator gets the immediate dominator: the “closest” dominator block of all the blocks that dominate the current one.

public class GlobalValueNumberer {
    final HashMap<BlockBegin, ValueMap> valueMaps;
    final InstructionSubstituter subst;
    ValueMap currentMap;

    public GlobalValueNumberer(IR ir) {
        this.subst = new InstructionSubstituter(ir);
        // reverse post-order
        List<BlockBegin> blocks = ir.linearScanOrder();
        valueMaps = new HashMap<BlockBegin, ValueMap>(blocks.size());
        optimize(blocks);
        subst.finish();
    }

    void optimize(List<BlockBegin> blocks) {
        int numBlocks = blocks.size();
        BlockBegin startBlock = blocks.get(0);

        // initial value map, with nesting 0
        valueMaps.put(startBlock, new ValueMap());

        for (int i = 1; i < numBlocks; i++) {
            // iterate through all the blocks
            BlockBegin block = blocks.get(i);
            BlockBegin dominator = block.dominator();

            // create new value map with increased nesting
            currentMap = new ValueMap(valueMaps.get(dominator));

            // << INSERT LOCAL VALUE NUMBERING HERE >>

            // remember value map for successors
            valueMaps.put(block, currentMap);
        }
    }
}

And that’s it! That’s the core of Maxine’s GVN implementation. I love how short it is. For not very much code, you can remove a lot of duplicate pure SSA instructions.

This does still work with loops, but with some caveats. From p7 of Briggs GVN:

The φ-functions require special treatment. Before the compiler can analyze the φ-functions in a block, it must previously have assigned value numbers to all of the inputs. This is not possible in all cases; specifically, any φ-function input whose value flows along a back edge (with respect to the dominator tree) cannot have a value number. If any of the parameters of a φ-function have not been assigned a value number, then the compiler cannot analyze the φ-function, and it must assign a unique, new value number to the result.

So: we can treat phis as opaque (not numberable) and it works as-is. Otherwise we have to do a little more work.

It also talks about eliminating useless phis, which is optional, but would the strengthen global value numbering pass: it makes more information transparent.

But what if we want to handle impure instructions?

State management and invalidation

Languages such as Java allow for reading fields from the this/self object within methods as if the field were a variable name. This makes code like the following common:

class CPU {
    private void exec_adc() {
        int result_int = regA + fetched_data + flagCARRY;
        byte result = (byte) result_int;
        // ...
        int a = result_int ^ regA;
        int b = result_int ^ fetched_data;
        // ...
        regA = result;
    }
}

Each of these reference to regA and fetched_data is an implicit reference to this.regA or this.fetched_data, which is semantically a field load off an object. You can see it in the bytecode (thanks, Matt Godbolt):

class CPU {
  int regA;

  int fetched_data;

  int flagCARRY;

  CPU();
         0: aload_0
         1: invokespecial #1                  // Method java/lang/Object."<init>":()V
         4: return


  private void exec_adc();
         0: aload_0
         1: getfield      #7                  // Field regA:I
         4: aload_0
         // ...
        20: getfield      #7                  // Field regA:I
        23: ixor
        24: istore_3
        25: iload_1
        26: aload_0
        27: getfield      #13                 // Field fetched_data:I
        30: ixor
        31: istore        4
        33: aload_0
        34: iload_2
        35: putfield      #7                  // Field regA:I
        38: return
}

When straightforwardly building an SSA IR from the JVM bytecode for this method, you will end up with a bunch of IR that looks like this:

v0 = LoadField self, :regA
v1 = LoadField self, :fetched_data
v2 = LoadField self, :flagCARRY
v3 = IntAdd v0, v1
v4 = IntAdd v3, v2
// ...
v7 = LoadField self, :regA
v8 = IntXor v4, v7
v9 = LoadField self, :fetched_data
v10 = IntXor v4, v9
// ...
StoreField self, :regA, ...

Pretty much the same as the bytecode. Even though no code in the middle could modify the field regA (which would require a re-load), we still have a duplicate load. Bummer.

I don’t want to re-hash this too much but it’s possible to fold Load and store forwarding into your GVN implementation by either:

doing load-store forwarding as part of local value numbering and clearing memory information from the value map at the end of each block, or
keeping track of effects across blocks

See, there’s nothing fundamentally stopping you from tracking the state of your heap at compile-time across blocks. You just have to do a little more bookkeeping. In our dominator-based GVN implementation, for example, you can:

track heap write effects for each block
at the start of each block B, union all of the “kill” sets for every block back to its immediate dominator
finally, remove the stuff that got killed from the dominator’s value map

Not so bad.

Maxine doesn’t do global memory tracking, but they do a limited form of load-store forwarding while building their HIR from bytecode: see GraphBuilder which uses the MemoryMap to help track this stuff. At least they would not have the same duplicate LoadField instructions in the example above!

We’ve now looked at one kind of value numbering and one implementation of it. What else is out there?

Out in the world

Apparently, you can get better results by having a unified hash table (p9 of Briggs GVN) of expressions, not limiting the value map to dominator-available expressions. Not 100% on how this works yet. They note:

Using a unified hash-table has one important algorithmic consequence. Replacements cannot be performed on-line because the table no longer reflects availability.

Which is the first time that it occurred to me that hash-based value numbering with dominators was an approximation of available expression analysis.

There’s also a totally different kind of value numbering called value partitioning (p12 of Briggs GVN). See also a nice blog post about this by Allen Wang from the Cornell compiler course. I think this mostly replaces the hashing bit, and you still need some other thing for the available expressions bit.

Ben Titzer and Seth Goldstein have some good slides from CMU where they talk about the worklist dataflow approach. Apparently this is slower but gets you more available expressions than just looking to dominator blocks. I wonder how much it differs from dominator+unified hash table.

While Maxine uses hash table cloning to copy value maps from dominator blocks, there are also compilers such as Cranelift that use scoped hash maps to track this information more efficiently. (Though Amanieu notes that you may not need a scoped hash map and instead can tag values in your value map with the block they came from, ignoring non-dominating values with a quick check. The dominance check makes sense but I haven’t internalized how this affects the set of available expressions yet.)

You may be wondering if this kind of algorithm even helps at all in a dynamic language JIT context. Surely everything is too dynamic, right? Actually, no! The JIT hopes to eliminate a lot of method calls and dynamic behaviors, replacing them with guards, assumptions, and simpler operations. These strength reductions often leave behind a lot of repeated instructions. Just the other day, Kokubun filed a value-numbering-like PR to clean up some of the waste.

ART has a recent blog post about speeding up GVN.

Implementations

Instead of iterating over the reverse post-order of the blocks in the CFG, some implementations build a top-down dominator tree (nodes pointing to blocks dominated by that node) and then walk that tree depth-first. This lets them push and pop from a scoped hash table, or use generations, or something else smart like a fast pre-order/post-order dominance check.

This iteration order is a different iteration order but the resulting algorithm is equivalent (except that the RPO does a lot more allocating). This is because the shared property is that if block A dominates block B, A will be visited before B.

Wrapping up; bits and bobbles

Go forth and give your values more numbers.

There’s been an ongoing discussion with Phil Zucker on SSI, GVN, acyclic egraphs, and scoped union-find. TODO summarize

Acyclic e-graphs

Commutativity; canonicalization

Seeding alternative representations into the GVN

Aegraphs and union-find during GVN https://cfallin.org/blog/2026/04/09/aegraph/ canonicalize

https://github.com/bytecodealliance/rfcs/blob/main/accepted/cranelift-egraph.md

https://github.com/bytecodealliance/wasmtime/issues/9049

https://github.com/bytecodealliance/wasmtime/issues/4371

Partial redundancy elimination

https://cs.wheaton.edu/~tvandrun/writings/cc04.pdf

Writing this post is roughly the time when I realized that the whole time I was wondering why Cinder did not use union-find for rewriting, it actually did! Optimizing instruction X = A + 0 by replacing with X = Assign A followed by copy propagation is equivalent to union-find. ↩
In some forms of SSA, like heap-array SSA or sea of nodes, it’s possible to more easily de-duplicate loads because the memory representation has been folded into (modeled in) the IR. ↩
The order is a little more complicated than that: reverse post-order (RPO). And there’s a paper called “A Simple Algorithm for Global Data Flow Analysis Problems” that I don’t yet have a PDF for that claims that RPO is optimal for solving dataflow problems. ↩
There’s the iterative dataflow way (described in the Cooper paper (PDF)), Lengauer-Tarjan (PDF), the Engineered Algorithm (PDF), hybrid/Semi-NCA approach (PDF), … ↩

Using Perfetto in ZJIT

Fri, 27 Mar 2026 00:00:00 +0000

Originally published on Rails At Scale.

Look! A trace of slow events in a benchmark! Hover over the image to see it get bigger.

A sneak preview of what the trace looks like.

Now read on to see what the slow events are and how we got this pretty picture.

The rules

The first rule of just-in-time compilers is: you stay in JIT code. The second rule of JIT is: you STAY in JIT code!

When control leaves the compiled code to run in the interpreter—what the ZJIT team calls either a “side-exit” or a “deopt”, depending on who you talk to—things slow down. In a well-tuned system, this should happen pretty rarely. Right now, because we’re still bringing up the compiler and runtime system, it happens more than we would like.

We’re reducing the number of exits over time.

Lies, damned lies, and statistics

We can track our side-exit reduction progress with --zjit-stats, which, on process exit, prints out a tidy summary of the counters for all of the bad stuff we track. It’s got side-exits. It’s got calls to C code. It’s got calls to slow-path runtime helpers. It’s got everything.

Here is a chopped-up sample of stats output for the Lobsters benchmark, which is a large Rails app:

$ WARMUP_ITRS=0 MIN_BENCH_ITRS=20 MIN_BENCH_TIME=0 ruby --zjit-stats benchmarks/lobsters/benchmark.rb
...
***ZJIT: Printing ZJIT statistics on exit***
...
Top-20 side exit reasons (100.0% of total 12,549,876):
                   guard_type_failure: 6,020,734 (48.0%)
                  guard_shape_failure: 5,556,147 (44.3%)
  block_param_proxy_not_iseq_or_ifunc:   445,358 ( 3.5%)
                   unhandled_hir_insn:   215,168 ( 1.7%)
                        compile_error:   181,474 ( 1.4%)
...
compiled_iseq_count:                               5,581
failed_iseq_count:                                     2
compile_time:                                    1,443ms
...
guard_type_count:                            133,425,094
guard_type_exit_ratio:                              4.5%
guard_shape_count:                            49,386,694
guard_shape_exit_ratio:                            11.3%
...
code_region_bytes:                            31,571,968
side_exit_size_ratio:                              33.1%
zjit_alloc_bytes:                             19,329,659
total_mem_bytes:                              50,901,627
...
ratio_in_zjit:                                     82.8%
$

(I’ve cut out significant chunks of the stats output and replaced them with ... because it’s overwhelming the first time you see it.)

The first thing you might note is that the thing I just described as terrible for performance is happening over twelve million times. The second thing you might notice is that despite this, we’re staying in JIT code seemingly a high percentage of the time. Or are we? Is 80% high? Is a 4.5% class guard miss ratio high? What about 11% for shapes? It’s hard to say.

The counters are great because they’re quick and they’re reasonably stable proxies for performance. There’s no substitute for painstaking measurements on a quiet machine but if the counter for Bad Slow Thing goes down (and others do not go up), we’re probably doing a good job.

But they’re not great for building intuition. For intuition, we want more tangible feeling numbers. We want to see things.

Building intuition

The third thing is that you might ask yourself “self, where are these exits coming from?” Unfortunately, counters cannot tell you that. For that, we want stack traces. This lets us know where in the guest (Ruby) code triggers an exit.

Ideally also we would want some notion of time: we would want to know not just where these events happen but also when. Are the exits happening early, at application boot? At warmup? Even during what should be steady state application time? Hard to say.

So we need more tools. Thankfully, Perfetto exists. Perfetto is a system for visualizing and analyzing traces and profiles that your application generates. It has both a web UI and a command-line UI.

We can emit traces for Perfetto and visualize them there.

A look at Perfetto

Take a look at this sample ZJIT Perfetto trace generated by running Ruby with --zjit-trace-exits¹. What do you see?

I see a couple arrows on the left. Arrows indicate “instant” point-in-time events. Then I see a mess of purple to the right of that until the end of the trace.

Hover over an arrow. Find out that each arrow is a side-exit. Scream silently.

But it’s a friendly arrow. It tells you what the side-exit reason is. If you click it, it even tells you the stack trace in the pop-up panel on the bottom. If we click a couple of them, maybe we can learn more.

We can also zoom by mousing over the track, holding Ctrl, and scrolling. That will get us look closer. But there are so many…

Fortunately, Perfetto also provides a SQL interface to the traces. We can write a query to aggregate all of the side exit events from the slice table and line them up with the topmost method from the backtrace arguments in the args table:

SELECT
  s.name AS reason,
  a.display_value AS method,
  COUNT(*) AS count
FROM slice s
JOIN args a ON a.arg_set_id = s.arg_set_id AND a.key = '0'
GROUP BY s.name, a.display_value
ORDER BY count DESC

This pulls up a query box at the bottom showing us that there are a couple big hotspots:

Query results showing in columns left to right: reason for side-exit, method that exited, and count. The top three are above 1k but it quickly falls off after that.

It even has a helpful option to export the results Markdown table so I can paste (an edited version) into this blog post:

reason	method	count
GuardShape(ShapeId(2475))	ActiveModel::AttributeRegistration::ClassMethods#attribute_types	5119
GuardShape(ShapeId(2099268))	ActiveRecord::ConnectionAdapters::AbstractAdapter#extended_type_map_key	2295
GuardType(FalseClass)	ActiveModel::Type::Value#cast	1025
GuardShape(ShapeId(2099698))	ActiveRecord::Associations#association_instance_get	904
BlockParamProxyNotIseqOrIfunc	ActiveRecord::AttributeMethods::Read#_read_attribute	902
GuardShape(ShapeId(526450))	Rack::Request::Env#get_header	636
GuardType(Class[class_exact*:Class@VALUE(0x128c60100)])	ActiveRecord::Base._reflections	622
GuardType(ObjectSubclass[class_exact:Story])	ActiveRecord::Associations#association	565
GuardShape(ShapeId(2098982))	ActiveRecord::Reflection::AssociationReflection#polymorphic?	510
GuardType(StringSubclass[class_exact:ActiveSupport::SafeBuffer])	ActionView::OutputBuffer#<<	500
GuardShape(ShapeId(2475))	ActiveRecord::AttributeMethods::PrimaryKey::ClassMethods#primary_key	492
GuardType(ObjectSubclass[class_exact:ActiveModel::Type::String])	ActiveModel::Type::Value#deserialize	442
GuardShape(ShapeId(2098982))	ActiveRecord::Reflection::AssociationReflection#deprecated?	376
GuardType(ObjectSubclass[class_exact:Bundler::Dependency])	Gem::Dependency#matches_spec?	355
UnhandledHIRInvokeBuiltin	Time#initialize	346

Looks like we should figure out why we’re having shape misses so much and that will clear up a lot of exits. (Hint: it’s because once we make our first guess about what we think the object shape will be, we don’t re-assess… yet.)

This has been a taste of Perfetto. There’s probably a lot more to explore. Please join the ZJIT Zulip and let us know if you have any cool tracing or exploring tricks.

Now I’ll explain how you too can use Perfetto from your system. Adding support to ZJIT was pretty straightforward.

Implementation

The first thing is that you’ll need some way to get trace data out of your system. We write to a file with a well-known location (/tmp/perfetto-PID.fxt), but you could do any number of things. Perhaps you can stream events over a socket to another process, or to a server that aggregates them, or store them internally and expose a webserver that serves them over the internet, or… anything, really.

Once you have that, you need a couple lines of code to emit the data. Perfetto accepts a number of formats. For example, in his excellent blog post, Tristan Hume opens with such a simple snippet of code for logging Chromium Trace JSON-formatted events (lightly modified by me):

event_name = ...
timestamp = ...
duration = ...
f = open('trace.json','a')
f.write("[\n")

# ... emit some events here ...

# Log a single event
f.write('{"name": "%s", "ts": %d, "dur": %d, "cat": "hi", "ph": "X", "pid": 1, "tid": 1, "args": {}},\n' %
  (event_name, timestamp, duration))

# ... emit some events here ...

# ... at process exit, close the file ...
f.write("]") # this closing ] isn't actually required
f.close()

This snippet is great. It shows, end-to-end, writing a stream of one event. It is a complete (X) event, as opposed to either:

two discrete timestamped begin (B) and end (E) events that book-end something, or
an instant (i) event that has no duration, or
a couple other event types in the Chromium Trace Event Format doc

It was enough to get me started. Since it’s JSON, and we have a lot of side exits, the trace quickly ballooned to 8GB large for a several second benchmark. Not great. Now, part of this is our fault—we should side exit less—and part of it is just the verbosity of JSON.

Thankfully, Perfetto ingests more compact binary formats, such as the Fuchsia trace format. In addition to being more compact, FXT even supports string interning. After modifying the tracer to emit FXT, we ended with closer to 100MB for the same benchmark.

We can reduce further by sampling—not writing every exit to the trace, but instead every K exits (for some (probably prime) K). This is why we provide the --zjit-trace-exits-sample-rate=K option.

Check out the trace writer implementation from the point this article was written.

Tracing more things

We could trace:

When methods get compiled
How big the generated code is
How long each compile phase takes
When (and where) invalidation events happen
When (and where) allocations happen from JITed code
Garbage collection events
and more!

Conclusion

Visualizations are awesome. Get your data in the right format so you can ask the right questions easily. Thanks for Perfetto!

Also, looks like visualizations are now available in Perfetto canary. Time to go make some fun histograms…

This is also sampled/strobed, so not every exit is in there. This is just 1/K of them for some K that I don’t remember. ↩

A fuzzer for the Toy Optimizer

Wed, 25 Feb 2026 00:00:00 +0000

Another entry in the Toy Optimizer series.

It’s hard to get compiler optimizers right. Even if you build up a painstaking test suite by hand, you will likely miss corner cases, especially corner cases at the interactions of multiple components or multiple optimization passes.

I wanted to see if I could write a fuzzer to catch some of these bugs automatically. But a fuzzer alone isn’t much use without some correctness oracle—in this case, we want a more interesting bug than accidentally crashing the optimizer. We want to see if the optimizer introduces a correctness bug in the program.

So I set off in the most straightforward way possible, inspired by my hazy memories of a former CF blog post.

Generating programs

Generating random programs isn’t so bad. We have program generation APIs and we can dynamically pick which ones we want to call. I wrote a small loop that generates loads from and stores to the arguments at random offsets and with random values, and escapes to random instructions with outputs. The idea with the escape is to keep track of the values as if there was some other function relying on them.

def generate_program():
    bb = Block()
    args = [bb.getarg(i) for i in range(3)]
    num_ops = random.randint(0, 30)
    ops_with_values = args[:]
    for _ in range(num_ops):
        op = random.choice(["load", "store", "escape"])
        arg = random.choice(args)
        a_value = random.choice(ops_with_values)
        offset = random.randint(0, 4)
        if op == "load":
            v = bb.load(arg, offset)
            ops_with_values.append(v)
        elif op == "store":
            value = random.randint(0, 10)
            bb.store(arg, offset, value)
        elif op == "escape":
            bb.escape(a_value)
        else:
            raise NotImplementedError(f"Unknown operation {op}")
    return bb

This generates random programs. Here is an example stringified random program:

var0 = getarg(0)
var1 = getarg(1)
var2 = getarg(2)
var3 = load(var2, 0)
var4 = load(var0, 1)
var5 = load(var1, 1)
var6 = escape(var0)
var7 = store(var0, 2, 3)
var8 = store(var2, 0, 7)

No idea what would generate something like this, but oh well.

Verifying programs

Then we want to come up with our invariants. I picked the invariant that, under the same preconditions, the heap will look the same after running an optimized program as it would under an un-optimized program¹. So we can delete instructions, but if we don’t have a load-bearing store, store the wrong information, or cache stale loads, we will probably catch that.

def verify_program(bb):
    before_no_alias = interpret_program(bb, ["a", "b", "c"])
    a = "a"
    before_alias = interpret_program(bb, [a, a, a])
    optimized = optimize_load_store(bb)
    after_no_alias = interpret_program(optimized, ["a", "b", "c"])
    after_alias = interpret_program(optimized, [a, a, a])
    assert before_no_alias == after_no_alias
    assert before_alias == after_alias

I have a very silly verifier that tests two cases: one where the arguments do not alias and one where they are all the same object. Generating partial aliases would be a good extension here.

Last, we have the interpreter.

Running programs

The interpreter is responsible for keeping track of the heap (as indexed by (object, offset) pairs) as well as the results of the various instructions.

We keep track of the escaped values so we can see results of some instructions even if they do not get written back to the heap. Maybe we should be escapeing all instructions with output instead of only random ones. Who knows.

def interpret_program(bb, args):
    heap = {}
    ssa = {}
    escaped = []
    for op in bb:
        if op.name == "getarg":
            ssa[op] = args[get_num(op, 0)]
        elif op.name == "store":
            obj = ssa[op.arg(0)]
            offset = get_num(op, 1)
            value = get_num(op, 2)
            heap[(obj, offset)] = value
        elif op.name == "load":
            obj = ssa[op.arg(0)]
            offset = get_num(op, 1)
            value = heap.get((obj, offset), "unknown")
            ssa[op] = value
        elif op.name == "escape":
            value = op.arg(0)
            if isinstance(value, Constant):
                escaped.append(value.value)
            else:
                escaped.append(ssa[value])
        else:
            raise NotImplementedError(f"Unknown operation {op.name}")
    heap["escaped"] = escaped
    return heap

Then we return the heap so that the verifier can check.

The harness

Then we run a bunch of random tests through the verifier!

def test_random_programs():
    # Remove random.seed if using in CI... instead print the seed out so you
    # can reproduce crashes if you find them
    random.seed(0)
    num_programs = 100000
    for i in range(num_programs):
        program = generate_program()
        verify_program(program)

The number of programs is configurable. Or you could make this while True. But due to how simple the optimizer is, we will find all the possible bugs pretty quickly.

I initially started writing this post because I thought I had found a bug, but it turns out that I had, with CF’s help, in 2022, walked through every possible case in the “buggy” situation, and the optimizer handles those cases correctly. That explains why the verifier didn’t find that bug!

Testing the verifier

So does it work? If you run it, it’ll hang for a bit and then report no issues. That’s helpful, in a sense… it’s revealing that it is unable to find a certain class of bug in the optimizer.

Let’s comment out the main load-bearing pillar of correctness in the optimizer—removing aliasing writes—and see what happens.

We get a crash nearly instantly:

$ uv run --with pytest pytest loadstore.py -k random
...
=========================================== FAILURES ============================================
_____________________________________ test_random_programs ______________________________________

    def test_random_programs():
        random.seed(0)
        num_programs = 100000
        for i in range(num_programs):
            program = generate_program()
>           verify_program(program)

loadstore.py:617:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

bb = [Operation(getarg, [Constant(0)], None, None), Operation(getarg, [Constant(1)], None, None), Operation(getarg, [Consta...], None, None)], None, None), Operation(load, [Operation(getarg, [Constant(0)], None, None), Constant(0)], None, None)]

    def verify_program(bb):
        before_no_alias = interpret_program(bb, ["a", "b", "c"])
        a = "a"
        before_alias = interpret_program(bb, [a, a, a])
        optimized = optimize_load_store(bb)
        after_no_alias = interpret_program(optimized, ["a", "b", "c"])
        after_alias = interpret_program(optimized, [a, a, a])
        assert before_no_alias == after_no_alias
>       assert before_alias == after_alias
E       AssertionError: assert {('a', 0): 4,...', 3): 1, ...} == {('a', 0): 9,...', 3): 1, ...}
E
E         Omitting 4 identical items, use -vv to show
E         Differing items:
E         {('a', 0): 4} != {('a', 0): 9}
E         Use -v to get more diff

loadstore.py:610: AssertionError
==================================== short test summary info ====================================
FAILED loadstore.py::test_random_programs - AssertionError: assert {('a', 0): 4,...', 3): 1, ...} == {('a', 0): 9,...', 3): 1, ...}
=============================== 1 failed, 15 deselected in 0.04s ================================
$

We should probably use bb_to_str(bb) and bb_to_str(optimized) to print out the un-optimized and optimized traces in the assert failure messages. But we get a nice diff of the heap automatically, which is neat. And it points to an aliasing problem!

Full code

See the full code.

Extensions

Synthesize (different) types for non-aliasing objects and add them in info
Shrink/reduce failing examples down for easier debugging
Use Hypothesis for property-based testing, which CF notes also gives you shrinking
Use Z3 to encode the generated programs instead of randomly interpreting them

Thanks

Thank you to CF Bolz-Tereick for feedback on this post!

CF notes that this notion of equivalence works for this optimizer but not for one that does allocation removal (escape analysis). If we removed allocations and writes to them, we would be changing the heap results and our verifier would appear to fail. This means we have to, if we are to delete allocations, pick a more subtle definition of equivalence.

Perhaps something that looks like escape analysis in the verifier’s interpreter? ↩

Max Bernstein's Blog

Sorry for marking all the posts as unread

A quick look at zero-knowledge proofs

The paper’s protocol

One iteration

Tips and tricks

Step P1

Step V1

Step P2

Step V2

Probabilities

A networked demo

Encoding other NP-complete problems

Sudoku

Reduction

Wrapping up

The inliner is yielding benefits for ZJIT

Ruby: a refresher

JITs: a refresher

The code locality is in another tower

Inlining

So what does performance look like?

Wrapping up

Travel notes: PLDI Boulder

Sunday

Monday

Tuesday

Wednesday

Thursday

Friday

Saturday

A survey of inlining heuristics

Inlining: the “easy” part

When: the harder part

The heuristics

Call context and profiles: the other harder part

Splitting

Profile splitting

Bytecode inlining

Early tier with counters

Inline and analyze and hope

Thanks

The survey: bits and bobbles

Cinder: new

Cinder: old

PyPy

V8

V8 Hydrogen

V8 TurboFan

V8 Maglev

JavaScriptCore

SpiderMonkey

Wasmtime and Cranelift

HotSpot

TruffleRuby

.NET

Dart

HHVM

ART

JikesRVM

Other/research

Graal

Checking assembly with Z3

Thanks

Update

Travel notes: RubyKaigi Hakodate

Part one

Part two

Part three

Partial static single information form

When do we insert type refinements?

During SSA optimization

After which conditionals do we refine?

How does this compare to the complicated looking papers?

In other compilers

Aside: logic for e.g. HeapObject upgrade

Conclusion

Value numbering

Eliminating common subexpressions

Pure vs impure