Beating the Compiler: Writing an Interpreter in Assembly

Matt Keeter wrote a fast interpreter for the Uxn CPU in Rust, but profiling revealed that the dispatch loop consumed over a third of runtime. Inspired by LuaJIT, he rewrote the interpreter in hand-written AArch64 assembly, using indirect threading and keeping all state in registers. The result is a 2400-line assembly implementation that outperforms the compiler-generated code.

Compilers are bad at generating code for interpreters, and it's possible to outperform them by writing your interpreter in assembly.
  1. compiler-guy

    There are literally thousands of compiler engineers who pore over the assembly a compiler generates, and then finds ways to make it better. I get paid to figure out where the compiler can do better, and often a step in that is to hand-code my own replacement. I then teach the compiler to do that.

    But even beyond that, the compiler can't make certain assumptions that an assembly writer can. Such as whether a callee-saved register really does need to be saved in some particular routine. Or even pushing an extra parameter in unusual cases.

    So it is entirely possible to beat the compiler, it's doable under certain circumstances, even today.

    But you also have the danger of your loving hand-crafted assembly beating the compiler today. But next year the compiler is even smarter, the hardware may have changed in subtle ways, and the compiler will know and improve the code it generates.

    Your hand-written code won't change unless you revisit it.

  2. MaulingMonkey

    > At this time there is no portable way to produce computed gotos or tail call optimization in compiled machine code from Rust.

    Nightly rust now has the `become` keyword:

    https://doc.rust-lang.org/std/keyword.become.html

    > `feature(explicit_tail_calls)` is currently incomplete and may not work properly.

    Works on my machine (tm). At least with toy examples. Including in full optimizationless debug mode, turning `call`s into `jmp`s ensuring `factorial(usize::MAX)` won't stack overflow probably maybe.

    https://rust.godbolt.org/z/7xf836E8K

    > checked_sub? wrapping_mul? MaulingMonkey, what's wrong with you?

    Eliminating debug-mode panic boilerplate.

  3. YuechenLi

    Compiler isn't really difficult in the way that many think it is, because it really isn't that difficult to write a basic C compiler for example, and writing a parser is pretty mechanical that there are a multitude of parser generators. The difficult part is that writing an optimized compiler is much more difficult than writing a correct compiler, and certain language features, like generics, closure, and async, propagates and touch every part of the language that the complexity grows exponentially, and it's really hard to balance performance, compile speed, and correctness against miscompiles.

    You can kinda see that in the many "compile TypeScript to native via LLVM" projects that showed up a lot recently. From my testing, none of them could beat V8/Node JIT in the majority of cases, and most of them are generally 10x-40x slower.

    But, we did have a great number of innovations in language design over the last decades that really closes the gap on how optimized a compiler can be over writing assembly directly: Rust's exhaustive match default null-less error handling and language level MIR, immutable data structures from functional languages to mainstream ones, TypeScript's compile time constraints as core part of the language, and Zig's `comptime` turning compile time metaprogramming to an integrated part of the language instead of C++ template metaprogramming.

    Obviously, it's not really possible to beat hand optimized C/C++ or directly authored assembly, but I think a well-de […]

  4. someonebaggy

    Compilers are pretty good. Really good, even. But languages, even C, are abstractions, which necessarily constrain the level below. This threaded-code jump thing is just not possible to express in C. Even the best abstraction can often be beaten by something the abstraction can't express. Self-modifying code is one example. So is this threaded interpreter with its non-structured control flow.

    But it takes longer. It's more difficult. That's why the abstraction exists and is still very useful despite its limitations.

  5. j2kun

    > In modern times, everyone knows that writing assembly is a fool's errand

    ffmpeg is like 10% assembly. I think something similar is true of all video encoders. OpenSSL and libsodium write some of their core math routines in assembly (e.g,. NTT).

    So maybe this myth should die?

More from this day

2026-10-05