How Our Rust-to-Zig Rewrite Is Going
Our team has spent a year and a half rewriting the Roc compiler from Rust into Zig, recently achieving feature parity. This massive effort unlocked hot code loading, zero-allocation pattern matching, and improved cross-compilation. Unlike quick ports, our rewrite addressed deep architectural flaws, proving that a fresh start was essential for the language's future performance and reliability.
Compilers are unusual in that scratch-rewrites are the norm among successful projects.
- steveklabnik
I think this is a fine post. But one comment:
> remember that for compilers which emit machine code, like roc and rustc, doing memory-unsafe things is a big part of the job
I don't really think that this is true, in the way that it's written.
I think that for the hot binary patching / code reloading features, yes, that is going to need unsafe. But for regular old "producing an executable" compilation? Emitting machine code isn't the part that requires unsafe. The language's runtime is a more likely site to find unsafe.
- landr0id
>ReleaseSafe catches use-after-free errors through runtime checks which panic if the program tries to use freed memory.
I don't know Zig so maybe they know something I don't, but I have seen no evidence that it catches any type of use-after-free including double-free?
While writing a blog post (below) I went through the documentation to figure out the possible runtime memory safety checks Zig can insert. The term "use-after-free" or "UaF" never occurs on that documentation page. Searching for "safety-checked" doesn't yield any related hits either.
Unless maybe they're using the DebugAllocator in release builds? Even that does not reliably surface UaF.
https://landaire.net/memory-safety-by-default-is-non-negotia...
- arthurbrown
Interesting that OCaml was flexible and expressive enough to be used as a prototype testbed but not chosen as the implementation language, especially given the maturity of both. I would be surprised if Zigs incremental builds could be meaningfully faster than dune's.
Cross compilation is great, but not mentioned in the "why Zig" section. Is memory control that crucial for a compiler?
Rust itself was originally written in OCaml, same with WASM. I'm curious about what milestone gets reached where the maintainers collectively decide to transition away.
- onlyrealcuzzo
Zig's incremental builds are DEFINITELY a killer feature. In the short term, I could see why you'd make a switch to get it. But, in the medium term, can we really not expect to see this in Rust in the somewhat near future?
I want to go fast, but I don't want to go fast just to shoot my foot off.
If only somehow we could get Rust's safety with all of Zig's features and Go's runtime without GC...
That's what I'm working on building [=
- bbkane
Tangentially relates, but if any Roc devs are around I'm curious about the use cases for Roc.
It's supposed to be a scripting language right you embed into your C ABI right?
Do you see it competing with WASM for the plugin use case (i.e. a really large Roc platform)? Why would an app author prefer to expose a Roc layer to their app rather than a WASM layer? With a WASM layer, plugin devs can write in any language.
Another use case I've heard from it is as a more app-level language (i.e. a really small Roc platform). Do you see it competing with Gleam for server side http code? Do you see it competing with Elm for client side code?
- maybebug
Nitpicking ahead:
I am not sure, but there might be a bug in their pattern matching example.
What happens if 'verb' is "GET" and 'path' is "/users/1234/posts/1234/extra_path/and/more/"? Will 'post_id' become "extra_path/and/more/"?
I tried running it in the sandbox, and it does indeed seem to buggily result in:
"Post ID: 1234/extra_path/and/more"
I suspect that the reason it is behaving like it is, is due to how it handles characters in the string literal. The example program exploits that only the slashes present in the string literal pattern are matched, to enable matching on 'page' having slashes. But then in the nested 'match', it forgot to account for any possible extra slashes.
Nitpicking end.
I have not read the whole post yet, but the pattern matching not requiring any allocations, seems very nice. The string literal patterns also seem interesting, though I am not completely sold on them, also as per the above possible bug. It seems really clean in some ways, but the specific semantics, I am not fully sure about. Maybe it is excellent, and is so clean and concise that it is overall less bug-prone than alternatives in other programming languages. I do not know.
- giancarlostoro
One thing I wish Rust would improve over time is the builds. Its one of the biggest sources of wasted storage space on all my computers, builds a ton of libraries can take tens of gigs, it adds up very quickly. Not sure what the best solution is, one I found is to set the global build folder so dependencies get reused across projects, but imho it should be an OOTB default behavior whatever the real solution should be.
- norir
This piece would have been a lot more compelling if they had actually done science on selecting a language for compiler development. From what I can tell, they had an untested hypothesis that a low level systems language is necessary for a high performance compiler https://www.roc-lang.org/faq#self-hosted-compiler and from that concluded that their only
choice besides rust was zig.
I know from experience that this initial assumption is wrong. Compiler performance is dominated by algorithms. The fastes managed languages tend to be at worst within a factor of two for wall time on any given algorithm. Algorithmic differences can be unbounded in their performance gaps. Zig itself is a perfect counterexample to the theory that writing a compiler in a low level systems language will lead to a fast compiler. Roc seems to compile at around 15k lines per second. That is not fast. There were evidently compilers written in ml that did 3k likes per second in 1998 https://flint.cs.yale.edu/cs421/case-for-ml.html
The zig rewrite of roc looks like the author's second compiler. Compiler and language design is a skill like any other and from my vantage point, they appear to have overcommitted to an initial design at the expense of developing their higher level design skills. In my opinion, the best thing they could do for the future of roc is stop working on their current compiler and use it to write a self hosting compiler for a much smaller subset of roc. They should be able to do that in l […]
- LAC-Tech
> I enjoy Rust, I've taught a course on it, and I happily use it daily for my work at Zed. Despite what Internet comments might have us believe, it's extremely normal for one language to be the best fit for one project, while a different language turns out to be the best fit for a different project. One size does not actually fit all!
Amen. I like both Zig and Rust, and if I praise/criticise one of them, people act like I'm "switching", as if they were two exclusionary religions (though I think some people may indeed view them that way).
The stuff on memory safety written in the article is well worth reading, because in a lot of programmer discourse it's talked of as if it's some binary, that Rust Is Memory Safe, and Zig Is Not Memory safe. That's simply not the case.
- dev_l1x_be
Zig is a pre-1.0 language while Rust is post-1.0. This alone is settles which one to pick for may developers. The library support is probably favours Rust too. Rust build times are much slower than Zig, I get that, but I rarely optimize software for build times.