This is the first in a planned series of posts about a new C compiler project called “novice”, a successor to my previous project naive. That project was intended to be a very from-first-principles, figure-it-out-as-I-go approach, with the goal of being self-hosting. In contrast, this project is intended as a way of learning about the state of the art in optimisation, with the rest done well enough to support this focus.

There are plenty of other interesting topics in modern compiler design: incremental compilation; query-based compilers; LSP; formatters / linters and other tooling; etc. That stuff is great and very necessary, it’s just not as interesting to me, and the scope is large enough already. For the same reason I’ll omit some components which I included in naive out of completionism. I won’t write a linker, libc, textual assembler, etc.

Ditching self-hosting allows me to use any language. I’ve picked Rust since it’s statically-typed, fast, has nice modern tooling, and has language features like generic data structures which I was sorely missing in naive. The overall design is pretty similar, based around an LLVM-like SSA form IR, but most components will be written pretty differently, using the lessons I learned from naive. The backend in particular I plan to make much more sophisticated, and to include a couple of targets, as multi-target support is an interesting topic.

I do want to make a nicer frontend than naive, but mainly because this makes debugging other parts of the compiler easier. For instance, in naive if I try compiling some large source file and it fails at some point, I have no way of telling where it fails in that file. I also want to support a larger set of language features so I can test on real-world projects, and having a well-written frontend makes that easier.

The ultimate goal is to get within some margin of clang/gcc. It’s hard to set an exact goal up front because I don’t really know what’s realistic. I assume that clang is very heavily into diminishing returns at this point. There are enough companies spending enough watts running code compiled by clang, so it makes sense for them to keep investing SWE-years for fractions of a percent improvement. But I have no idea what the curve looks like. I’ll have to see where I’m at initially and go from there.

There aren’t really any good open-source C benchmark suites which measure performance on a range of real-world projects, so this is something I plan to build out as a supplementary project. This could be of more general interest, comparing gcc/clang or other C compilers across the compile-time/runtime Pareto curve. This will be a nice motivator as new optimisation passes are implemented, and I can see the curves get closer.

Compile-time performance is another goal, and an area where novice might have a legitimate advantage. It definitely can’t match clang on the far end of the Pareto frontier, where we’re spending as much compile time as possible to eke out every minuscule scrap of runtime performance. But as a simpler project supporting fewer features, maybe it can beat clang at some other point on the frontier. This is only plausible to the extent that clang’s compile-time performance is degraded by supporting many complicated features. To be fair to clang, it is a well-engineered project written in a performance-minded style, and optimisation passes are difficult to write efficiently. Again, I’ll just have to see where it lands initially.

I also want to document the process this time — hopefully it serves as a nice window into modern compiler construction, while being fairly accessible due to being from the perspective of someone learning rather than an experienced professional or academic. In the same spirit, I want to make the source easy to read and well-documented, to serve as a learning resource of its own.

The next post will cover the first step of implementation — the CLI, driver, and some other infrastructure.