Building FPlus: what a compiler teaches you about software

I started FPlus to understand programming languages from the inside out. A compiler takes something that feels expressive and flexible to a person, then moves it through a sequence of increasingly precise representations until a machine can do something with it.

That makes compiler work a concentrated lesson in software design. Every stage needs a clear contract, errors need context, and decisions made early in the pipeline shape everything that follows.

The core flow is simple to describe: source becomes tokens, tokens become an abstract syntax tree, the tree is validated, and valid programs are emitted into a lower-level form. The interesting work lives inside the boundaries between those stages.

• Lexing turns characters into meaningful tokens.
• Parsing turns the token stream into a structured syntax tree.
• Semantic analysis checks names, types, and rules that grammar alone cannot express.
• Emission translates a valid program into its target representation.

The lexer is the first place raw text gains meaning. Keywords, identifiers, operators, literals, and punctuation become distinct token types with source positions. Those positions are not decorative metadata; they are what later stages need to point back to the exact place where a problem began.

A good lexer also makes ambiguity explicit. It has to decide whether two characters form one operator, when a number ends, and how comments or whitespace affect position tracking without affecting the program itself.

Parsing is where a flat stream becomes a hierarchy. Operator precedence, blocks, function calls, declarations, and expressions become nodes in an abstract syntax tree. The useful word is abstract: the tree should preserve the program's structure without carrying every piece of punctuation from the source.

Once the tree is clean, later passes become easier to write. A type checker can visit expression nodes instead of reinterpreting tokens, and an emitter can focus on meaning instead of syntax trivia.

A parser can confirm that a program has the right shape, but it cannot prove that every name exists or every value is used correctly. Semantic analysis handles those rules. It builds and consults symbol information, checks types, and reports conflicts that only become visible once different parts of the program are connected.

This stage taught me to prefer explicit passes over one large function that tries to understand everything at once. Separate passes are easier to test, easier to explain, and easier to extend when the language grows.

A compiler is often experienced through its failures. An accurate message with a useful source location can turn a broken program into a quick correction. A vague message can make even a small language feel hostile.

Building FPlus reinforced a broader product lesson: developer tools need interface design too. Their interface happens to be tokens, diagnostics, commands, and output, but clarity still determines whether the tool feels trustworthy.