Concepts — Template and JSX Compilers

Phase 5 · Spec areas §31, §39. Prerequisites: bi-03 (you have written a tokenizer), fw-02, fw-05.

0. Orientation

The 60-second version

Compilation is how a framework buys information it cannot get at runtime.

That single sentence explains every compiler-first framework. A template is statically analysable; a render() function generally is not. Everything Vue does that React historically could not — static hoisting, patch flags, compile-time dependency analysis — comes from that one asymmetry, and nothing else.

source -> lexer -> parser -> AST -> transform -> codegen -> render function
                                     ▲
                              knowledge is added HERE

The risk that makes this module matter more than it looks

Every compiler optimisation in the table below fails the same way when it is wrong: silently, as a correctness bug, with no error and no stack trace pointing at the compiler.

A slow program is annoying. A program that silently shows stale data is a data-integrity incident. Any team shipping a compiler optimisation needs a testing strategy that runs the same suite with optimisations on and off and compares — which is Chromium's virtual test suites idea (bi-13) applied to a compiler.

The second thread: a compiler's UX is its error messages

The build order puts source locations at step 2, before transforms exist, and that ordering is not incidental. You cannot recover position information you did not record, and retrofitting it into an AST is painful and always incomplete.

A compiler's error messages are the majority of its user interface. If you ever build a DSL, a config validator, or a schema checker for your organisation, this is the part that decides whether anyone adopts it.

What you should be able to do after this module

1. Why a Principal Engineer needs this

Compilation is how a framework buys information it cannot get at runtime. A template is statically analysable; a render() function generally is not. Everything a compiler-first framework does better — static hoisting, patch flags, dependency analysis, dead-code elimination — comes from that one asymmetry.

This module is also where bi-03's tokenizer pays off a second time. You have already written a state machine over markup; the skill transfers directly.

2. Mental Model

source -> lexer -> parser -> AST -> transform -> codegen -> render function

The interesting stage is transform, because that is where knowledge is added:

• Static hoisting — a subtree with no dynamic bindings is created once, not per render.
• Dynamic-node detection / patch flags — annotate which parts of a node can change, so the runtime diff can skip everything else. The compiler tells the runtime what to look at.
• Compile-time dependency analysis — determine what a piece of markup reads, without running it.

The general principle: move work from runtime to build time, and move knowledge from the author to the runtime. Compare bi-07: Blink compiles stylesheets into RuleFeatureSet indices for exactly the same reason.

3. Build order

1. Lexer for a small template language: text, interpolation {{ }}, elements, attributes, directives.
2. Parser → AST, with source locations on every node.
3. Transforms: interpolation, attribute binding, event binding, conditionals, loops.
4. Codegen → a render function returning your fw-05 VNodes.
5. Static hoisting: detect fully-static subtrees; emit them once.
6. Patch flags: annotate dynamic bindings; make your renderer honour them.
7. Error reporting with source locations — a caret-and-line message, not a stack trace.
8. Then: a JSX-to-createElement transform over real JS, using an existing JS parser.

4. Failure Lab

1. Drop source locations. Produce a compile error and try to act on it. This is why step 2 says every node.
2. Hoist a subtree that is not actually static (it reads a variable). Show the stale render.
3. Emit a wrong patch flag — mark a dynamic node static. Show the missed update. This is the central risk of compiler optimisation: a wrong annotation is a silent correctness bug, not a slow one.
4. Codegen that breaks on a legal input (nested quotes, unicode, an expression containing }}).

4.5 Deep dive: what a compiler can know, precisely

The asymmetry that justifies compiler-first frameworks, stated concretely.

A template compiler can determine, without running anything:

• which nodes are entirely static (no bindings anywhere in the subtree),
• which attributes on a node are dynamic, and which specific ones,
• the shape of the tree — how many children, of what types,
• which variables an expression references,
• whether a list has a stable key expression,
• whether an event handler is a stable reference or recreated each render.

A JSX/JS render function generally cannot, because the "template" is arbitrary code:

{items.map(renderRow)}          // renderRow could be anything
{cond ? <A/> : <B/>}            // fine, analysable
{makeElement()}                  // opaque

The compiler must be conservative wherever it cannot prove a fact — and conservatism costs exactly the optimisation you wanted.

This is why React's compiler arrived a decade after Vue's: it had to become a whole-function analysis with escape hatches, rather than a template transform, and that is a substantially harder problem. Understanding the difficulty is more useful than tracking who shipped what.

4.6 Deep dive: the optimisations, and what each risks

Every row's failure is a correctness bug that manifests as "the UI didn't update," with no error and no stack trace pointing at the compiler.

This is the defining risk of compiler optimisation and the reason fw-06 matters more than it looks. A slow program is annoying; a program that silently shows stale data is a data-integrity incident. Any team shipping a compiler optimisation needs a testing strategy that runs the same suite with optimisations on and off and compares — which is exactly Chromium's virtual test suites idea (bi-13) applied to a compiler.

4.7 Deep dive: source maps, and why they are always slightly wrong

Source maps map generated positions back to original ones. They are lossy by construction:

• Inlining destroys the one-to-one correspondence between call sites and functions.
• Hoisting moves code to a position that has no meaningful original location.
• Minification renames variables; the names field helps, but scopes get merged.
• Multiple transforms chained (TS → JSX → bundler → minifier) require composing maps, and each composition loses fidelity.

The practical consequences you will meet: breakpoints landing one line off, variables not inspectable under their original names, and stack traces that point at the wrong function after inlining.

Design guidance: every transform in your pipeline must produce a map, and the maps must compose. A single transform that does not is enough to break debugging for the whole chain — which is why "we just do a quick regex replace on the output" is a decision with a debugging cost nobody prices at the time.

4.8 Deep dive: error messages are the compiler's real UX

Step 7 of the build order asks for caret-and-line errors, and it is not busywork.

error: unclosed element <div>
  --> Card.vue:12:3
   |
12 |   <div class="card">
   |   ^^^^ opened here, never closed
   |

versus

TypeError: Cannot read properties of undefined (reading 'children')
    at transform (compiler.js:412)

Both indicate the same bug. One takes five seconds to fix; the other takes twenty minutes and teaches the user to distrust the tool.

Getting this right requires that every AST node carries its source range, from lexing onward — which is why the build order puts source locations at step 2, before transforms exist. Retrofitting positions into an AST is painful and always incomplete; you cannot recover information you did not record.

The general rule: a compiler's error messages are the majority of its user interface. If you ever build a DSL, a config validator, or a schema checker for your organisation, this is the part that determines whether people adopt it.

5. Trade-offs

Compile-time vs runtime knowledge. The compiler knows the template shape; the runtime knows the actual data. Neither is sufficient alone, which is why compiled frameworks still have runtimes.

Optimisation vs debuggability. Hoisting and flags make generated code less like the source. Source maps are the mitigation and they are never perfect.

A template DSL vs plain JS. A DSL is analysable and constrained; JSX is expressive and mostly opaque to static analysis. This is the actual root of the React/Vue architectural divergence — not syntax preference.

Core terminology

Mental models

Move work from runtime to build time, and move knowledge from the author to the runtime. Those are two different transfers and both are happening. Compare bi-07: Blink compiles stylesheets into RuleFeatureSet indices for exactly the same reason — precompute what is knowable so the hot path does less.

A compiler must be conservative wherever it cannot prove a fact, and conservatism costs exactly the optimisation you wanted. This is why the analysability asymmetry is decisive rather than incremental:

{cond ? <A/> : <B/>}     // analysable
{items.map(renderRow)}    // renderRow could be anything
{makeElement()}           // opaque

A wrong annotation is a correctness bug, not a performance bug. This inverts how you review compiler work. A performance regression shows up in a metric; a missed update shows up as a support ticket about stale data, weeks later, with no trace back to the compiler.

Every transform in the pipeline must emit a map, and the maps must compose. One transform that does not is enough to break debugging for the whole chain — which is why "we just do a quick regex replace on the output" is a decision with a debugging cost nobody prices at the time.

Source maps are always slightly wrong, and knowing why prevents wasted debugging. Inlining destroys the correspondence between call sites and functions; hoisting moves code to positions with no meaningful original; minification merges scopes. Expect breakpoints one line off and stack traces pointing at the wrong function after inlining.

Common misconceptions

1. "React and Vue differ on syntax." They differ on what a compiler can prove. React's compiler arrived a decade later because it had to become a whole-function analysis with escape hatches rather than a template transform — a substantially harder problem.

2. "Compiler optimisations are safe because the compiler is tested." The optimisations' failures are silent and data-shaped. Only differential testing — same suite, optimisations on and off — catches a wrong patch flag.

3. "Source maps solve the debuggability cost." They mitigate it. They are lossy by construction and get lossier with every composed transform.

4. "Static hoisting is obviously safe." Only if the subtree is genuinely static. Hoist something that reads a variable and you get a stale render shared across instances.

5. "Error messages are polish." They are the interface. A caret-and-line message takes five seconds to act on; a TypeError from inside the compiler takes twenty minutes and teaches the user to distrust the tool.

6. "Add source locations later." You cannot recover information you did not record. That is why it is step 2.

7. "More compile-time is strictly better." Compiled frameworks still have runtimes, because the compiler knows the template shape and the runtime knows the actual data. Neither is sufficient alone.

Interview talking points

• "Compilation is how a framework buys information it can't get at runtime. Templates are statically analysable and render functions generally aren't — that one asymmetry produces static hoisting, patch flags and dependency analysis, and it's why React's compiler took a decade: it had to be a whole-function analysis with escape hatches rather than a template transform."
• "The thing I'd insist on before enabling an aggressive compiler optimisation in a large org is differential testing — the same suite with optimisations on and off, compared. Every one of these optimisations fails silently as a correctness bug. A wrong patch flag means the update never appears, with no error and nothing pointing at the compiler."
• "A compiler's error messages are the majority of its user interface. That's why source ranges go on every AST node from lexing onward — you can't recover position information you didn't record, and retrofitting it is always incomplete. It's the same lesson for any DSL or config validator you build internally."
• "Source maps are lossy by construction — inlining destroys call-site correspondence, hoisting moves code to positions with no original, minification merges scopes. The practical rule is that every transform in the pipeline must emit a composable map, because one that doesn't breaks debugging for the whole chain."
• "The general principle is moving work to build time and knowledge from the author to the runtime. Blink does the same thing compiling stylesheets into RuleFeatureSet indices — precompute what's knowable so the hot path does less."

6. Principal Engineer Review

1. What can a template compiler know that a JSX compiler cannot, and what follows from that?

2. A wrong patch flag is a silent correctness bug. Design the testing strategy that makes compiler optimisations safe to ship.

3. Argue that frameworks should move as much as possible to compile time. Then argue that runtime flexibility is worth more. What decides for a given product?

4. Source maps are always imperfect. What would you require before enabling an aggressive optimisation in a large org?

5. React added a compiler after a decade of runtime-only. Reconstruct why that took so long, and what changed.

Connections to other modules

• bi-03 — prerequisite; you already wrote a state machine over markup, and the tokenizer skill transfers directly.
• fw-05 — the consumer. Patch flags are meaningless until patch() honours them, and the wrong-flag failure is only visible there.
• fw-07 — the bundler is the next compiler, with the same cache-invalidation problem and the same author-assertion escape hatch (sideEffects).
• fw-02 — compiler-assigned hook slots are the fourth row of the positional-storage table: move analysis to build time and you relax the runtime constraint.
• fw-11 — read Vue's compiler-core transform pipeline and patch-flag generation against yours.
• bi-07 — RuleFeatureSet is the same move: precompute an index at parse time so invalidation is cheap at runtime.
• bi-13 — virtual test suites are the differential-testing idea this module needs, already solved at Chromium scale.