Implementation roadmap

This page separates the intended product from the current implementation. Treat examples in design notes as proposals unless the text guide supports them.

Area	Current checkout	Next boundary
Text	Versioned reader/printer, on-demand DFA, mutable locals, sparse promotion	Complete metadata round trips and grammar hardening
In-memory IR	Opaque unit with SSA builder and passes	Public local IDs, local reads/writes, and inspection after optimization
Distribution	Static library and C/C++ interfaces	Generated standalone core header and separately selectable backend headers
Combining units	Independent-result merge, import resolution, private names, ID remapping	General callback flows, weak/common symbols, richer target/linkage metadata
Binary format	Versioned, bounded archives of core IR and debug tables	Schema evolution, configurable limits, streaming and allocation-cost reductions
C frontend	Separate C-shaped serialization experiment	Actual preprocessing, parsing and C semantic lowering
Performance	Synthetic phase benchmarks	Validated native/LLVM runtime comparisons

The intended backend headers cover LLVM, WebAssembly and C99, with native x86_64 and aarch64 through tinyasm. LLVM version adapters should have explicit version and capability boundaries. Shader emitters already exist in this checkout but still need independent packaging and coverage work. Vroom bytecode emission is a future backend; no working Vroom emitter is claimed here.

Merging is a compiler operation

esper_merge remaps types, constants, functions, globals, signatures and debug references, resolves imports, and returns a new independent unit. It rejects conflicting definitions and incompatible layouts without modifying the inputs. Existing whole-unit optimization can then see calls across the original unit boundaries. See the save/load/link guide for the API, function-reference restrictions, and exact preservation rules.

Weak symbols, common storage, a general relocatable callback representation, and full target/visibility metadata remain separate work. Plain integer function IDs cannot safely be treated as ordinary constants during linking.

Layout is part of meaning

The accepted design distinguishes these policies:

Policy	Intended use
`layout(c)`	Target C ABI field alignment and padding
`layout(explicit)`	Offsets, size and alignment chosen by the frontend
`layout(std140)`	Uniform-buffer layouts in shader interfaces
`layout(std430)`	Shader storage-buffer layouts
`layout(auto)`	Internal representation selected by Esper

Only natural layout(c) is accepted by the current reader. Packed layout and explicit alignment need real storage and lowering rules before their annotations can be accepted. C bitfields, unions and ABI classification require per-target checks; a layout name alone cannot make a C frontend interoperable.

Debug types should stay near their storage types

The intended extension attaches source names and display descriptions to type declarations. A slice has a concrete pointer/length storage layout and a source name such as []i32. Describe which fields are data and length using a restricted, validated expression model. Keep debug identity separate from storage identity, so two language types can share a machine representation.

DWARF and CodeView describe types and locations. LLDB/GDB visualizers and Visual Studio Natvis provide richer displays; these are related outputs, not simply additional DWARF fields. The current core has debug support, but does not yet accept or generate this portable custom-display schema. Track unavailable optimized variables honestly rather than assigning a misleading storage address.

Supporting a C compiler

Threads normally lower to platform/runtime calls, not a universal spawn opcode. The runtime contract must define calling conventions, ownership and synchronization. TLS needs per-thread storage, model/visibility rules and relocations in each backend; an ordinary global with a decorative tag is insufficient.

Atomics need validated load/store/RMW orders, compare-exchange success and failure orders, weak/strong behavior, alignment, and lock-free capability queries. Unsupported widths need an explicit library-call fallback or diagnostic. Volatile is separate from atomic synchronization. Dynamic libraries need import/export visibility, relocations, loader behavior and optional runtime symbol lookup.

Implement these capabilities in the core, verifier, serializers and each relevant backend together. A frontend should be able to query support and receive a clear error. Silently accepting a semantic annotation would produce incorrect programs.