Continuous integration

This page describes the current SSA builder and backend implementation. Start with the new text walkthrough for esper 1 syntax and the roadmap for the library-first migration boundary.

.github/workflows/ci.yml runs on every push to main and ssa, on every pull request, and on demand from the Actions tab. It exists because the suite had nothing running it: the recurring fault this project’s tests keep turning up is a check that cannot fail, and a suite nobody executes is the limiting case of one.

That shapes the workflow. Every step either asserts a number it expects to see or is not there at all. A step that shells out to a script and trusts its exit status is not enough when the script exits 0 on “no container runtime” and 0 again on “no driver built”.

The workflow sets shell: bash for every run: block, which is what turns pipefail on. The default is bash -e without it, and several steps pipe a suite into tee so that what was printed can be asserted on as well as what was returned. Without pipefail those steps would report tee’s status, and a failing suite would read as a pass — the same fault, introduced by the very thing meant to catch it.

The jobs

style

task stylecheck exits non-zero today: 32 long-function violations stand in the tree, and fixing them is separate work. Neither of the two easy answers is honest. Running it informationally makes it furniture nobody reads; deleting the rule silences it.

So it is a budget. The full report prints on every run, and the job fails when the count goes above STYLE_BUDGET, set in the workflow. When the number comes down the job says so and asks for the budget to be lowered, which is what keeps the ratchet from slipping.

The job also fails when the report has no TOTAL line at all. A tool that did not get as far as printing one has not found zero violations, and reading the absence as zero is the exact failure this workflow is here to avoid.

suite

Configures with xmake, builds, then runs three things:

• The spec corpus through task test. The runner’s summary line is parsed rather than trusted: it exits 0 when it finds no specs at all, so the job asserts that at least SPEC_FLOOR of them ran, that none failed, and that no more than SKIP_CEILING were skipped. The skip ceiling is exact. Three specs carry an @skip annotation; a fourth means a test quietly stopped running, and that should be a visible edit rather than a line in a log.
• The white-box pass suites, each xmake run invoked separately with the statuses accumulated. See the note on task unit-test below.
• task api-test, which compiles tests/api_smoke.c against the installed header alone and runs the builder example.

clang is checked for up front. esper_runner shells out to it for every @checkC and @checkLLVM fence, and without it those fences report “C compilation failed” for four hundred specs, which is a confusing way to learn the runner image changed.

This job builds without the machine-code backend, and says so in its log. xmake.lua drops src/backends/tinyasm and disables esper-native-check and esper-target-check when it cannot find the sibling library, quietly and by design. That is the right default for a checkout without it, but it means this job runs a smaller library than a developer’s machine does.

machine-code

The differential check, the object-container check, the ABI interop check and the ELF cross lane. All four are the only steps that execute what the native backend emits rather than inspecting it.

All four also need tinyasm and tinylinker, which are private repositories. A runner with no credentials for them cannot build the native backend, and no amount of workflow changes that. So the lane is opt-in:

1. Create a token with read access to NickGlenn/tinyasm and NickGlenn/tinylinker.
2. Add it to this repository as a secret named SIBLING_REPOS_TOKEN.

Without the secret the gate job reports it and machine-code does not appear. With it, the lane checks out both siblings next to the main checkout, so the ../tinyasm default in xmake.lua is what gets exercised rather than a path only CI ever passes.

Order matters in the build: tinylinker first. tinyasm’s own configure looks for a built libtinylinker.a and drops object-file output without it, and esper’s configure would then find a tinyasm that cannot write an object. Only the libraries are built; the CLIs and examples in those repositories are their own suites’ business.

After the build the job asserts that esper, esper-native-check and esper-target-check all exist. Without that check, a checkout that silently lost a sibling would leave the job green having skipped every step below it.

Which cross targets

x86_64-linux only. The lane runs the emitted ELF in a container, and on an amd64 runner that container is native: no qemu, no binfmt, no emulation tax.

arm64-linux is deliberately left out. It would need qemu on an amd64 runner, and an emulated second ELF ABI costs far more minutes than it finds bugs; riscv64-linux needs two containers with one of them emulated, which is worse. Both stay a local and pre-release concern, via task cross-check.

tests/cross/run.sh exits 0 with “no container runtime” when Docker is absent, so the job asserts Docker is present first, and then greps the output for the “N program(s) agree” line rather than reading a zero exit as success. tests/abi/run.sh gets the same treatment: it exits 0 when it finds no driver and prints a skip line when it finds no C toolchain, so a green step is not by itself evidence that it ran.

What CI does not cover

• macOS and Windows. The suite runs on ubuntu-latest only. Mach-O and COFF output is checked by task target-check, which inspects the container each target emits, but nothing executes a Mach-O or a COFF binary in CI. task cross-check:local and task codeview-check remain local lanes.
• task debug-check and task codeview-check, which need lldb and LLVM’s CodeView reader respectively.
• task bench:check. A shared runner’s timings are too noisy to gate on.
• The riscv64 and Windows cross lanes, for the reasons above.

A note on task unit-test

The task used to be a for loop:

for t in copy_prop cse inline strength dominator debug builder; do
  xmake run esper-unit-$t
done

Task runs each command with errexit, so the loop stopped at the first suite that failed and abandoned the other six. It did report a non-zero status, so a failure was not hidden — but a run that found a fault in copy_prop said nothing at all about cse, inline, strength, dominator, debug or builder, which is six suites’ worth of coverage lost to one fault. In the measurement behind this change the loop produced six lines of output and no passes; accumulating the statuses produced all seven suites, fifty-three passing checks, the same failure, and the same non-zero exit.

That is the shape native-test-all in the same file already uses, for a related reason recorded in its own comment.

The workflow still invokes the suites itself rather than calling the task, so that a suite added to Taskfile.yml without being added to the workflow is a visible omission rather than a silent one.