Coroutines and Async

This page describes the current SSA builder and backend implementation. Its unversioned esp examples are compatibility snapshots, still accepted by the reader. Start with the new text walkthrough for esper 1 syntax and the roadmap for the library-first migration boundary.

Esper has no built-in coroutine or async support. Coroutines are implemented by the frontend through state machine lowering: the frontend transforms each coroutine into a regular function that takes a frame and dispatches on the current state to resume at the right point.

What we’re lowering

func countUp(start: Int32) -> Generator<Int32> {
    var i = start
    while true {
        yield i
        i += 1
    }
}

let gen = countUp(1)
gen.next()  // yields 1
gen.next()  // yields 2
gen.next()  // yields 3

The yield keyword suspends the function, returning a value to the caller, and later resumes from where it left off. The local variable i must survive across yields. The frontend must transform this into a regular function that can be called repeatedly, each time continuing from the last suspension point.

The lowering strategy

Every coroutine becomes:

A frame in memory holding the current state index and any local variables that live across yield points
A resume function that dispatches on the state index with jmp.table
An init function that allocates the frame and sets the initial state

Step-by-step lowering

Step 1: Identify yield points

The compiler scans the coroutine body and numbers each yield point. countUp has one yield inside the loop, so:

State 0: Initial entry. Execute from the top down to the first yield.
State 1: Resume after the yield. Increment i, loop back, yield again.

Step 2: Decide what lives in the frame and what stays in SSA

This is the decision the whole lowering turns on. An SSA value is live along an edge inside a function. A suspension is a ret, so nothing SSA holds survives it: the next resume enters through a fresh call with a fresh value table.

The rule is therefore mechanical. A local whose live range crosses a yield point goes in the frame and is reached with load and store. A local whose live range sits entirely between two yield points stays an SSA value and costs nothing. In countUp, i is written before the yield and read after it, so it goes in the frame. The address arithmetic that reaches it does not, and neither does the incremented value before it is stored back.

Step 3: Lay out the frame

// CounterFrame:
//   offset 0: i32 state_index (0 = initial, 1 = after yield)
//   offset 4: i32 i (the counter value)
//   offset 8: i32 yielded_value (output slot)

Twelve bytes. The state_index tells the resume function where to jump. The yielded_value slot is where the resume function writes the value that was yielded, so the caller can read it after resuming.

The frame is allocated with halloc, not salloc. salloc reserves storage for the duration of the call that reserved it, and the init function returns before the first resume runs, so a frame in that storage would already be gone.

Step 4: Build the init function and the accessor

Two of the three entry points are straight-line: one allocates the frame and writes its initial fields, the other reads the output slot back out. Both are addressed the same way, by adding a constant displacement to the frame pointer, which is worth a helper in the frontend because every coroutine field access goes through it.

/// Offsets into the coroutine frame, in the order the fields were laid
/// out: the state index first, then one slot per local that outlives a
/// suspension, then the output slot.
enum { STATE_OFF = 0, I_OFF = 4, YIELD_OFF = 8, FRAME_SIZE = 12 };

/// Address of a frame field, as `frame + offset`.
static EsperVal frame_slot(EsperBuilder *b, EsperVal frame, int32_t offset) {
  if (offset == 0)
    return frame;

  EsperVal disp = esper_build_ldi(b, ESP_TYPE_I64, offset);
  return esper_build_binary(b, ESP_OP_ADD, ESP_TYPE_I64, frame, disp);
}

/// Allocates the frame and writes the two fields the first resume reads.
static void build_init(EsperUnit *unit) {
  EsperTypeRef params[1] = {ESP_TYPE_I32};
  EsperTypeRef rets[1] = {ESP_TYPE_PTR};

  EsperBuilder *b = esper_build_begin(unit, "count_up_init", params, 1, rets, 1);
  esper_build_bind(b, esper_build_block(b, "entry"));

  // The frame outlives this call, so it cannot be frame storage.
  EsperVal size = esper_build_ldi(b, ESP_TYPE_I32, FRAME_SIZE);
  EsperVal frame = esper_build_alloc(b, ESP_OP_HALLOC, size);

  EsperVal zero = esper_build_ldi(b, ESP_TYPE_I32, 0);
  esper_build_store(b, ESP_OP_STORE, ESP_TYPE_I32,
                    frame_slot(b, frame, STATE_OFF), zero);
  esper_build_store(b, ESP_OP_STORE, ESP_TYPE_I32,
                    frame_slot(b, frame, I_OFF), esper_build_param(b, 0));

  esper_build_ret(b, &frame, 1);

  if (!esper_build_end(b))
    fprintf(stderr, "count_up_init: %s\n", esper_builder_last_error(b));

  esper_builder_free(b);
}

/// Reads the output slot, which is where the last resume left its value.
static void build_value(EsperUnit *unit) {
  EsperTypeRef params[1] = {ESP_TYPE_PTR};
  EsperTypeRef rets[1] = {ESP_TYPE_I32};

  EsperBuilder *b = esper_build_begin(unit, "count_up_value", params, 1, rets, 1);
  esper_build_bind(b, esper_build_block(b, "entry"));

  EsperVal slot = frame_slot(b, esper_build_param(b, 0), YIELD_OFF);
  EsperVal value = esper_build_load(b, ESP_OP_LOAD, ESP_TYPE_I32, slot);
  esper_build_ret(b, &value, 1);

  if (!esper_build_end(b))
    fprintf(stderr, "count_up_value: %s\n", esper_builder_last_error(b));

  esper_builder_free(b);
}

static void esper_doc_build(EsperUnit *unit) {
  build_init(unit);
  build_value(unit);
}

func $count_up_init(%0 i32) -> ptr {
entry:
  %1 = ldi i32 12
  %2 = halloc %1
  %3 = ldi i32 0
  store i32 %2, %3
  %5 = ldi i64 4
  %6 = add i64 %2, %5
  store i32 %6, %0
  ret ptr %2
}

func $count_up_value(%0 ptr) -> i32 {
entry:
  %1 = ldi i64 8
  %2 = add i64 %0, %1
  %3 = load i32 %2
  ret i32 %3
}

Note the gaps in the numbering. store returns nothing but still occupies an SSA id, so %4 and %7 name no value. The printer skips them rather than renumbering, because an id is an instruction’s position and not a name the frontend chose.

Step 5: Dispatch on the state

Each state is a block, and the state index selects between them. Writing that as a chain of comparisons costs a compare and a branch per state, and the chain gets longer as the coroutine grows. jmp.table is the one instruction for it: the index is an offset into a list of targets, and the default arm catches anything past the end.

  jmp.table %1 [state_0, state_1] finished

The table names blocks, so every block in it has to exist before the dispatch is built. That is why esper_build_block is separate from esper_build_bind: a block id is handed out as soon as the block is created, and the block is filled later. All four blocks below are created up front and bound one at a time.

The resume function returns 1 if a value was yielded, 0 if the generator is finished.

/// Offsets into the coroutine frame, in the order the fields were laid out.
enum { STATE_OFF = 0, I_OFF = 4, YIELD_OFF = 8 };

/// Address of a frame field, as `frame + offset`.
static EsperVal frame_slot(EsperBuilder *b, EsperVal frame, int32_t offset) {
  if (offset == 0)
    return frame;

  EsperVal disp = esper_build_ldi(b, ESP_TYPE_I64, offset);
  return esper_build_binary(b, ESP_OP_ADD, ESP_TYPE_I64, frame, disp);
}

static void esper_doc_build(EsperUnit *unit) {
  EsperTypeRef params[1] = {ESP_TYPE_PTR};
  EsperTypeRef rets[1] = {ESP_TYPE_I32};

  EsperBuilder *b = esper_build_begin(unit, "count_up_resume", params, 1, rets, 1);
  EsperVal frame = esper_build_param(b, 0);

  // Every resumption point is a block, and the table names them all, so
  // they have to exist before the dispatch is emitted.
  EsperBlockId entry = esper_build_block(b, "entry");
  EsperBlockId state_0 = esper_build_block(b, "state_0");
  EsperBlockId state_1 = esper_build_block(b, "state_1");
  EsperBlockId finished = esper_build_block(b, "finished");

  esper_build_bind(b, entry);
  EsperVal state = esper_build_load(b, ESP_OP_LOAD, ESP_TYPE_I32, frame);
  EsperBlockId targets[2] = {state_0, state_1};
  esper_build_jmp_table(b, state, targets, 2, finished);

  // State 0: first entry. Yield i unchanged and arm state 1.
  esper_build_bind(b, state_0);
  EsperVal i0 = esper_build_load(b, ESP_OP_LOAD, ESP_TYPE_I32,
                                 frame_slot(b, frame, I_OFF));
  esper_build_store(b, ESP_OP_STORE, ESP_TYPE_I32,
                    frame_slot(b, frame, YIELD_OFF), i0);

  EsperVal next_state = esper_build_ldi(b, ESP_TYPE_I32, 1);
  esper_build_store(b, ESP_OP_STORE, ESP_TYPE_I32,
                    frame_slot(b, frame, STATE_OFF), next_state);

  EsperVal yielded = esper_build_ldi(b, ESP_TYPE_I32, 1);
  esper_build_ret(b, &yielded, 1);

  // State 1: resumed after the yield. Increment, store back, yield again.
  esper_build_bind(b, state_1);
  EsperVal i_slot = frame_slot(b, frame, I_OFF);
  EsperVal i1 = esper_build_load(b, ESP_OP_LOAD, ESP_TYPE_I32, i_slot);
  EsperVal step = esper_build_ldi(b, ESP_TYPE_I32, 1);
  EsperVal next = esper_build_binary(b, ESP_OP_ADD, ESP_TYPE_I32, i1, step);
  esper_build_store(b, ESP_OP_STORE, ESP_TYPE_I32, i_slot, next);
  esper_build_store(b, ESP_OP_STORE, ESP_TYPE_I32,
                    frame_slot(b, frame, YIELD_OFF), next);

  // The state index stays at 1, because the loop has only one body.
  EsperVal again = esper_build_ldi(b, ESP_TYPE_I32, 1);
  esper_build_ret(b, &again, 1);

  // The generator never finishes, so the default arm is unreachable.
  esper_build_bind(b, finished);
  EsperVal no = esper_build_ldi(b, ESP_TYPE_I32, 0);
  esper_build_ret(b, &no, 1);

  if (!esper_build_end(b))
    fprintf(stderr, "count_up_resume: %s\n", esper_builder_last_error(b));

  esper_builder_free(b);
}

func $count_up_resume(%0 ptr) -> i32 {
entry:
  %1 = load i32 %0
  jmp.table %1 [state_0, state_1] finished

state_0:
  %3 = ldi i64 4
  %4 = add i64 %0, %3
  %5 = load i32 %4
  %6 = ldi i64 8
  %7 = add i64 %0, %6
  store i32 %7, %5
  %9 = ldi i32 1
  store i32 %0, %9
  %11 = ldi i32 1
  ret i32 %11

state_1:
  %13 = ldi i64 4
  %14 = add i64 %0, %13
  %15 = load i32 %14
  %16 = ldi i32 1
  %17 = add i32 %15, %16
  store i32 %14, %17
  %19 = ldi i64 8
  %20 = add i64 %0, %19
  store i32 %20, %17
  %22 = ldi i32 1
  ret i32 %22

finished:
  %24 = ldi i32 0
  ret i32 %24
}

The loop in the source has disappeared. Nothing branches back to state_1 from inside state_1, because the caller is the loop now: it re-enters through a fresh call and the table sends it straight back to the same block. That is the shape every state machine lowering ends up with, and it is why the frame has to hold i.

A branch table carries no arguments, so none of its targets may declare block parameters. State that has to reach a resumption point reaches it through the frame, which is where it already lives.

Using the generator

let gen = countUp(1)
var sum = 0
// Take 3 values and sum them: 1 + 2 + 3 = 6
for _ in 0..<3 {
    gen.next()
    sum += gen.value
}

The caller is an ordinary function, so its locals are ordinary SSA values: neither sum nor the loop counter crosses a suspension, and both ride the loop’s back edge as branch arguments rather than through memory. This is the contrast the frame exists for. The coroutine pays for memory because its live ranges are cut by a ret; the caller’s are not, so it pays nothing.

esper_build_call looks its callee up by name and the name has to resolve, so the three generator functions are declared before main is built. esper_func_declare records a signature without a body, which is what a call site is checked against. A declaration that never receives a body prints as a function with no blocks, which is what the three below are here.

static void esper_doc_build(EsperUnit *unit) {
  EsperTypeRef ptr_ret[1] = {ESP_TYPE_PTR};
  EsperTypeRef i32_ret[1] = {ESP_TYPE_I32};
  EsperTypeRef i32_param[1] = {ESP_TYPE_I32};
  EsperTypeRef ptr_param[1] = {ESP_TYPE_PTR};

  // The generator's three entry points, declared so main can name them.
  esper_func_declare(unit, "count_up_init", i32_param, 1, ptr_ret, 1, false);
  esper_func_declare(unit, "count_up_resume", ptr_param, 1, i32_ret, 1, false);
  esper_func_declare(unit, "count_up_value", ptr_param, 1, i32_ret, 1, false);

  EsperBuilder *b = esper_build_begin(unit, "main", NULL, 0, i32_ret, 1);
  EsperBlockId entry = esper_build_block(b, "entry");
  EsperBlockId header = esper_build_block(b, "header");
  EsperBlockId body = esper_build_block(b, "body");
  EsperBlockId done = esper_build_block(b, "done");

  esper_build_bind(b, entry);
  EsperVal start = esper_build_ldi(b, ESP_TYPE_I32, 1);
  EsperVal gen = esper_build_call(b, ESP_TYPE_PTR, "count_up_init", &start, 1);
  EsperVal zero = esper_build_ldi(b, ESP_TYPE_I32, 0);
  EsperVal seed[2] = {zero, zero};
  esper_build_jmp_args(b, header, seed, 2);

  // Neither the running total nor the loop counter crosses a suspension,
  // so both stay SSA values rather than joining the frame.
  esper_build_bind(b, header);
  EsperVal sum = esper_build_block_param(b, ESP_TYPE_I32);
  EsperVal taken = esper_build_block_param(b, ESP_TYPE_I32);
  EsperVal limit = esper_build_ldi(b, ESP_TYPE_I32, 3);
  EsperVal more = esper_build_binary(b, ESP_OP_LT_S, ESP_TYPE_I32, taken, limit);
  esper_build_jmp_if(b, more, body, done);

  esper_build_bind(b, body);
  esper_build_call(b, ESP_TYPE_I32, "count_up_resume", &gen, 1);
  esper_build_discard_result(b);

  EsperVal value = esper_build_call(b, ESP_TYPE_I32, "count_up_value", &gen, 1);
  EsperVal next_sum = esper_build_binary(b, ESP_OP_ADD, ESP_TYPE_I32, sum, value);
  EsperVal one = esper_build_ldi(b, ESP_TYPE_I32, 1);
  EsperVal next_taken = esper_build_binary(b, ESP_OP_ADD, ESP_TYPE_I32, taken, one);
  EsperVal carry[2] = {next_sum, next_taken};
  esper_build_jmp_args(b, header, carry, 2);

  esper_build_bind(b, done);
  esper_build_ret(b, &sum, 1);

  if (!esper_build_end(b))
    fprintf(stderr, "main: %s\n", esper_builder_last_error(b));

  esper_builder_free(b);
}

func $count_up_init(%0 i32) -> ptr {
}

func $count_up_resume(%0 ptr) -> i32 {
}

func $count_up_value(%0 ptr) -> i32 {
}

func $main() -> i32 {
entry:
  %0 = ldi i32 1
  %1 = call ptr $count_up_init, %0
  %2 = ldi i32 0
  jmp header(%2, %2)

header(%4 i32, %5 i32):
  %6 = ldi i32 3
  %7 = lt.s i32 %5, %6
  jmp.if %7 body done

body:
  call i32 $count_up_resume, %1
  %10 = call i32 $count_up_value, %1
  %11 = add i32 %4, %10
  %12 = ldi i32 1
  %13 = add i32 %5, %12
  jmp header(%11, %13)

done:
  ret i32 %4
}

The call to count_up_resume is followed by esper_build_discard_result, which records that nothing reads what the call returned. The call still runs; what changes is the type the value table holds for that id, so the printer omits the assignment and a reader can see at a glance that the result was thrown away.

Async/await

Async functions are the same lowering with a different caller. Instead of yielding a value, the resume function returns a status and the runtime scheduler decides when to call it again.

async func fetchAndParse(url: Int32) -> Int32 {
    let response = await fetch(url)    // await point 0
    let parsed = await parse(response) // await point 1
    return parsed
}

Both url and response cross an await, so both go in the frame; parsed is written into the result slot for whoever collects the result. The frame is state, url, response, parsed, and each await splits the body at exactly the point where the two halves need a table arm between them.

The scheduler and the I/O it drives are the host’s. A frontend declares them extern and Esper never sees their bodies.

/// Offsets into the async frame: the state index, then one slot per
/// local that has to survive an await.
enum { A_STATE = 0, A_URL = 4, A_RESPONSE = 8, A_PARSED = 12 };

/// Address of a frame field, as `frame + offset`.
static EsperVal frame_slot(EsperBuilder *b, EsperVal frame, int32_t offset) {
  if (offset == 0)
    return frame;

  EsperVal disp = esper_build_ldi(b, ESP_TYPE_I64, offset);
  return esper_build_binary(b, ESP_OP_ADD, ESP_TYPE_I64, frame, disp);
}

/// Records the next state index and hands control back to the scheduler.
static void suspend(EsperBuilder *b, EsperVal frame, int32_t next) {
  EsperVal state = esper_build_ldi(b, ESP_TYPE_I32, next);
  esper_build_store(b, ESP_OP_STORE, ESP_TYPE_I32, frame, state);

  EsperVal pending = esper_build_ldi(b, ESP_TYPE_I32, 0);
  esper_build_ret(b, &pending, 1);
}

static void esper_doc_build(EsperUnit *unit) {
  EsperTypeRef i32_1[1] = {ESP_TYPE_I32};
  EsperTypeRef ptr_1[1] = {ESP_TYPE_PTR};

  // The scheduler and the I/O it drives are the host's, not the IR's.
  esper_func_declare(unit, "fetch_begin", i32_1, 1, NULL, 0, true);
  esper_func_declare(unit, "fetch_take", NULL, 0, i32_1, 1, true);
  esper_func_declare(unit, "parse_begin", i32_1, 1, NULL, 0, true);
  esper_func_declare(unit, "parse_take", NULL, 0, i32_1, 1, true);

  EsperBuilder *b = esper_build_begin(unit, "fetch_and_parse_resume",
                                      ptr_1, 1, i32_1, 1);
  EsperVal frame = esper_build_param(b, 0);

  EsperBlockId entry = esper_build_block(b, "entry");
  EsperBlockId await_0 = esper_build_block(b, "await_0");
  EsperBlockId await_1 = esper_build_block(b, "await_1");
  EsperBlockId await_2 = esper_build_block(b, "await_2");
  EsperBlockId settled = esper_build_block(b, "settled");

  // One table arm per await point. Adding a fourth await adds a block
  // and an entry, and costs the dispatch nothing.
  esper_build_bind(b, entry);
  EsperVal state = esper_build_load(b, ESP_OP_LOAD, ESP_TYPE_I32, frame);
  EsperBlockId targets[3] = {await_0, await_1, await_2};
  esper_build_jmp_table(b, state, targets, 3, settled);

  // Await 0: start the fetch and suspend.
  esper_build_bind(b, await_0);
  EsperVal url = esper_build_load(b, ESP_OP_LOAD, ESP_TYPE_I32,
                                  frame_slot(b, frame, A_URL));
  esper_build_call(b, ESP_TYPE_VOID, "fetch_begin", &url, 1);
  suspend(b, frame, 1);

  // Await 1: the fetch is ready. Save the response, start the parse.
  esper_build_bind(b, await_1);
  EsperVal response = esper_build_call(b, ESP_TYPE_I32, "fetch_take", NULL, 0);
  esper_build_store(b, ESP_OP_STORE, ESP_TYPE_I32,
                    frame_slot(b, frame, A_RESPONSE), response);
  esper_build_call(b, ESP_TYPE_VOID, "parse_begin", &response, 1);
  suspend(b, frame, 2);

  // Await 2: the parse is ready. Publish the result and settle.
  esper_build_bind(b, await_2);
  EsperVal parsed = esper_build_call(b, ESP_TYPE_I32, "parse_take", NULL, 0);
  esper_build_store(b, ESP_OP_STORE, ESP_TYPE_I32,
                    frame_slot(b, frame, A_PARSED), parsed);

  // Past the end of the table, so a later resume lands on the default.
  EsperVal end_state = esper_build_ldi(b, ESP_TYPE_I32, 3);
  esper_build_store(b, ESP_OP_STORE, ESP_TYPE_I32, frame, end_state);
  esper_build_jmp(b, settled);

  esper_build_bind(b, settled);
  EsperVal done = esper_build_ldi(b, ESP_TYPE_I32, 1);
  esper_build_ret(b, &done, 1);

  if (!esper_build_end(b))
    fprintf(stderr, "fetch_and_parse_resume: %s\n", esper_builder_last_error(b));

  esper_builder_free(b);
}

func extern $fetch_begin(i32)
func extern $fetch_take() -> i32
func extern $parse_begin(i32)
func extern $parse_take() -> i32
func $fetch_and_parse_resume(%0 ptr) -> i32 {
entry:
  %1 = load i32 %0
  jmp.table %1 [await_0, await_1, await_2] settled

await_0:
  %3 = ldi i64 4
  %4 = add i64 %0, %3
  %5 = load i32 %4
  call $fetch_begin, %5
  %7 = ldi i32 1
  store i32 %0, %7
  %9 = ldi i32 0
  ret i32 %9

await_1:
  %11 = call i32 $fetch_take
  %12 = ldi i64 8
  %13 = add i64 %0, %12
  store i32 %13, %11
  call $parse_begin, %11
  %16 = ldi i32 2
  store i32 %0, %16
  %18 = ldi i32 0
  ret i32 %18

await_2:
  %20 = call i32 $parse_take
  %21 = ldi i64 12
  %22 = add i64 %0, %21
  store i32 %22, %20
  %24 = ldi i32 3
  store i32 %0, %24
  jmp settled

settled:
  %27 = ldi i32 1
  ret i32 %27
}

await_2 sets the state to 3, which is past the end of a three-entry table, so a resume after the result has been collected lands on the default arm and reports done rather than running anything again. A state index no arm covers is the natural place to put “finished”, and it costs no extra comparison.

The three-state version is where the table earns its place. A chain of comparisons grows a compare and a branch for every await the source adds; the table grows one entry, and the dispatch stays a single instruction whether the function has two suspension points or forty.

Nothing here needs a coroutine-shaped instruction. A resume function is an ordinary function, a frame is ordinary memory, and the scheduler that drives them is host code the frontend declares extern. The one piece of IR that exists specifically for this shape is jmp.table, and it exists because dispatching on a small dense integer is what every state machine does.

Key takeaways

Coroutines are state machines at the IR level: a frame plus a resume function
A local goes in the frame exactly when its live range crosses a suspension, because a suspension is a ret and no SSA value survives one
The frame is halloc, not salloc: it outlives the call that created it
jmp.table dispatches on the state index in one instruction, and its targets must exist before it is built
A branch table carries no arguments, so its targets may not declare block parameters
The frontend performs the transformation; Esper sees only regular functions and memory operations