# Live state: how a committed change reaches the screen

Every claim here describes code that exists in this repository, and each section names the file to
read. Measured numbers are marked; anything not measured is marked **not measured**.

## The invariant

A committed change and its journal row are written in the same SurrealDB transaction. That is the
whole reason the live system can be simple: if a write exists, its event exists; if the transaction
rolls back, no sequence number is burned and no phantom notification appears.

`apps/api/src/journal.ts` allocates the per-workspace `seq` inside the transaction that mutates the
row, and `apps/api/src/mutations.ts` is the only module that issues `CREATE`/`UPDATE`/`DELETE`.
Nothing else writes operational data, which is what makes the journal complete and the optimistic
version checks meaningful.

## Write path

```
browser action form
  -> POST /api/actions/:type/preview   (validation + change set, writes nothing)
  -> POST /api/actions/:type/commit    (idempotency key + expected versions)
       BEGIN
         version check  -> reject stale, never overwrite
         mutate entity / relation rows (incremental version)
         INSERT journal row (seq = workspace.lastSeq + 1)
         UPSERT idempotency_key -> actionRunUid + result
       COMMIT
  -> SSE invalidation carrying the durable seq
```

A replayed commit with the same idempotency key returns the stored result instead of writing again
(`apps/api/src/actions.ts`, `commitAction`). A rejected action writes nothing, so it cannot show a
successful effect.

## Read path: one snapshot contract

`GET /api/graph?lens=…&maxNodes=…` returns the nodes, edges and metric inputs for one scope.
The 3D scene, the 2D view, the table and the metric tiles all read that same payload
(`apps/api/src/graph.ts`), so a number cannot disagree with a picture. Ordering is deterministic —
nodes by `(objectType, uid)` before selection, then by degree — so the busiest context is what
survives truncation rather than an accident of storage order.

## What the stream carries

The SSE payload is deliberately an **invalidation**, not a patch (`apps/api/src/sse.ts`). On an
invalidation the browser re-reads `/api/graph` with its own scope. That makes duplicated, reordered
or dropped notifications harmless: convergence never depends on the transport having delivered
exactly one message per change. A heartbeat is sent every 10 s and the timer is `unref()`'d so it
cannot hold the process open during shutdown.

## Three cooperating mechanisms (browser)

`apps/web/src/live/useLiveGraph.ts`:

1. **SSE with a durable cursor.** Events carry journal `seq` numbers; a reconnect resumes from the
   last one applied, so a gap is requested explicitly rather than assumed away.
2. **A slow poll** of `/api/events?after=cursor` every 12 s catches anything the socket missed
   (backgrounded tab, proxy hiccup) without waiting for the next event. A stream that goes quiet for
   35 s is treated as stalled and reconnected; refreshes are debounced by 450 ms so a burst of
   invalidations reads the database once.
3. **A `baseline` invalidation** discards cached node positions and re-reads the whole scope. This
   is what a direct database edit or a restore asks for, because there is no per-row delta to apply.

The scope always comes from the session cookie — a browser cannot widen its own workspace by editing
a query string (`apps/api/src/routes-live.ts`).

## Out-of-band detection, and the honest caveat

SurrealDB `CHANGEFEED` is **not observable on this installation**. Measured on SurrealDB 3.2.3 with
SurrealKV over HTTP: `SHOW CHANGES FOR TABLE entity SINCE 0` returns an empty list even immediately
after a write, and `INFO FOR TABLE entity` reports no changefeed. `changesSince()` in
`apps/api/src/db.ts` is kept for a later engine, and its empty result must never be read as
"nothing changed".

So the watcher in `apps/api/src/runtime.ts` reconciles identity+version sets instead
(`stateReconciliation()` in `apps/api/src/db.ts`), on three loops:

1. **Journal catch-up** (default `ATLAS_JOURNAL_POLL_MS=1000`): publishes what the API wrote.
2. **Out-of-band reconciliation** (default `ATLAS_WATCH_INTERVAL_MS=3000`): diffs stored uid+version
   sets against the versions journaled writes recorded. A difference that no journal event names is
   an out-of-band edit; the watcher journals `outofband.detected`, captures a fresh baseline, and
   marks history coverage **partial**. A difference must survive one extra poll before it is
   reported, so a write landing between the two reads cannot be misreported.
3. **Scenario clock** (default base tick 6 s): advances scenario time and applies what stored
   records say is due, through the same journaled write path as a human action.

## History and replay

History is **a stored baseline plus the journal after it** (`apps/api/src/history.ts`). A baseline is
an explicit complete copy of one workspace's graph and metric inputs; the journal says what changed
afterwards. Anything that happened outside the API has no before/after pair, so it is reported as a
coverage gap ("Partial: an out-of-band edit created a gap") rather than quietly reconstructed.

## Recovery guarantees, stated plainly

| Situation | Guarantee |
|---|---|
| SSE message lost or duplicated | Converges anyway — the browser re-reads by cursor and by scope |
| Browser tab backgrounded | The 12 s poll closes the gap without a new event |
| Row edited directly in SurrealDB | Detected within roughly two watch passes (~6 s default), journalled as `outofband.detected`, history marked partial |
| Transaction rolled back | No journal row, no notification, no burned sequence number |
| Same action replayed | Stored result returned; nothing duplicated |
| Two conflicting previews of one constrained resource | The second commit is rejected on version, not silently overwritten |
| Store restored from backup | Baseline invalidation forces a full re-read of every open scope |

**Not measured:** behaviour under a SurrealDB failover or a network partition between API and store
(single local node here), multi-tab write contention, and behaviour with more than a few hundred
concurrent SSE subscribers.

## What production hardening would add

Durable per-subscriber cursors instead of in-memory state, authentication behind a real identity
provider, a changefeed-capable engine so out-of-band detection stops being a poll-and-diff, and
backpressure on the invalidation fan-out. None of those are required for the POC, and none are
implemented here.