# VALIDATION — what was run, what it measured, what is still unproven

Every number below came from a command on this Mac during 2026-10-17 (Asia/Singapore).
Nothing here is extrapolated; anything not measured says "not measured".

This file records the **build** validation for M1–M6. The **system audit** that ran afterwards — a
separate audit → validation → fix → optimize → re-verify pass with its own logs — is in
`docs/audit/2026-10-17T152518+08/`. Where a figure here is superseded by a re-measurement, the audit
folder holds both the before and after log; start with
`docs/audit/2026-10-17T152518+08/README.md`.

## Environment (measured, not assumed)

| Item | Value |
|---|---|
| Machine | MacBook Pro Mac17,6 — Apple M5 Max, 18 cores (6P + 12E), 128 GB RAM |
| OS | macOS 27.0.1 (build 26A434) |
| Node | v24.21.0 (`~/.local/node-v24.21.0-darwin-arm64/bin/node`, runs `.ts` directly) |
| pnpm | 11.25.0 |
| TypeScript | 5.9.3 |
| SurrealDB | 3.2.3+20260721.40522d1 (aarch64), `/opt/homebrew/bin/surreal`, persistent `surrealkv://` |
| `@surrealdb/node` | 3.1.0 |
| API | Fastify 5.12.7, loopback `http://127.0.0.1:8787` |
| UI | React 19.3.1, Vite 8.3.4 (rolldown), Tailwind 4.3.3, dev server `http://127.0.0.1:5173` |
| Renderer | three 0.186.1, @react-three/fiber 9.8.2, drei 10.7.9, d3-force-3d 3.0.6 |
| Browser test driver | puppeteer-core 25.14.0 driving Google Chrome 155.0.8059.82 |
| Local inference | oMLX at `http://127.0.0.1:8000/v1`, model `Qwen3.8-Flash-Next-Uncensored-oQ5e-mtp` (user-owned, never started or reconfigured by this build) |

Data scope for every performance number: the Commercial pack at `maxNodes=500`. On a freshly
seeded store that query draws **500 nodes / 1,462 edges of 542 matching** (defense: 500 / 1,396 of
559; full scope at `maxNodes=2000&maxEdges=6000` is defense 559/1,500 and commercial 542/1,500,
both `truncated: false`). Journeys write rows, so the matching count grows after every suite — the
numbers below were measured on the 12:30 run, at **500 nodes / 1,465 edges of 568 matching**
(`truncated: true`).

## Commands that were run, and how long they took

| Command | Result | Wall time |
|---|---|---|
| `pnpm -r typecheck` | all four packages clean (`contracts`, `ontology`, `api`, `web`) | 2.6 s |
| `pnpm -s build` | `dist/assets/index-*.js` 1,318.47 kB (gzip 359.63 kB), CSS 34.63 kB (gzip 7.21 kB); Vite chunk-size advisory only | 208 ms |
| `node tools/run-tool.mjs test` | **22 pass / 0 fail** (14 integration + 8 unit) | 38.1 s (12:30, immediately before the e2e run below) |
| `node tools/run-tool.mjs test --file tests/integration/live-state.test.mjs` | 14/14, run twice back-to-back as a stability re-run | ~50 s each |
| `pnpm -s test:e2e` | **82 checks / 0 failures**, 9 journeys PASS, exit 0 | 2 m 27 s (12:30:56 → 12:33:23) |
| `node tools/run-tool.mjs seed --clean` | pruned 1 synthetic stray, preserved all 8 non-seed rows (see the seed section below) | 2.5 s |
| `node tools/run-tool.mjs counts` | 1,128 entities / 3,009 relations across both workspaces (matches the check-12 transcript below) | 0.2 s |
| `node tools/run-tool.mjs backup` | 4,434 rows + 141 evidence files → 8.0 MB (8,372,699 bytes) | 787 ms |
| `node tools/run-tool.mjs restore --file <path> --confirm` | verified, `failures: []` | 23.7 s |
| `node tests/e2e/race-scope-proof.mjs` | deterministic cross-workspace scope-race proof, see below | ~8 s |

## Section 20 checks 1–12 (integration suite)

`node tools/run-tool.mjs test` runs `tests/**/*.test.mjs`. Each check maps 1:1 to a numbered
acceptance test in the master prompt.

| §20 | Test name as reported | Result |
|---|---|---|
| 1 | restart persistence: committed entities and relations are readable through an independent path into the store | pass (341 ms) |
| 2 | native graph check: relation records carry real record links and answer a traversal | pass (68 ms) |
| 3 | API create/update/delete converges scene, table, inspector and metrics without a reload | pass (591 ms) |
| 4 | direct database edit is detected out of band and the history gap is stated, not hidden | pass (6.35 s) |
| 5 | filter exit: an object that stops matching leaves the filtered graph and the count | pass (303 ms) |
| 6 | no dangling edges: removing a node removes its relationships from the rendered snapshot | pass (439 ms) |
| 7 | stream recovery: dropped connection, mutation during the gap, reconnect with the old cursor converges | pass (1.08 s) |
| 8 | snapshot race: a mutation during the initial graph load is not lost | pass (254 ms) |
| 9 | version conflict: two previews of the same constrained stock cannot both reserve it | pass (798 ms) |
| 10 | idempotency and rollback: a replayed commit does not duplicate, a rejected action writes nothing | pass (576 ms) |
| 10b | ingestion is idempotent and rejected rows are reported, not silently dropped | pass (1.36 s) |
| 11 | scope protection: cross-workspace reads, evidence, actions and SSE are refused | pass (213 ms) |
| 12 | backup and restore verify node counts, edge counts, ontology version and evidence hashes | pass (24.0 s) |
| 12b | a clean re-seed prunes synthetic drift and keeps imported rows | pass (1.58 s) |

Story arithmetic is covered by eight further tests in the same run: defense baseline 4 ready /
48 places against demand 42 (margin +6), a maintenance failure turning +6 into a shortfall of 6,
reserve-activation preconditions reading stored records, commercial demand 300 against 80 on hand
with the delay exposing 220, transfer alone leaving 80 short then expedite closing at SGD 1,380.00,
money staying in integer minor units and formatting the Singapore way, generator determinism with a
consistent reservation ledger, and data-health findings derived from rows. All pass.

## Seed reconciliation and `seed --clean` (found by re-seeding, not by review)

The first seed verification compared **exact counts** (`stored == expected`), so any record a later
journey legitimately added made the seed look broken. It was rewritten as set reconciliation: every
seeded uid must be present (`missing`), and anything else in the workspace is reported as `drift`
with its uid, count and a `synthetic` flag (`apps/api/src/seed.ts`, `readback()` / `readbackAll()`).
A seed is additive and now says so instead of failing.

Re-seeding the live store then exposed a real defect: `seed --clean` deleted **every** extra
relationship and evidence row, which silently destroyed the lineage of rows an import had asserted.
`pruneDrift()` now deletes only generator strays — entities with `synthetic = true` that are outside
the seed set — plus the edges whose endpoint is one of those strays, and journals `entity.deleted` /
`relation.deleted` for each so a live tab reconciles. Imported objects survive because accepted
ingestion lines are written with `synthetic: false` (`apps/api/src/ingest.ts:329`), which is also
what the Inspector's provenance label reads.

Measured CLI proof (a stray created through the API, then one clean re-seed):

```
node tools/run-tool.mjs seed --clean
# commercial  pruned {"entities":1,"relations":0}  preserved 8
#   drift kept: inventorylot-bx40012/13/15 (imports) · evd-inventorylot-bx4001{2,3,5}-* (their
#   evidence) · e2e_inspection_mvbvrj9f (Ontology Studio type) ·
#   affects-com-risk-013-com-wh-tanjung (a committed AFFECTS edge)
# defense     pruned {"entities":0,"relations":0}  preserved 1 (link-reserve-def-vessel-07-9g6us)
# both packs verified: missing 0 on every table, validationIssues []
```

Integration check **12b** asserts the same contract on live data: the synthetic stray leaves the
store and the rendered graph, the imported row survives, `stored == seeded + extra` on both
`entity` and `entity_link`, every uid named in `drift.*.extraUids` is still readable, and no
surviving extra carries `synthetic = true`.

## Section 20 journeys 13–20 and the performance sample

`pnpm test:e2e` resets and reseeds the store itself, then drives real Chrome. Machine-readable
report: `docs/validation/e2e-results.json`.

| Journey | Checks | Result | Measured outcome worth quoting |
|---|---|---|---|
| 13 Defense readiness disruption | 9 | PASS | launcher card reports 559 objects; baseline 4 of 7 ready, 48 places against demand 42 (margin +6) → step 1 gives 3 ready, 36 places and shortfall 6, and CORMORANT stops crediting the broken vessel (1/2 ready) → reserve activation returns 4 ready and margin +6, with the reserve linked to a live mission in stored relations |
| 14 Commercial supplier disruption | 10 | PASS | demand 300, on hand 80 (96 − 16 reserved), timely 200 → delay exposes **220** with the late arrival named as excluded → transfer cuts to **80** while on hand stays 80 → expedite closes to **0**, committed cost **SGD 1,380.00**; tile reached "Shortfall 0 units" in the open tab with no reload |
| 15 CSV ingestion, rejects, lineage | 12 | PASS | fixture split 3 accepted / 3 rejected with a printed reason per row; store gained exactly the accepted rows; lineage resolves source → run → observation → record; re-import reports the committed rows as already present and lot count does not move |
| 16 Ontology studio + link rules | 9 | PASS | new object type written and published in the ontology snapshot; instance validates against it and is searchable; valid link stored as a native edge record; type-mismatched link refused with `endpoint_type_mismatch` reasons |
| 17 Grounded copilot answer | 6 | PASS | provider probe **0.71 s** this run (band 0.7–5.8 s across runs, oMLX load dependent); answer cited **26 stored uids**; a citation is a clickable reference into the graph (`Commercial · Regional distribution control tower`); `Show supporting graph` re-scoped Atlas to 548 of 548 objects · 1,265/1,265 relationships drawn |
| 18 What-if branch + history replay | 14 | PASS | patch stored on the branch (=250); a patch type that cannot change the target is refused with a reason and leaves exactly one patch; live stock unchanged by the branch (`on hand 80 → 80`), patched object keeps its live version, journal cursor does not advance; baseline captured; replay shows `as of journal #N`; exit returns to live |
| 19 Offline with internet blocked | 7 | PASS | cold boot renders; local service panel reports real state (SurrealDB UP 12 ms, oMLX UP in 709 ms with models listed); metrics render offline; copilot answered offline in **15.06 s** (two runs measured 15.06 s and 15.2 s) with `Show supporting graph (26)`; **zero non-local requests attempted**; the 4 shipped files scan clean of external hosts |
| 20 Keyboard-only + reduced motion | 11 | PASS | search reached by Tab alone after 17 stops; next Tab stop is the first suggestion ("MV Sentosa Guard"); Enter opens that object's inspector; a status filter chip toggles from the keyboard; an action preview is reachable from the keyboard (armed "Update object status", 4 eligible, Enter on Preview opened the preview panel and its precondition list); reduced motion reaches `documentElement.dataset.reducedMotion=true`; narrow 900×800 renders the workspace and Atlas scope bar |
| P Performance sample | 4 | PASS | see the table below |

## Section 18 performance sample (measured on this Mac)

| Area | Target | Measured | Method / limit |
|---|---|---|---|
| Default visible scope | ~300–500 nodes, ≤1,500 edges | 500 nodes / 1,465 edges drawn of 568 matching (`truncated: true`); on a clean seed, 1,462 of 542 | `GET /api/graph?lens=instance&maxNodes=500` |
| Typical bounded query | < 500 ms after warm-up | **p50 88 ms · p95 101 ms · max 103 ms** | `GET /api/graph?lens=instance&maxNodes=500`, 3 warm-up + 25 samples from Node. Five runs today measured p50 85–109 / p95 87–122 / max 88–143 ms — treat ~90–150 ms as the observed band |
| Live update latency | p95 < 1 s commit→visible at ~10 updates/s | **p50 1 ms · p95 104 ms · max 104 ms, converged 20/20** | 20 committed `POST /api/entities` at ~10/s, UI live-cursor badge polled every 100 ms. The p50 of 1 ms is the poll quantum, not a claim that convergence is instant |
| Interaction | ≥ 45 FPS on the settled default scene | **p50 59.9 fps · p05 59.5** (idle) and **p50 59.9 · p05 59.5** with a copilot request in flight; app-reported `fps p50 60 · p05 52` in the perf sample, `p05 47` during the keyboard journey | rAF deltas during emulated camera drags (300 idle frames, 600 with inference in flight) across a 996×694 canvas, renderer `ANGLE (Apple, ANGLE Metal Renderer: Apple M5 Max)`, headless Chrome. **Not a GPU benchmark** — an emulated drag in a headless tab overstates what a trackpad feels like |
| Resource use | no unbounded queues/caches; report RSS | API **314 MB** RSS (pid 37544, after a full journey run); Chrome renderer processes **1,082 MB** combined | `ps` RSS; the browser total covers every Chrome renderer on the machine, so it is an upper bound |
| Idle behaviour | settled layout, no needless GPU work | `frameloop="demand"`; 601 frames during the drag window, none reported once settled | Settings panel and `docs/validation/e2e-results.json` → `perf.samples.idle` |
| Expanded scope cap (1,000 nodes / 3,000 edges) | configurable and measured | **not measured** in this run — the Atlas Max-nodes control is settable to 400 by default and the sample above used 500 | needs a dedicated run at the cap |
| Database stress fixture (10,000 / 30,000) | bounded queries | seeded via `pnpm seed:stress`, **latency not measured** against it in this session | the 500-node scope above is the default demo scope |

## Section 20 check 12 as an operator ran it (CLI proof)

```
node tools/run-tool.mjs backup
node tools/run-tool.mjs counts                       # 1,128 entities / 3,009 relations
# one record deleted straight out of the store with the surreal CLI → counts show 1,127
node tools/run-tool.mjs restore --file data/backups/atlas-2026-10-17T02-52-01-592Z.json --confirm
node tools/run-tool.mjs counts                       # back to 1,128 / 3,009
```

What restore reported back, and what was checked independently:

- `verification.countsMatch: true` for every table in the backup.
- Ontology version per workspace matches: `workspace-commercial 1.0.1`, `workspace-defense 1.0.0`.
- All **8 traversal probes re-walked with exact edge counts**: com-comp-bx4 70, com-wh-tuas 267,
  com-sup-borealis 177, com-order-7712 67, def-vessel-03 132, def-vessel-07 115,
  def-mission-marlin 160, def-mission-cormorant 191. `passed: true` on all eight.
- `evidenceChecked: 99`, `evidenceMissing: []` — every artefact re-hashed against the manifest.
- `failures: []`. The raw-deleted record came back with its original `version` intact.
- `journalDeltas` honestly reports what the restore itself journaled: `atlas_event 43 → 58`,
  `snapshot 4 → 17`, `snapshot_part 32 → 106`. These are new rows, not lost ones: restore journals
  a `baseline.captured` event per workspace so live clients stop trusting a dead cursor.

## Fixes made during M6 validation (each found by a failing check, not by taste)

1. **Cross-workspace scope race (real defect).** `chooseWorkspace()` set the API session *before*
   clearing client state, so an Inspector load still in flight continued under the new scope and
   asked `/api/lineage/<previous-workspace-uid>` — a 404 against an object no longer in scope.
   Journey 14 caught it once out of several runs. Fixed by clearing client state first in
   `apps/web/src/app/store.ts`, plus an attempt-counter/`stale()` guard after every await in
   `apps/web/src/panels/Inspector.tsx` and `apps/web/src/screens/Evidence.tsx`.
   Deterministic proof: `node tests/e2e/race-scope-proof.mjs` holds the entity response open for
   1.5 s, switches Defense → Commercial mid-load, and asserts afterwards — measured output:
   `stale lineage requests after switch: 0`, `extra entity requests for the old object after
   switch: 0`, `UI still names the abandoned uid: false`, `UI shows the Commercial object: true`,
   and a normal post-switch read pair (`entity` + `lineage`) still goes out. The two 404s in that
   probe's log are the deliberately delayed requests landing after the switch — harness artefact.
2. **A search term survived a workspace switch.** A Defense uid left in the box made Commercial
   report an empty scope whose message blamed the filters. `chooseWorkspace()` now clears `search`,
   and the Atlas empty state names the term it could not match (`apps/web/src/screens/Atlas.tsx`).
3. **"2 of 0 nodes shown".** The footer read the renderer's last report; when a scope empties, the
   scene unmounts and that number froze. It now reads from the snapshot (`apps/web/src/app/App.tsx`).
4. **Guided-demo copy pointed at controls that do not exist.** Steps said "Overview → Incidents",
   "Overview → Tower", blamed the delay on Alvina (shipment NB2291 is Borealis Components),
   reversed the transfer direction (it is Tanjung → Tuas), and named Marlin as the short mission
   (the seeded shortfall lands on CORMORANT). Corrected to the labels actually rendered.
5. **`.env.example` / `.env.local` claimed `ATLAS_SESSION_SECRET` was "generated by
   `pnpm preflight`".** Nothing generates it; with the value empty the API signs cookies with a
   per-process random key, so sessions die on every restart. The comment now says that.
6. **An in-flight Inspector chain outlived its workspace.** The first re-run after fix 1 failed
   journey 14 with `[http 404] /api/lineage/def-vessel-03 :: No def-vessel-03 in this workspace`.
   Cause, proven by joining request/response lines in `data/logs/api.log` on pid + request id: the
   session cookie moves to the new workspace the moment `POST /api/session/workspace` lands — one
   frame before React re-renders and bumps the Inspector's attempt counter — so an
   `entity -> neighborhood -> lineage` chain that was already in flight issued its **lineage leg**
   under the new scope. The `stale()` guard could not see it, because it compared render-time
   values. `apps/web/src/panels/Inspector.tsx` now derives `inScope()` from the live store
   (`getState().workspace?.uid === workspaceId && getState().selectedUid === uid`), folds it into
   `stale()`, and checks scope *before* each of the three requests; `apps/web/src/screens/Evidence.tsx`
   carries the same live-workspace guard. A complete re-run of all nine journeys afterwards came back
   82/82 with the browser problem list unchanged at six attributed lines.
7. **A clean re-seed destroyed imported lineage (real defect).** Seed verification had compared
   exact counts, so legitimate drift read as a failure. Once verification became set reconciliation,
   re-seeding the live store exposed the worse bug underneath: `seed --clean` deleted *every* extra
   relationship and evidence row, which silently wiped the lineage of rows an import had asserted.
   `pruneDrift()` now removes only synthetic strays plus the edges whose endpoint is one of them,
   and journals `entity.deleted` / `relation.deleted`; accepted ingestion rows are written with
   `synthetic: false` (`apps/api/src/ingest.ts:329`) so they survive. Measured numbers and the CLI
   proof are in "Seed reconciliation and `seed --clean`" above; integration check 12b locks it.

## Live-state design facts worth knowing before you extend it

- Journal rows are the truth; SSE is a convenience. `publishCommitted()` in
  `apps/api/src/journal.ts` reads the just-committed row back and pushes it inside the write's own
  round trip, so a watching tab converges without waiting for the poll — but only when
  `hasSubscribers()` is true. The `/api/events?after=cursor` poll stays as the net for writes this
  API process did not make, so a failed push costs latency and never correctness.
- Stream frames are **named** (`hello`, `change`, `ping`, `closed`, `scope_invalidated`) and the
  unnamed default channel is wired to the same handler, because a bare payload must keep working.
- The inline push and the poll can deliver the same `seq`; the client de-duplicates by journal
  sequence rather than assuming one source wins.
- Reconnect uses exponential backoff capped at 15 s and resumes from the last *applied* cursor.

## Browser problems captured during the journeys

`pnpm test:e2e` fails a journey on an unexplained console error, failed request, or 4xx/5xx.
The final run (12:30 → 12:33) captured **6 problems and attributed all 6** (`claimed: true` on every
entry in `docs/validation/e2e-results.json`):

- 4 × `GET /api/stream?lens=instance net::ERR_ABORTED` — the client closing a stream on purpose
  when the lens changes or a reconnect is scheduled. Matched by the harness's benign pattern.
- 2 × one event: `POST /api/relations` **400** `affects cannot be created` with
  `endpoint_type_mismatch: affects cannot start from Warehouse`, recorded once as an HTTP status and
  once as the matching console error. Journey 16 provokes this invalid link and asserts the refusal.

No unattributed errors, no warnings, and no request that failed without being asserted on.

## Screenshots (regenerated by the run above)

`docs/screenshots/`: `13-a-launcher-defense.png`, `13-b-vessel-inspector.png`,
`13-c-readiness-shortfall.png`, `13-d-readiness-restored.png`, `14-a-commercial-baseline.png`,
`14-b-commercial-zero-shortfall.png`, `14-c-inventory-current-vs-planned.png`,
`15-a-validation-rejects.png`, `15-b-lineage.png`, `16-a-ontology-studio.png`,
`16-b-invalid-link-rejected.png`, `17-a-copilot-answer.png`, `17-b-supporting-subgraph.png`,
`18-a-branch-compare.png`, `18-b-history-replay.png`, `19-a-offline.png`,
`20-a-keyboard-inspector.png`, `20-b-reduced-motion.png`, `20-c-narrow-900.png`,
`20-d-narrow-atlas.png`.

## Still unverified, and material limitations

- **Answer quality from oMLX is not judged.** Grounding is tested mechanically (every citation
  resolves to a stored record, an invalid JSON reply is reported as `invalid_output`, no prose is
  invented when the model is unreachable). Whether the sentences read well needs a human.
- **Answer latency is uncontrolled.** One offline answer took 54.3 s; an earlier one the same day
  took about 17 s. That is oMLX load on a shared Mac, not something ATLAS paces. No p95 is
  claimed for inference.
- **FPS numbers come from headless Chrome with an emulated drag.** Real trackpad interaction,
  label legibility at a glance, and colour/contrast need your own eyes.
- **Narrow viewport is machine-checked only** as far as "the text renders and the scope bar
  appears". Nothing asserts that a dense panel is *comfortable* at 900 px wide.
- **Expanded-scope cap (1,000 nodes / 3,000 edges) and the 10,000/30,000 stress fixture have no
  latency numbers attached in this session.** `truncated: true` at 500 nodes proves the cap works;
  it does not prove the frame rate at the cap.
- **Single JS bundle: 1,318.47 kB (gzip 359.63 kB).** No code-splitting; Vite warns about it.
  Acceptable on loopback, worth fixing before this is served over a real network.
- **`/health` reports the database version as `unknown`.** The driver does not expose a version
  string here; the running binary is `3.2.3+20260721.40522d1`, verified with `surreal version`.
- **`seed --clean` keeps imported evidence as reported drift.** It prunes synthetic strays and
  their edges only, so an import's `evidence` rows survive as extras (measured: 3 lots + 3 evidence
  rows kept on commercial). Nothing orphans them, but the report is the only place that says so —
  there is no "imported rows" view yet. `reset --confirm` remains the way to get exact counts.
- **`changes[].field` for a `relation.create` journal entry holds the target uid** rather than a
  field name. Honest enough to read, but it is not a field name.
- **`compareResultSchema` in `packages/contracts/src/index.ts:554` does not match the live compare
  payload** and is not used at runtime. Dead contract surface, left in place rather than half-edited.
- Single-user local POC: the session cookie is a signed actor/role/workspace binding, not
  authentication. There is no TLS, no password store, and no multi-tenant isolation story.