# ARCHITECTURE Spectra 3D ATLAS is a local, loopback-only operational graph platform: one authoritative SurrealDB holds ontology definitions, operational records, typed relations, evidence and an event journal; one Fastify API is the only writer on behalf of users; one React app renders that same snapshot as 3D, 2D, spatial scene and table. A local oMLX engine answers questions about it. Nothing here is a mock of a remote service. Every number on screen is computed from stored rows at request time, and every visible change was committed in a database transaction. ## System shape ``` ┌───────────────────────────────────────────────┐ browser (Chrome) │ React 19 + Vite dev server :5173 │ │ launcher → workspace shell │ │ Atlas (Graph3D | Graph2D | Scene | Table) │ │ Overview · Evidence · Data · Ontology │ │ Scenarios · Operations · Settings │ │ Inspector + Copilot panels │ └───────────────┬───────────────────────────────┘ fetch + SSE │ (cookie session, loopback only) ┌───────────────▼───────────────────────────────┐ │ Fastify API apps/api/src :8787 │ │ session/scope → zod contracts (@atlas/contracts)│ │ graph snapshot · mutations · actions │ │ ingest · lineage · ontology studio │ │ copilot (query plan → grounded answer) │ │ journal publisher + reconciliation watcher │ │ scenario clock (simulator) │ └──────┬────────────────────────┬───────────────┘ ws:// RPC │ │ HTTPS (OpenAI-compatible) ┌──────▼───────────────┐ ┌────▼──────────────────┐ │ SurrealDB 3.2.3 │ │ oMLX (user-owned) │ │ 127.0.0.1:18080 │ │ 127.0.0.1:8000/v1 │ │ disk: data/db/atlas-skv│ │ never started by ATLAS│ └──────────────────────┘ └───────────────────────┘ ``` Workspace layout: `packages/contracts` (zod schemas shared by API and UI), `packages/ontology` (segment packs, seed generators, deterministic metric functions), `apps/api` (routes + engine), `apps/web` (screens/panels/atlas views), `db/migrations` (10 ordered `.surql` files), `tools` (the only operational entry points), `tests/integration` (13 data/security/live-state checks), `tests/e2e/run.mjs` (browser journeys 13–20 plus the performance sample). ## Data model in one paragraph `workspace` scopes everything and carries the durable journal sequence (`lastSeq`). `ontology_type` and `ontology_link` are stored definitions (a live schema, editable at runtime), `entity` holds operational records with a strictly-typed envelope and a FLEXIBLE `properties` object, and `entity_link` is a real `TYPE RELATION FROM entity TO entity` table — edges are database rows with their own uid, version and properties, not an array inside a node. `atlas_event` is the append-only journal (`FOR update NONE, FOR delete NONE`). Evidence lives in `evidence` plus files under `data/evidence/` with a recorded sha256. Import runs, action runs, idempotency keys, AI runs, scenario patches/baselines and view preferences each get their own table. See `docs/ONTOLOGY.md` for the domain view. ## Write path (the only way state changes) 1. Route parses input with a zod contract and resolves the session scope (`workspaceId`, `segment`). 2. `apps/api/src/mutations.ts` validates the payload against the *stored* ontology registry (`packages/ontology` + `apps/api/src/registry.ts`) — unknown property, wrong type, or a link whose endpoints violate the stored `ontology_link` is rejected with per-field detail. 3. One `BEGIN … COMMIT` request writes the row(s), bumps `entity.version`, appends the `atlas_event` journal row, and allocates `workspace.lastSeq` in that same transaction (`apps/api/src/journal.ts`). A committed change can therefore never be missing its journal row, and a rolled-back change cannot burn a sequence number. 4. The journaled row also records `explained = { entities: [{uid, version}], relations: [...] }`, read inside the same transaction. That receipt is what lets the watcher tell "the API wrote this" from "somebody edited the database behind our back". Actions add a review step in front of that path: `POST /api/actions/:type/preview` computes preconditions and the projected change set against explicit base versions, and `POST /api/actions/:type/commit` re-validates from scratch, refuses a stale preview with a version conflict, and enforces idempotency through a unique index rather than an in-memory set. ## Read path `GET /api/graph` is the single snapshot contract (`apps/api/src/graph.ts`). The 3D scene, the 2D graph, the spatial Scene and the Table all render from that one response, and metric tiles come from `GET /api/metrics`, which runs the same deterministic functions over the same rows — so a picture and a number cannot disagree. Ordering is deterministic (nodes by `objectType`, then `uid`, then degree) so the busiest context survives truncation, and `truncated`/`totalMatchingNodes` say honestly when a cap bit. ## Live state See `docs/LIVE_STATE.md` for the full protocol. Summary: SSE carries *invalidations* carrying journal `seq`, never patches; the browser re-reads the authoritative scope, which makes duplicated, reordered or dropped notifications harmless; a slower `/api/events?after=cursor` poll and a stall detector close gaps without user action; the server independently reconciles out-of-band writes because `SHOW CHANGES` is not observable on this installation. ## Rendering `apps/web/src/atlas/Graph3D.tsx` renders with `frameloop="demand"`: frames are produced only for interaction, a live change, or while the layout settles. Node positions come from d3-force-3d in a Web Worker (`apps/web/src/layout/useLayout.ts`, `layout.worker.ts`), so layout never blocks the main thread; settled positions are cached per scope. Labels are drawn only for the highest-degree subset, and reduced motion switches to a static layout with no camera easing. If WebGL is missing the app says so in the panel and keeps 3D off — Graph2D and Table read the same snapshot, so a machine without GPU acceleration loses the effect but not the data. ## Local AI `apps/api/src/ai.ts` talks to oMLX through its OpenAI-compatible endpoint. The model is never trusted with SQL: it selects from a fixed tool set, and the API validates every query plan against the stored ontology before running it. Answers are assembled from retrieved rows and must cite the uids they came from; a response that cites an object not in the retrieved set is reported as an honest degraded state (`No grounded answer`) rather than passed off as a model failure. Retrieval is lexical + graph traversal; vector search is used only if `OMLX_EMBEDDING_MODEL` is set *and* verified. One inference runs at a time (`ATLAS_AI_MAX_CONCURRENCY=1`) because the renderer and the model share this Mac. ## Decisions (with the reason, not just the choice) 1. **SurrealDB is authoritative; Neo4j projection is deferred.** Relations are first-class rows, so traversal, edge properties and constraints live in one place with disk persistence and a real transaction boundary. A Neo4j copy would be a projection to validate later, not a second source of truth — building it now would mean synchronising two masters for no demonstrated need. 2. **Operational ontology, not RDF/OWL.** Types, link rules and property definitions are stored rows validated by server code. That keeps the ontology editable at runtime by Ontology Studio and keeps validation in one language. W3C PROV-O informs the evidence model (entities, activities, agents, derivation) without importing a triple store or a reasoner. 3. **Stable identity is separate from the storage key.** Every entity and every edge carries a `uid` (plus a unique index); record ids are treated as incidental. Reconciliation, evidence references, action targets and scenario patches all match on `uid`, which is what makes restore, replay and cross-workspace copies safe. 4. **Event journal + version receipts + out-of-band watcher.** Clients resume by cursor, so a reconnect is a re-read rather than a guess. Because `SHOW CHANGES FOR TABLE entity SINCE 0` returns an empty list on this build (measured), out-of-band detection compares stored uid+version sets against journal receipts; an unexplained difference must survive two consecutive passes before it is reported, and when it is, the watcher journals `outofband.detected`, captures a fresh baseline and marks history coverage *partial* instead of pretending the timeline is whole. 5. **3D has explicit limits and three fallbacks.** Default scope is a few hundred nodes; the cap is user-adjustable and reported when it truncates. Instanced/simple geometry, worker layout, cached positions and label pruning come before infrastructure. 2D graph, spatial Scene and Table are first-class views of the identical snapshot, not degraded screenshots of it. 6. **Local AI is capability-discovered and plan-validated.** The API probes the endpoint for model list, chat, tool calling and streaming at runtime instead of assuming them; every proposed query is validated against the stored ontology; grounding is enforced by citation checking. If a probe fails, the UI shows the degraded state and the answer path falls back to retrieval it can prove. 7. **Demo access controls, honestly labelled.** A signed cookie names the actor and role (`viewer`/`analyst`/`operator`/`owner`) and the API checks roles on every route, including SSE scope. That is enough to demonstrate authorisation and cross-workspace leakage protection. It is not production identity: there is no IdP, no password store, no rotation, and the signing secret lives in `.env.local`. Production hardening is listed in `docs/ROADMAP.md`. ## Consequences a maintainer should know - HTTP bodies above roughly 1 MB are rejected by this SurrealDB build (413), so history baselines are chunked into `snapshot_part` rows and restore batches are bounded by bytes *and* statement count. Do not "simplify" those loops into one giant request. - `RELATE type::record(...)` is a parse error here; endpoints are bound with `LET` first. - SCHEMAFULL tables reject undeclared columns: adding a field to a `SCHEMAFULL` table needs a migration, or the write fails opaquely. - `reservedQuantity` is always the sum of `properties.reservations`, enforced by both the seed generators and validation, so no screen can quote an inventory number that contradicts its ledger.