From 45c35a08f0bc23cca44c90a11b2ceb44248da91f Mon Sep 17 00:00:00 2001 From: gavrielc Date: Fri, 17 Apr 2026 10:37:26 +0300 Subject: [PATCH] docs(v2/db): add database architecture reference MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Split into three files linked from CLAUDE.md's v2 docs index: - v2-db.md — overview: three-DB model, cross-mount rules, central-vs-session decision, design patterns, readers/writers map. - v2-db-central.md — every table in data/v2.db plus migration history. - v2-db-session.md — per-session inbound.db / outbound.db schemas, session folder layout, seq parity invariant, lazy schema evolution. Co-Authored-By: Claude Opus 4.7 (1M context) --- CLAUDE.md | 3 + docs/v2-db-central.md | 319 ++++++++++++++++++++++++++++++++++++++++++ docs/v2-db-session.md | 183 ++++++++++++++++++++++++ docs/v2-db.md | 119 ++++++++++++++++ 4 files changed, 624 insertions(+) create mode 100644 docs/v2-db-central.md create mode 100644 docs/v2-db-session.md create mode 100644 docs/v2-db.md diff --git a/CLAUDE.md b/CLAUDE.md index e4b61dc21..06331365a 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -138,6 +138,9 @@ This project uses pnpm with `minimumReleaseAge: 4320` (3 days) in `pnpm-workspac |-----|---------| | [docs/v2-architecture-draft.md](docs/v2-architecture-draft.md) | Full architecture writeup | | [docs/v2-api-details.md](docs/v2-api-details.md) | Host API + DB schema details | +| [docs/v2-db.md](docs/v2-db.md) | DB architecture overview: three-DB model, cross-mount rules, readers/writers map | +| [docs/v2-db-central.md](docs/v2-db-central.md) | Central DB (`data/v2.db`) — every table + migration system | +| [docs/v2-db-session.md](docs/v2-db-session.md) | Per-session `inbound.db` + `outbound.db` schemas + seq parity | | [docs/v2-agent-runner-details.md](docs/v2-agent-runner-details.md) | Agent-runner internals + MCP tool interface | | [docs/v2-isolation-model.md](docs/v2-isolation-model.md) | Three-level channel isolation model | | [docs/v2-setup-wiring.md](docs/v2-setup-wiring.md) | What's wired, what's open in the setup flow | diff --git a/docs/v2-db-central.md b/docs/v2-db-central.md new file mode 100644 index 000000000..20e3896b3 --- /dev/null +++ b/docs/v2-db-central.md @@ -0,0 +1,319 @@ +# NanoClaw v2 — Central DB Schema + +Complete reference for `data/v2.db`, the host-owned admin-plane database. Start with [v2-db.md](v2-db.md) for the three-DB overview, the map, and the cross-mount rules. + +Access layer: `src/db/`. Authoritative schema reference: `src/db/schema.ts` (comments only — actual creation runs via migrations in `src/db/migrations/`). + +--- + +## 1. Tables + +### 1.1 `agent_groups` + +Agent workspaces. Each maps 1:1 to a `groups//` directory containing `CLAUDE.md`, skills, and `container.json`. Container config lives on disk, not in the DB. + +```sql +CREATE TABLE agent_groups ( + id TEXT PRIMARY KEY, + name TEXT NOT NULL, + folder TEXT NOT NULL UNIQUE, + agent_provider TEXT, + created_at TEXT NOT NULL +); +``` + +- **Readers:** `src/session-manager.ts`, `src/delivery.ts`, `src/router.ts` +- **Writers:** `src/db/agent-groups.ts` + +### 1.2 `messaging_groups` + +One row per platform chat (one WhatsApp group, one Slack channel, one 1:1 DM, etc.). + +```sql +CREATE TABLE messaging_groups ( + id TEXT PRIMARY KEY, + channel_type TEXT NOT NULL, + platform_id TEXT NOT NULL, + name TEXT, + is_group INTEGER DEFAULT 0, + unknown_sender_policy TEXT NOT NULL DEFAULT 'strict', + created_at TEXT NOT NULL, + UNIQUE(channel_type, platform_id) +); +``` + +- `unknown_sender_policy`: `strict` (drop), `request_approval` (ask admin), `public` (allow). +- **Readers:** `src/router.ts`, `src/delivery.ts`, `src/session-manager.ts` +- **Writers:** `src/db/messaging-groups.ts`, channel setup flows + +### 1.3 `messaging_group_agents` + +Wiring: which agent group handles which messaging group. Many-to-many — the same channel can route to multiple agents (see [v2-isolation-model.md](v2-isolation-model.md)). + +```sql +CREATE TABLE messaging_group_agents ( + id TEXT PRIMARY KEY, + messaging_group_id TEXT NOT NULL REFERENCES messaging_groups(id), + agent_group_id TEXT NOT NULL REFERENCES agent_groups(id), + trigger_rules TEXT, + response_scope TEXT DEFAULT 'all', + session_mode TEXT DEFAULT 'shared', + priority INTEGER DEFAULT 0, + created_at TEXT NOT NULL, + UNIQUE(messaging_group_id, agent_group_id) +); +``` + +- `session_mode`: `shared` (one session per channel), `per-thread` (one per thread), `agent-shared` (one per agent group across all channels). +- `trigger_rules`: JSON; e.g. regex for native channels. +- **Side effect:** creating a wiring must also populate `agent_destinations` — don't mutate one without the other (see §1.10). + +### 1.4 `users` + +Platform user identities. ID is namespaced: `tg:123456`, `discord:abc`, `phone:+1555...`, `email:a@x.com`. One human may own several rows — no cross-channel linking yet. + +```sql +CREATE TABLE users ( + id TEXT PRIMARY KEY, + kind TEXT NOT NULL, + display_name TEXT, + created_at TEXT NOT NULL +); +``` + +- **Writers/readers:** `src/db/users.ts`; channel auth flows + +### 1.5 `user_roles` + +Permissions. **Privilege is user-level, never agent-group-level.** + +```sql +CREATE TABLE user_roles ( + user_id TEXT NOT NULL REFERENCES users(id), + role TEXT NOT NULL, + agent_group_id TEXT REFERENCES agent_groups(id), + granted_by TEXT REFERENCES users(id), + granted_at TEXT NOT NULL, + PRIMARY KEY (user_id, role, agent_group_id) +); +CREATE INDEX idx_user_roles_scope ON user_roles(agent_group_id, role); +``` + +Invariants: +- `role = 'owner'` → must be global (`agent_group_id IS NULL`). Enforced in `grantRole()`. +- `role = 'admin'` → global (NULL) or scoped to one agent group. +- Admin @ A implies membership in A — no `agent_group_members` row required. + +Access layer: `src/db/user-roles.ts`, `src/access.ts`. + +### 1.6 `agent_group_members` + +Explicit membership for non-privileged users. Owner and admins don't need rows here — they're implicit members. + +```sql +CREATE TABLE agent_group_members ( + user_id TEXT NOT NULL REFERENCES users(id), + agent_group_id TEXT NOT NULL REFERENCES agent_groups(id), + added_by TEXT REFERENCES users(id), + added_at TEXT NOT NULL, + PRIMARY KEY (user_id, agent_group_id) +); +``` + +### 1.7 `user_dms` + +Cache of DM channel discovery. Lets the host send a cold DM (approval card, pairing code) without hitting the platform's `openConversation` API every time. + +```sql +CREATE TABLE user_dms ( + user_id TEXT NOT NULL REFERENCES users(id), + channel_type TEXT NOT NULL, + messaging_group_id TEXT NOT NULL REFERENCES messaging_groups(id), + resolved_at TEXT NOT NULL, + PRIMARY KEY (user_id, channel_type) +); +``` + +Populated lazily by `ensureUserDm()` in `src/user-dm.ts`. + +### 1.8 `sessions` + +Session registry. One row per (agent group, messaging group, thread) tuple subject to `session_mode`. Stores lifecycle metadata only — no messages. + +```sql +CREATE TABLE sessions ( + id TEXT PRIMARY KEY, + agent_group_id TEXT NOT NULL REFERENCES agent_groups(id), + messaging_group_id TEXT REFERENCES messaging_groups(id), + thread_id TEXT, + agent_provider TEXT, + status TEXT DEFAULT 'active', + container_status TEXT DEFAULT 'stopped', + last_active TEXT, + created_at TEXT NOT NULL +); +CREATE INDEX idx_sessions_agent_group ON sessions(agent_group_id); +CREATE INDEX idx_sessions_lookup ON sessions(messaging_group_id, thread_id); +``` + +- **Resolved by:** `resolveSession()` in `src/session-manager.ts`. +- Creating a session also provisions the session folder and both session DBs via `initSessionFolder()` — see [v2-db-session.md](v2-db-session.md). + +### 1.9 `pending_questions` + +The `ask_user_question` MCP tool parks an interactive question here, and the container matches incoming `system` messages back to it by `questionId`. + +```sql +CREATE TABLE pending_questions ( + question_id TEXT PRIMARY KEY, + session_id TEXT NOT NULL REFERENCES sessions(id), + message_out_id TEXT NOT NULL, + platform_id TEXT, + channel_type TEXT, + thread_id TEXT, + title TEXT NOT NULL, + options_json TEXT NOT NULL, + created_at TEXT NOT NULL +); +``` + +### 1.10 `agent_destinations` + +Permission ACL *and* name-resolution map for outbound sending. An agent asking to `send_message(to="dev-channel")` must have a row here with `local_name = 'dev-channel'`, or the send is rejected as `unknown destination`. + +```sql +CREATE TABLE agent_destinations ( + agent_group_id TEXT NOT NULL REFERENCES agent_groups(id), + local_name TEXT NOT NULL, + target_type TEXT NOT NULL, -- 'channel' | 'agent' + target_id TEXT NOT NULL, -- messaging_group_id | agent_group_id + created_at TEXT NOT NULL, + PRIMARY KEY (agent_group_id, local_name) +); +CREATE INDEX idx_agent_dest_target ON agent_destinations(target_type, target_id); +``` + +**Projection invariant (load-bearing).** The central table is the source of truth, but each running container reads from a projection in its own `inbound.db` (see [v2-db-session.md §2.3](v2-db-session.md#23-destinations)). Any code that mutates `agent_destinations` while a container is running must also call `writeDestinations()` (`src/session-manager.ts`) or the container will reject sends with stale data. Known call sites: `createMessagingGroupAgent()` in `src/db/messaging-groups.ts`, the `create_agent` system action in `src/delivery.ts`. + +Access layer: `src/db/agent-destinations.ts`. + +### 1.11 `pending_approvals` + +Two workflows share this table: + +- **Session-bound MCP approvals** — `install_packages`, `request_rebuild`, `add_mcp_server`. `session_id` is set. +- **OneCLI credential approvals** — `session_id` may be NULL; `agent_group_id` + `channel_type` + `platform_id` route the admin card. + +```sql +CREATE TABLE pending_approvals ( + approval_id TEXT PRIMARY KEY, + session_id TEXT REFERENCES sessions(id), + request_id TEXT NOT NULL, + action TEXT NOT NULL, + payload TEXT NOT NULL, + created_at TEXT NOT NULL, + agent_group_id TEXT REFERENCES agent_groups(id), + channel_type TEXT, + platform_id TEXT, + platform_message_id TEXT, + expires_at TEXT, + status TEXT NOT NULL DEFAULT 'pending', + title TEXT NOT NULL DEFAULT '', + options_json TEXT NOT NULL DEFAULT '[]' +); +CREATE INDEX idx_pending_approvals_action_status ON pending_approvals(action, status); +``` + +- `status`: `pending` | `approved` | `rejected` | `expired`. +- `platform_message_id` lets the host edit the admin card in place after a decision. +- Access layer: `src/db/sessions.ts`; sweep + delivery: `src/onecli-approvals.ts`. + +### 1.12 `unregistered_senders` + +Audit trail: every time a message gets dropped (unknown sender, strict policy), we increment a counter here so admins can see who's been trying to knock. + +```sql +CREATE TABLE unregistered_senders ( + channel_type TEXT NOT NULL, + platform_id TEXT NOT NULL, + user_id TEXT, + sender_name TEXT, + reason TEXT NOT NULL, + messaging_group_id TEXT, + agent_group_id TEXT, + message_count INTEGER NOT NULL DEFAULT 1, + first_seen TEXT NOT NULL, + last_seen TEXT NOT NULL, + PRIMARY KEY (channel_type, platform_id) +); +CREATE INDEX idx_unregistered_senders_last_seen ON unregistered_senders(last_seen); +``` + +Writer: `recordDroppedMessage()` in `src/db/dropped-messages.ts`. On conflict, bumps `message_count` + `last_seen`. + +### 1.13 Chat SDK bridge tables + +State backing the `SqliteStateAdapter` used by the Chat SDK bridge (see [v2-api-details.md](v2-api-details.md)). NanoClaw code rarely touches these directly — they're owned by `src/state-sqlite.ts`. + +```sql +CREATE TABLE chat_sdk_kv ( + key TEXT PRIMARY KEY, + value TEXT NOT NULL, + expires_at INTEGER -- unix ts, nullable +); + +CREATE TABLE chat_sdk_subscriptions ( + thread_id TEXT PRIMARY KEY, + subscribed_at TEXT NOT NULL DEFAULT (datetime('now')) +); + +CREATE TABLE chat_sdk_locks ( + thread_id TEXT PRIMARY KEY, + token TEXT NOT NULL, + expires_at INTEGER NOT NULL +); + +CREATE TABLE chat_sdk_lists ( + key TEXT NOT NULL, + idx INTEGER NOT NULL, + value TEXT NOT NULL, + expires_at INTEGER, + PRIMARY KEY (key, idx) +); +``` + +### 1.14 `schema_version` + +Migration ledger, written by the migration runner (§2). + +```sql +CREATE TABLE schema_version ( + version INTEGER PRIMARY KEY, + name TEXT NOT NULL, + applied TEXT NOT NULL +); +``` + +--- + +## 2. Migration system + +Migrations live in `src/db/migrations/`, one file per migration. Runner: `runMigrations()` in `src/db/migrations/index.ts`. It: + +1. Creates `schema_version` if absent. +2. Reads `MAX(version)` — call it `current`. +3. For each migration with `version > current`, executes `up(db)` inside a transaction and appends a `schema_version` row. + +| # | File | Introduces | +|---|------|------------| +| 001 | `001-initial.ts` | Core v2 tables: `agent_groups`, `messaging_groups`, `messaging_group_agents`, `users`, `user_roles`, `agent_group_members`, `user_dms`, `sessions`, `pending_questions` | +| 002 | `002-chat-sdk-state.ts` | `chat_sdk_kv`, `chat_sdk_subscriptions`, `chat_sdk_locks`, `chat_sdk_lists` | +| 003 | `003-pending-approvals.ts` | `pending_approvals` (session-bound + OneCLI fields) | +| 004 | `004-agent-destinations.ts` | `agent_destinations` + backfill from existing `messaging_group_agents` wirings | +| 007 | `007-pending-approvals-title-options.ts` | `ALTER TABLE pending_approvals` add `title`, `options_json` (retrofits DBs created between 003 and 007) | +| 008 | `008-dropped-messages.ts` | `unregistered_senders` | +| 009 | `009-drop-pending-credentials.ts` | Drop the defunct `pending_credentials` table | + +Numbers 005 and 006 are intentionally absent — migrations were renumbered during v2 development. + +Session DB schemas (`INBOUND_SCHEMA`, `OUTBOUND_SCHEMA`) are **not** versioned here. They're `CREATE TABLE IF NOT EXISTS` so new columns land via the session-DB lazy migration helpers (`migrateDeliveredTable()` etc.) when a session file from an older build is reopened. See [v2-db-session.md](v2-db-session.md). diff --git a/docs/v2-db-session.md b/docs/v2-db-session.md new file mode 100644 index 000000000..55dc66483 --- /dev/null +++ b/docs/v2-db-session.md @@ -0,0 +1,183 @@ +# NanoClaw v2 — Per-Session DB Schema + +Reference for the two SQLite files each session owns: `inbound.db` (host writes, container reads) and `outbound.db` (container writes, host reads). Start with [v2-db.md](v2-db.md) for the three-DB overview, the single-writer rule, and the cross-mount visibility constraints. + +Schemas live in `src/db/schema.ts` as the `INBOUND_SCHEMA` and `OUTBOUND_SCHEMA` constants. Both files are created by `ensureSchema()` in `src/session-manager.ts` when a new session folder is provisioned. + +--- + +## 1. Session folder layout + +``` +data/v2-sessions/// + inbound.db ← host writes, container reads (read-only mount) + outbound.db ← container writes, host reads (read-only open) + .heartbeat ← mtime touched by container (not a DB write) + inbox// ← user attachments, decoded from inbound message content + outbox// ← attachments the agent produced +``` + +One session = one folder = one pair of DBs. The `agent_group_id` parent directory also holds per-group state (`.claude-shared/`, `agent-runner-src/`) that is shared across every session of that agent group. + +Path helpers in `src/session-manager.ts`: `sessionDir()`, `inboundDbPath()`, `outboundDbPath()`, `heartbeatPath()`. + +--- + +## 2. Inbound DB (`inbound.db`) + +Host-owned, container-read-only. Schema constant: `INBOUND_SCHEMA` in `src/db/schema.ts`. + +### 2.1 `messages_in` + +Every message landing in the session: user chat, scheduled task, recurring task, question response, internal system message. + +```sql +CREATE TABLE messages_in ( + id TEXT PRIMARY KEY, + seq INTEGER UNIQUE, -- EVEN only (host assigns) — see §3 + kind TEXT NOT NULL, + timestamp TEXT NOT NULL, + status TEXT DEFAULT 'pending', -- pending|completed|failed|paused + process_after TEXT, + recurrence TEXT, -- cron expr for recurring + series_id TEXT, -- groups occurrences of a recurring task + tries INTEGER DEFAULT 0, + platform_id TEXT, + channel_type TEXT, + thread_id TEXT, + content TEXT NOT NULL -- JSON; shape depends on kind +); +CREATE INDEX idx_messages_in_series ON messages_in(series_id); +``` + +Content shapes: see [v2-api-details.md §Session DB Schema Details](v2-api-details.md#session-db-schema-details). + +**Writers (host):** `insertMessage()`, `insertTask()`, `insertRecurrence()` — all in `src/db/session-db.ts`. Each calls `nextEvenSeq()`. +**Reader (container):** `container/agent-runner/src/db/messages-in.ts` — polls `status='pending' AND (process_after IS NULL OR process_after <= now)`. + +### 2.2 `delivered` + +Host writes here after handing a `messages_out` row to the channel adapter. Container reads `platform_message_id` to target edits and reactions. + +```sql +CREATE TABLE delivered ( + message_out_id TEXT PRIMARY KEY, + platform_message_id TEXT, + status TEXT NOT NULL DEFAULT 'delivered', -- delivered|failed + delivered_at TEXT NOT NULL +); +``` + +Writer: `markDelivered()` / `markDeliveryFailed()` in `src/db/session-db.ts`. Older session DBs are brought up to schema lazily by `migrateDeliveredTable()`. + +### 2.3 `destinations` + +Projection of the central `agent_destinations` table (see [v2-db-central.md §1.10](v2-db-central.md#110-agent_destinations)) for this session's agent. The container resolves `to="name"` against this table; if the row is absent, the send is rejected as `unknown destination`. + +```sql +CREATE TABLE destinations ( + name TEXT PRIMARY KEY, + display_name TEXT, + type TEXT NOT NULL, -- 'channel' | 'agent' + channel_type TEXT, -- for type='channel' + platform_id TEXT, -- for type='channel' + agent_group_id TEXT -- for type='agent' +); +``` + +Rewritten wholesale (DELETE + INSERT in a transaction) by `writeDestinations()` on every container wake and on demand when wiring changes mid-session. The comment on the table in `src/db/schema.ts` is the canonical statement of the refresh semantics. + +### 2.4 `session_routing` + +Single-row (`id=1`) default routing: where outbound messages go when the agent doesn't specify a destination. + +```sql +CREATE TABLE session_routing ( + id INTEGER PRIMARY KEY CHECK (id = 1), + channel_type TEXT, + platform_id TEXT, + thread_id TEXT +); +``` + +Written by `writeSessionRouting()` on every container wake, derived from `sessions.messaging_group_id` + `sessions.thread_id`. + +--- + +## 3. Sequence numbering invariant + +Every message (in or out) gets a monotonic integer `seq`, unique *within the session* across both tables. + +- **Host writes even seq** (2, 4, 6, …) to `messages_in` — `nextEvenSeq()` at `src/db/session-db.ts:75`. +- **Container writes odd seq** (1, 3, 5, …) to `messages_out` — logic at `container/agent-runner/src/db/messages-out.ts:54` (`max % 2 === 0 ? max + 1 : max + 2`), reading `MAX(seq)` across *both* tables to preserve global ordering. + +Why disjoint? `seq` is the agent-facing message ID. When the agent calls `edit_message(seq=5)` or `add_reaction(seq=6)`, `getMessageIdBySeq()` uses the parity to route the lookup: odd → `messages_out`, even → `messages_in`. The parity alone disambiguates without a join. Collisions would break editing. + +If you add a code path that writes to either table, preserve parity — the invariant isn't enforced by a constraint, only by the two helper functions. + +--- + +## 4. Outbound DB (`outbound.db`) + +Container-owned, host reads only. Schema constant: `OUTBOUND_SCHEMA` in `src/db/schema.ts`. + +### 4.1 `messages_out` + +Everything the agent produces: chat replies, edits, reactions, cards, question sends, agent-to-agent messages, system actions. + +```sql +CREATE TABLE messages_out ( + id TEXT PRIMARY KEY, + seq INTEGER UNIQUE, -- ODD only (container assigns) — see §3 + in_reply_to TEXT, + timestamp TEXT NOT NULL, + deliver_after TEXT, + recurrence TEXT, + kind TEXT NOT NULL, -- chat|chat-sdk|system|… + platform_id TEXT, + channel_type TEXT, + thread_id TEXT, + content TEXT NOT NULL -- JSON; operation lives inside (edit/reaction/card/…) +); +``` + +Content shapes: see [v2-api-details.md §Session DB Schema Details](v2-api-details.md#session-db-schema-details). + +**Writer (container):** `writeMessageOut()` in `container/agent-runner/src/db/messages-out.ts`. +**Readers (host):** `src/delivery.ts` (polling delivery), `getMessageIdBySeq()` / `getRoutingBySeq()` for edit/reaction targeting. + +### 4.2 `processing_ack` + +Container-side status for each `messages_in.id` it has touched. The host polls this and syncs status back into `messages_in` — this avoids the container ever writing to `inbound.db`. + +```sql +CREATE TABLE processing_ack ( + message_id TEXT PRIMARY KEY, + status TEXT NOT NULL, -- processing|completed|failed + status_changed TEXT NOT NULL +); +``` + +Crash recovery: on container startup, stale `processing` entries get cleared. Host-side sync: `syncProcessingAcks()` in `src/host-sweep.ts`. + +### 4.3 `session_state` + +Persistent container-owned KV store. Main consumer is the Chat SDK session ID — storing it here lets the agent's conversation resume across container restarts. Cleared by `/clear`. + +```sql +CREATE TABLE session_state ( + key TEXT PRIMARY KEY, + value TEXT NOT NULL, + updated_at TEXT NOT NULL +); +``` + +Access: `container/agent-runner/src/db/session-state.ts`. + +--- + +## 5. Schema evolution + +Unlike the central DB, session DBs do **not** go through numbered migrations. Both `INBOUND_SCHEMA` and `OUTBOUND_SCHEMA` use `CREATE TABLE IF NOT EXISTS`, so a fresh session always gets the current shape. For session folders created under older builds, column-level gaps are patched lazily on open — e.g. `migrateDeliveredTable()` in `src/db/session-db.ts` adds `platform_message_id` and `status` to the `delivered` table if missing. + +If you add a column to either schema, add a matching lazy migration for existing session folders, and prefer nullable columns or defaulted values so no data backfill is required. diff --git a/docs/v2-db.md b/docs/v2-db.md new file mode 100644 index 000000000..7590d6797 --- /dev/null +++ b/docs/v2-db.md @@ -0,0 +1,119 @@ +# NanoClaw v2 Database Architecture — Overview + +Orientation for the v2 data model: the three databases, how they fit together, and the invariants that hold across them. For table-level schemas, follow the links below. + +- **[v2-db-central.md](v2-db-central.md)** — every table in `data/v2.db` (identity, wiring, approvals, Chat SDK state) plus the migration system. +- **[v2-db-session.md](v2-db-session.md)** — the per-session `inbound.db` + `outbound.db` pair, seq parity, and session folder layout. + +Related: [v2-architecture-draft.md](v2-architecture-draft.md) for the high-level design; [v2-api-details.md](v2-api-details.md) for inbound/outbound message content shapes; [v2-isolation-model.md](v2-isolation-model.md) for channel-to-agent wiring modes. + +--- + +## 1. The three databases + +v2 uses **three kinds of SQLite database**, all on the host filesystem: + +| DB | Location | Writer | Readers | Purpose | +|----|----------|--------|---------|---------| +| **Central** | `data/v2.db` | host | host | Identity, permissions, routing, wiring — the admin plane | +| **Session inbound** | `data/v2-sessions///inbound.db` | host | host (sync), container (read-only) | Host → container messages + routing projections | +| **Session outbound** | `data/v2-sessions///outbound.db` | container | host (poll), container | Container → host messages + processing status | + +**Single-writer rule.** Every SQLite file has exactly one writer. Host writes the central DB and every `inbound.db`; container writes only its own `outbound.db`. This eliminates write contention across the Docker/Apple Container mount boundary — SQLite locking across that boundary is unreliable. + +**Everything is a message.** There is no IPC, stdin piping, or file watcher between host and container. The two session DBs are the sole IO surface. Heartbeat is a file `touch(2)` on `.heartbeat`, not a DB write. + +**Journal mode.** Session DBs use `journal_mode = DELETE` (not WAL). Cross-mount WAL visibility is a bug farm; DELETE mode + open-write-close forces the page cache to flush so the other side sees changes. + +--- + +## 2. Database map + +``` +data/ + v2.db ← CENTRAL (host ↔ host) + v2-sessions/ + / + .claude-shared/ ← shared Claude state for the agent group + agent-runner-src/ ← per-group agent-runner overlay + / + inbound.db ← host writes, container reads + outbound.db ← container writes, host reads + .heartbeat ← mtime touched by container + inbox// ← decoded user attachments + outbox// ← attachments the agent produced +``` + +Path helpers: `sessionDir()`, `inboundDbPath()`, `outboundDbPath()`, `heartbeatPath()` — all in `src/session-manager.ts`. + +--- + +## 3. Central vs. session: what goes where + +| Kind of data | Where | Why | +|--------------|-------|-----| +| Identities, roles, memberships | central | Stable, cross-session, rarely written | +| Channel wiring, routing rules | central | Admin plane | +| Destination ACL | central (+ projection per session) | Source of truth centrally; fast local lookup per session | +| Session registry (ids, status) | central | Host orchestrates lifecycle | +| Approvals & pending questions | central | Survive container restarts, admin-visible | +| Dropped-message audit | central | Global ops view | +| Inbound messages, retry state | session `inbound.db` | Per-session workload; host is sole writer | +| Outbound messages, agent state | session `outbound.db` | Container is sole writer; host polls | +| Delivery outcome | session `inbound.db` (`delivered`) | Host writes on success; container reads for edit targeting | +| Processing status | session `outbound.db` (`processing_ack`) | Container can't write to `inbound.db` | + +Heuristic: if the value is a message, routing projection, or runtime ack, it goes per-session. Everything else is central. + +--- + +## 4. Cross-mount visibility + +Session DBs are bind-mounted into the container. A few rules you need to know before touching the DB code: + +- **`journal_mode = DELETE`, not WAL.** WAL files don't reliably cross the mount and the container can read stale pages. DELETE mode forces each writer to flush the main file. +- **Open-write-close on the host.** Host-side writes to `inbound.db` open a connection, write, and close it. Keeping a handle open makes cached pages invisible to the container. +- **Container reads read-only.** The container opens `inbound.db` with `readonly: true` and never writes — all container→host state goes through `outbound.db` (see `processing_ack` in [v2-db-session.md](v2-db-session.md#52-processing_ack)). +- **Heartbeat is a file touch.** `.heartbeat` mtime is the liveness signal, not a DB column. A DB write per heartbeat would serialize behind other writers. + +These rules are enforced by convention in `src/session-manager.ts` and `container/agent-runner/src/db/`. If you change how the DBs are opened, re-read that code first. + +--- + +## 5. Design patterns at a glance + +1. **Two-DB session split.** `inbound.db` and `outbound.db` each have one writer, one direction of flow — no cross-mount lock contention. +2. **Seq parity.** Even = host, odd = container. Disjoint namespace across both tables lets the agent reference any message by `seq` alone. Details in [v2-db-session.md §3](v2-db-session.md#3-sequence-numbering-invariant). +3. **Projection pattern.** `agent_destinations` and `session_routing` are projected from the central DB into each session's `inbound.db` on container wake — the container gets a fast, local read path without querying across the mount. +4. **Ack via reverse channel.** Container never writes to `inbound.db`. Status sync happens through `processing_ack` in `outbound.db`, which the host polls and reconciles. +5. **Heartbeat out of band.** File `touch` on `.heartbeat`, not a DB write, so liveness doesn't serialize behind other writers. +6. **Lazy session-DB migrations.** Central DB uses numbered migrations; per-session DBs use `IF NOT EXISTS` + ad-hoc `ALTER TABLE` helpers for older session folders. +7. **ACL = row existence.** `agent_destinations` membership is itself the permission — no separate `permissions` table. + +--- + +## 6. Readers & writers — at a glance + +| Table | DB | Writer(s) | Reader(s) | +|-------|----|-----------|-----------| +| `agent_groups` | central | `src/db/agent-groups.ts` | session resolver, delivery, router | +| `messaging_groups` | central | `src/db/messaging-groups.ts`, channel setup | router, delivery, session resolver | +| `messaging_group_agents` | central | `src/db/messaging-groups.ts` | router | +| `users` | central | `src/db/users.ts`, auth flows | permission checks | +| `user_roles` | central | `src/db/user-roles.ts` | `src/access.ts`, all permission gates | +| `agent_group_members` | central | `src/db/agent-group-members.ts` | membership checks | +| `user_dms` | central | `src/user-dm.ts` (`ensureUserDm`) | approval + pairing delivery | +| `sessions` | central | `src/db/sessions.ts`, `src/session-manager.ts` | delivery, sweep, container runner | +| `pending_questions` | central | `src/db/sessions.ts` (via `ask_user_question`) | container response matcher | +| `agent_destinations` | central | `src/db/agent-destinations.ts`, migration 004 backfill | `writeDestinations()`, delivery ACL | +| `pending_approvals` | central | `src/db/sessions.ts`, `src/onecli-approvals.ts` | admin-card delivery, sweep | +| `unregistered_senders` | central | `src/db/dropped-messages.ts` | ops tooling | +| `chat_sdk_*` | central | `src/state-sqlite.ts` | Chat SDK bridge | +| `schema_version` | central | `src/db/migrations/index.ts` | migration runner | +| `messages_in` | inbound | `src/db/session-db.ts` | `container/agent-runner/src/db/messages-in.ts` | +| `delivered` | inbound | `src/db/session-db.ts` (`markDelivered`) | container edit/reaction targeting | +| `destinations` | inbound | `writeDestinations()` in `src/session-manager.ts` | container routing / ACL | +| `session_routing` | inbound | `writeSessionRouting()` in `src/session-manager.ts` | container `send_message` defaults | +| `messages_out` | outbound | `container/agent-runner/src/db/messages-out.ts` | `src/delivery.ts` poll loop | +| `processing_ack` | outbound | `container/agent-runner/src/db/messages-in.ts` | `src/host-sweep.ts` (`syncProcessingAcks`) | +| `session_state` | outbound | `container/agent-runner/src/db/session-state.ts` | container on startup |