Compare commits

..

58 Commits

Author SHA1 Message Date
Moshe Krupper 0b5ba5cee2 feat: /add-audit skill — opt-in ncl-surface audit log on the guard seam
Ships the local audit log as an install skill, scoped to the ncl command
surface: one withAudit(dispatch) composition at the definition site covers
both transports and the grant-carrying approved replay; holds are recorded as
pending events correlated by approval id (recovered from the hold row), so
--correlation returns the whole gated chain. src/audit/ installs as a
domain-free leaf — schema v1, NDJSON day-files under data/audit/ (bucketed by
event time), fail-open + loud emit that brackets the dispatcher so a thrown
command still leaves a record, post-write exporter hooks, boot writability
refusal, self-contained retention prune. `ncl audit list` is host +
global-scope only (off the group-scope allowlist, fails closed), validates its
filter flags (a typo is rejected, not silently ignored), reads --since/--until
as UTC, and streams the full window on --format ndjson (no silent 100-row cap).
Core reach-ins: the dispatch composition and the resource-barrel import, both
guarded by a shipped AST/behavior wiring test that also pins guard conformance
clean.

Value-free recording model (hardened per the max-effort review): the event
stores WHO did WHICH action to WHAT target and the outcome — never raw
argument values. `details` carry the flag names that were passed plus a small
allowlist of governance-relevant enum values (role, mode, session_mode,
cli_scope, access, engage_mode, sender_scope, provider, model) echoed
verbatim; every other value is name-only, and a failure keeps the error CODE,
never the free-text message. This replaces the key-pattern redactor outright —
a secret in an error message, a malformed-JSON arg, or an innocent key can no
longer land, because no free-form value is ever written. Target ids surface
structurally in resources.

Other review fixes: emit-on-throw; an approved replay records as `approved`
(rejected dropped until the approval-lifecycle increment); --help probes
record under a neutral cli.help action instead of masquerading as the real
verb succeeding; --limit 0 is honored as zero rows; and an exporter hook whose
module loads after boot still runs its one-time init() — the registration seam
is import-order-insensitive (onEvent/maintain/shutdown already read the live
list; init() now self-heals on late registration).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-13 14:29:47 +03:00
github-actions[bot] 2d3257c601 chore: bump version to 2.1.47 2026-07-13 11:24:03 +00:00
github-actions[bot] a467359d86 docs: update token count to 240k tokens · 120% of context window 2026-07-13 11:24:00 +00:00
Moshe Krupper 40d26893f8 Guard seam: one decision function for every privileged action (guarded-actions phase 2) (#2986)
* feat: guard seam — decision function, registration wrapping, grant-carrying replay, boot conformance

Every privileged action crossing the container or channel boundary now
passes one decision function — guard() in the new src/guard/ leaf —
before it executes: allow | hold | deny (hub:
engineering/requirements/guarded-actions +
engineering/discovery/guarded-actions-decisions, phase 2).

- Registration-derived catalog: registry.register() derives one entry
  per ncl command (dotted action names stamped by registerResource);
  module-edge guard.ts adapters register the domain baselines
  (agents.create, a2a.send incl. the agent_message_policies hold,
  self_mod.*, senders.admit, channels.register). The baseline is the
  decision — policy-as-data (tighten-only rules) is deferred to
  phase 3.
- All four handler registries wrap at registration: dispatch consults
  the guard; guarded delivery actions (create_agent, install_packages,
  add_mcp_server) store only the precheck → guard → handler wrapper
  (the raw handler is never stored; spec-less re-registration throws);
  response handlers + message interceptors take guard specs (channel
  registration's click + free-text name capture).
- Grant-carrying replay: approved continuations re-enter their entry
  point with the verified approval row as the grant
  (ApprovalHandlerContext.approval). A grant satisfies a hold, never a
  deny — live-row + approvalAction + grantMatches checks; the forgeable
  approved:true boolean is deleted.
- Boot conformance: the registry walk (src/guard-conformance.ts) runs
  in CI and at boot — an unmapped privileged registration stops the
  host with a banner (skill-installed code never runs this repo's CI).

Baselines are main's behavior verbatim (host trusted-caller, cli_scope
allowlist, create_agent scope branch, a2a ACL order, unconditional
self-mod hold, unknown_sender_policy, channel click auth incl. the
anchor-group approver). Sender/channel holds stay on their own tables;
guard holds map onto the existing requestApproval options
(approverUserId).

Deliberate outcome changes inherent to the replay semantics (called out
in CHANGELOG too): (a) a2a approve-then-revoke no longer delivers — the
structural baseline re-runs live on replay; (b) forged, already-consumed,
or mismatched grants refuse instead of executing; (c) the channel-name
free-text reply re-checks approver eligibility at reply time. The
D1/D2/D4 click-auth fixes are deliberately NOT here — they belong to the
approval-contract PR (D2's host fallback at dispatch replay is preserved
verbatim).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* refactor: rename src/guard/catalog.ts → guard-actions.ts

File + internal map (catalog → guardedActions). The concept stays "the
action catalog" in prose (the term the requirements/decisions docs and
the conformance banner use); exported symbols (registerGuardedAction,
GuardedActionSpec, …) unchanged.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* refactor(guard): consults carry the defined action value — delete the registry walk

The guard's weak point was the wiring: catalog entries registered by
side-effect import, consult sites naming them by string, a map miss
resolving to ALLOW, and a boot walk that could only see 2 of the 4
registries (response handlers and interceptors erased their specs into
closures). Dropping one unreferenced import silently disarmed
channel-registration click-auth.

Make the broken wiring unconstructible instead of detected:

- defineGuardedAction returns a branded GuardedAction value; guard(action,
  input) takes the value, not a name. A dropped module-edge import or a
  typo'd action is now a compile error; there is no lookup and no
  fail-open branch. A forged value (outside TS) is denied at runtime.
- Every registry (delivery actions, response handlers, interceptors)
  requires a guard spec or an explicit unguarded(<reason>) declaration at
  the registration site — unguarded-by-omission is not representable, and
  the justification travels with the registration (grep "unguarded(" for
  the complete inventory). CLI commands derive their guard inside
  register(), so a command cannot exist without one.
- guardConformanceViolations() and EXEMPT_DELIVERY_ACTIONS are gone. The
  boot check shrinks to the one cross-registry invariant the compiler
  can't see: every holding action has a registered approve continuation
  (grantContinuationGaps), same fail-closed refuse-to-start posture.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* refactor(guard): one runtime discriminator for Unguarded — isUnguarded()

The unguarded brand becomes a real (module-private) symbol and
isUnguarded() its exported type guard, replacing three hand-rolled
`'reason' in guardDecl` checks (delivery, router, response-registry)
plus the one inside isUnguardedEntry. Only unguarded() can mint the
brand now, so a look-alike { reason } object — or a guard spec that
someday grows a `reason` field — can't pass as an unguarded
declaration at runtime.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* refactor(guard): unwrap the broadcast registries — response handlers and interceptors consult inline

registerResponseHandler and registerMessageInterceptor return to main's
single-argument shape: the guard-spec/unguarded declaration machinery
(claims, onDeny, the registry wrappers) is deleted. The declaration
requirement stays where registration is keyed — every ncl command derives
its guard inside register(), and registerDeliveryAction demands a spec or
unguarded(<reason>) — while broadcast hooks gate inline at the point of
privilege, like the a2a route and the unknown-sender gate:

- handleChannelApprovalResponse consults guard(channelsRegister) inline
  where main had the click-auth if — same decision, same timing.
- The free-text name capture returns to main verbatim: the click arms it,
  the reply is not re-authorized (the body's own pending-row re-fetch
  still refuses a vanished registration). Behavior delta (c) is withdrawn
  — the deliberate deltas are back to (a) approve-then-revoke and
  (b) grant refusals.

Also gone with the wrappers: the claims/body predicate duplication (the
same early-exits existed twice and could drift apart) and the repeated
extractAndUpsertUser upsert per captured event.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* refactor(guard): drop boot conformance, rename the spec vocabulary, extract runGuarded

Three trims to the seam, no behavior change (713/713 green):

- Boot-time conformance check deleted (src/guard-conformance.ts + its
  main() call). The invariant — every holding action has a registered
  approve continuation — is already covered in-tree by the conformance
  test, and at runtime response-handler.ts handles a missing continuation
  loudly (warn + "approved, but no handler is installed" notify + row
  cleanup). Refusing to boot bought no safety over that and could brick
  the host on a mis-installed skill.

- GuardedActionSpec vocabulary renamed to read as what each field does:
  baseline → decide (the decision function — allow | hold | deny),
  approvalAction → grantActionName (the pending_approvals.action string a
  grant's row must carry; "Name" because at consult sites `action` means
  the branded GuardedAction value), grantMatches → grantCoversRequest.
  Comment prose follows ("structural baseline" → the checks / the
  decision).

- runGuarded + DeliveryGuardSpec extracted to src/delivery-guard.ts.
  delivery.ts is a high-traffic file for forks: the registry itself
  (tagged entries, spec-or-unguarded registration overloads, disarm-throw,
  one-door getDeliveryAction, grant-carrying reenter) stays there, evolved
  in place — only the consult pipeline (precheck → guard → deny/hold/allow)
  and the spec type move out; runGuarded takes spec + handler explicitly so
  the new file has no import back into delivery.ts.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* docs: drop internal design-process references from the guard change

The public tree shouldn't point at internal artifacts: phase numbering
("guarded-actions phase 2/3") and the team-hub decisions-doc pointer are
gone from the CHANGELOG bullet, the CLAUDE.md guard row, and the guard
file headers. The facts stay ("policy-as-data is deliberately deferred —
a generalized rules table can arrive later"); only the internal roadmap
vocabulary goes. Scope: lines this branch introduced only.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* docs(guard): the name reply is not re-authorized — stop claiming it is

The channelsRegister docstring and the CLAUDE.md guard row still described
the withdrawn reply-time re-check (behavior delta (c), removed in fe235c7):
the docstring claimed the decision is consulted by the name-capture
interceptor "so a privilege revoked mid-flow is re-checked at each step",
and the CLAUDE.md row said message interceptors consult the guard inline.
Neither is true — channels.register is consulted only by the card-click
handler, and the free-text reply deliberately authorizes off the click
(main's behavior). Both docs now state that explicitly, including the
consequence: a privilege revoked between click and reply still completes
the flow.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* fix(guard): an approved-then-refused a2a replay is a policy outcome, not a crash (GS-002)

Runtime testing (PR #2986 validation, finding GS-002) showed the
approve-after-revoke refusal — the PR's own deliberate behavior delta —
surfacing as "ERROR Approval handler threw" with a stack trace, and the
requester being told the apply "failed". Enforcement was correct; the
telemetry dressed an expected guard refusal as a runtime failure.

The a2a route's deny now throws a typed GuardDenyError (guard leaf), and
applyA2aMessageGate catches exactly that: the requester is notified
"Message approved, but not delivered — no longer authorized: <reason>"
and the host logs a warning. Anything else still propagates to the
response handler's crash path. Covers all three replay refusals the same
way: destination revoked while pending, mismatched grant, already-consumed
grant.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* docs: shrink the guard CHANGELOG bullets and CLAUDE.md rows to house length

The guard-seam CHANGELOG bullet was a design document; it now reads like
the neighboring entries — what changed, the two behavior deltas, done.
Same for the src/guard/ and src/delivery-guard.ts Key Files rows.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
Co-authored-by: gavrielc <gabicohen22@yahoo.com>
2026-07-13 14:23:45 +03:00
gavrielc 9390037761 Merge pull request #3021 from nanocoai/whatsapp-shared-warning
Warn before connecting a shared WhatsApp number
2026-07-13 12:51:23 +03:00
Koshkoshinsk ef5d09680a fix: render WhatsApp warning title cleanly 2026-07-12 13:22:28 +03:00
Koshkoshinsk 33b33b84b9 fix: warn before shared WhatsApp setup 2026-07-12 13:05:36 +03:00
github-actions[bot] a30547fb6a docs: update token count to 232k tokens · 116% of context window 2026-07-10 20:29:39 +00:00
github-actions[bot] 27de55647b chore: bump version to 2.1.46 2026-07-10 20:29:33 +00:00
gavrielc 6cd15c6d13 Merge pull request #3010 from nanocoai/feat/channel-adapter-defaults
feat: adapter-declared channel defaults (engage mode, threading, sender policy)
2026-07-10 23:29:21 +03:00
gavrielc 40a21fe8f3 fixup: address minor review findings on channel defaults
- wirings preUpdate: treat legacy pattern rows with NULL engage_pattern
  as match-all so unrelated updates no longer reject; still rejects when
  the pattern fields themselves change invalidly (+ test)
- resolveWiringDefaults: drop a declared \b adjacent to a non-word char
  in the {name} substitution so names like 'Andy (backup)' still engage
  (+ tests; mirrors selfChatEngagePattern)
- wireApprovedChannel returns whether the wiring was created; the
  free-text interceptor now reports failure instead of a false success
- ncl create hooks log a one-line notice when a channel has no declared
  defaults and legacy static defaults apply
- stale comments/docs reconciled: chat-sdk-bridge DM thread-policy note,
  seed-discord DISCORD_DEFAULTS sync note, user-dm deliberate 'strict'
  rationale, manage-channels/update-nanoclaw/init-first-agent SKILL.md
  policy wording

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PJkDAnLGWJUqjgJNXpyqGZ
2026-07-10 23:20:32 +03:00
gavrielc 91d30713b5 fixup: restart WhatsApp service after mode/name env writes
- restartService() ran before ASSISTANT_HAS_OWN_NUMBER / ASSISTANT_NAME
  landed in .env, so the adapter (which reads both once at module load)
  kept running in shared mode with the default name until an unrelated
  restart
- single restart now sits after the env writes and the late
  dedicated→shared flip, right before init-first-agent's welcome DM
- doc header + spinner copy updated to match the new ordering

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PJkDAnLGWJUqjgJNXpyqGZ
2026-07-10 23:20:32 +03:00
gavrielc 1ad258622b PR12: document channel adapter defaults across docs and CLAUDE.md
- api-details.md: ChannelDefaults/ChannelContextDefaults interfaces, five-tier getChannelDefaults resolution chain, creation helpers, and messaging_group_agents.threads runtime semantics
- setup-wiring.md: two-level model, shared-identity mention-suppression pattern (reusable by iMessage/Signal), and the trunk-update back-compat contract with the one deliberate isGroup exception
- isolation-model.md: cross-reference from the entity model's engage columns to the defaults docs
- CLAUDE.md: channel-defaults.ts key-file row and a short Channel defaults paragraph

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PJkDAnLGWJUqjgJNXpyqGZ
2026-07-10 23:20:32 +03:00
gavrielc e370cae4e7 PR11: migrate skills to ncl-based wiring and channel-defaults docs
- Replace raw wiring/mg SQL with ncl invocations across eight skill artifacts (add-wechat wire-dm.ts rewritten to shell out to ncl; add-github/add-linear/add-signal/add-deltachat/add-emacs/migrate-from-openclaw snippets), each noting the running-host prerequisite and fixing the invalid 'isolated' session mode
- add-whatsapp: new "Dedicated vs personal number" section (ASSISTANT_HAS_OWN_NUMBER semantics, absent=shared inference), update path for flag-unset installs, spam-era group-wiring migration audit, stale pending_channel_approvals cleanup, stale-adapter skew troubleshooting
- manage-channels: two-level defaults model, --threads override + sticky/threads coercion, mention-capability constraints and {name}-pattern rename caveat, legacy WhatsApp group check
- update-nanoclaw: release notes for the two adapter-update behavior changes (Slack/Discord DM replies top-level, shared-identity stranger cards gone) + /add-whatsapp re-run pointer

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PJkDAnLGWJUqjgJNXpyqGZ
2026-07-10 23:20:32 +03:00
gavrielc e40d43a43b PR10: steer WhatsApp setup by number ownership with shared-mode interception
- askNumberOwnership is now the first prompt: dedicated number (recommended),
  personal number, or back to channel selection; the personal path shows an
  interception screen (self-chat-only consequences + Telegram / dedicated
  number / /add-whatsapp-cloud alternatives) defaulting to '← Pick a
  different channel' via the existing BACK_TO_CHANNEL_SELECTION sentinel
- dedicated path keeps askChatPhone reframed around the operator's personal
  number; entering the bot's own number routes through the same interception
  screen and flips to shared mode
- shared path skips askChatPhone (chat number = bot number), optionally asks
  for an @<AgentName>-only self-chat engage pattern passed to
  init-first-agent --engage-pattern, and ends with self-chat-only completion
  copy pointing at /manage-channels
- writeAssistantHasOwnNumber generalized to writeEnvVar(key, value); both
  modes persist ASSISTANT_NAME (fixes the 'Andy' outbound-prefix mismatch)
  and ASSISTANT_HAS_OWN_NUMBER so mode switches on re-run don't go stale;
  WhatsApp option in setup:auto gains a 'best with a dedicated number' hint

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PJkDAnLGWJUqjgJNXpyqGZ
2026-07-10 23:19:50 +03:00
gavrielc 642c5ecd37 PR6: deprecate ASSISTANT_NAME/ASSISTANT_HAS_OWN_NUMBER re-exports
- Mark both config.ts exports @deprecated; WhatsApp adapter copies read the .env keys directly now
- Re-exports retained one release so stale adapter copies (origin/channels whatsapp.ts:42) keep compiling
- No value or readEnvFile changes

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PJkDAnLGWJUqjgJNXpyqGZ
2026-07-10 23:19:50 +03:00
gavrielc 0bb9016e94 PR5: prettier reflow of PR3 resolveThreadPolicy call in router.ts
- formatting-only: the pre-commit prettier hook reflowed the long
  resolveThreadPolicy argument list left by the PR3 slice
- no behavior change

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PJkDAnLGWJUqjgJNXpyqGZ
2026-07-10 23:19:50 +03:00
gavrielc f797024770 PR5: migrate offline creation paths to declared channel defaults
- setup/register.ts: import the channels barrel (registration-only, no
  factory invocation), require --channel explicitly, resolve wiring engage
  defaults and unknown_sender_policy through the declaration helpers gated
  on hasDeclaredChannelDefaults (legacy heuristic/'strict' for undeclared
  adapters), add --is-group / --engage-mode / --unknown-sender-policy flags
  with --trigger kept as the explicit pattern override
- scripts/init-first-agent.ts: same barrel import + helper gating for the
  DM wiring and messaging-group policy; new optional --engage-pattern flag;
  sender_scope/'drop' hardcodes kept as deliberate owner-bootstrap choices
- scripts/init-cli-agent.ts: mg + wiring values now come from the cli
  adapter's declaration (value-identical to the old hardcodes)
- move validateEngageAgainstChannel + EngageValues from
  src/cli/resources/wirings.ts to src/channels/channel-defaults.ts so ncl
  and the setup register step share one mention/pattern/sticky validator

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PJkDAnLGWJUqjgJNXpyqGZ
2026-07-10 23:19:50 +03:00
gavrielc bea2dab171 PR4: wire card-approved channels through declared channel defaults
- Factor both approve paths (connect button, new-agent name reply) into one
  wireApprovedChannel helper; engage defaults now come from
  resolveWiringDefaults with isGroup = event.message.isGroup ?? mg.is_group,
  never threadId — the one deliberate behavior change: groups on
  non-threaded platforms wire via the group default instead of pattern '.'
- Fix channel-approval adapter resolution to mg.instance ?? channel_type
  (reviewer correction 5) and enrich the approval card with the resolved
  engage rule ('will respond to @-mentions in this group' / 'all messages')
- sender_scope 'known' / accumulate / shared / priority 0 stay hardcoded as
  deliberate card-flow choices, not channel defaults
- Tests: non-threaded group wires group default (sticky→mention through the
  faithful fallback), undeclared DM stays pattern '.', both approve paths
  produce identical wirings; telegram fixture declared at registration tier

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PJkDAnLGWJUqjgJNXpyqGZ
2026-07-10 23:19:50 +03:00
gavrielc 53ffa3d347 PR3: apply per-wiring thread policy and declared auto-create defaults in router
- Auto-created messaging_groups take unknown_sender_policy from the channel
  declaration (DM vs group context); undeclared adapters resolve through the
  behavior-faithful fallback, identical to the old hardcoded value
- Fanout computes each wiring's effectiveThreadId via resolveThreadPolicy
  (threads override, declaration, live capability) and threads it through
  sticky engage lookup, session-mode force, resolveSession, delivery address,
  typing, and the subscribe gate; event.replyTo is operator intent and never
  policy-stripped
- Capability pre-strip for non-threaded adapters stays unchanged
- Tests: NULL-threads inherit parity, threads=0 opt-out, replyTo exemption,
  declared vs fallback auto-create policy; metadata-preservation tests now
  register a live threaded adapter (the realistic config the router sees)

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PJkDAnLGWJUqjgJNXpyqGZ
2026-07-10 23:19:49 +03:00
gavrielc 8d39368352 PR2: add wiring threads override migration + declaration-aware ncl create/update
- Migration 019: nullable messaging_group_agents.threads (NULL = inherit
  adapter declaration, 1/0 = per-wiring override), no backfill; schema.ts
  reference DDL and MessagingGroupAgent type updated to match.
- crud.ts genericCreate restructured into two passes with a new
  ResourceDef.resolveDefaults hook between explicit-arg collection and
  static col.default, so static defaults never pre-empt context-aware
  resolution; new preUpdate hook gives update the same cross-checks.
- wirings resource: new --threads and --priority flags, engage defaults
  filled from the channel declaration ({name} substituted), validation on
  create AND update (pattern-needs-pattern, mentions:'never' rejection,
  sticky->mention coercion when the effective thread policy is off).
- messaging-groups resource fills unknown_sender_policy from the channel
  declaration; hasDeclaredChannelDefaults gates both hooks so stale
  (undeclared) adapters keep the legacy static defaults exactly.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PJkDAnLGWJUqjgJNXpyqGZ
2026-07-10 23:19:49 +03:00
gavrielc 59fc8e3cc7 PR1: add channel default declarations + tiered resolution foundation
- ChannelContextDefaults/ChannelDefaults types on ChannelAdapter, ChannelRegistration, and ChatSdkBridgeConfig (bridge copies verbatim like supportsThreads)
- getChannelDefaults tiered lookup (live instance -> live channelType -> registration key -> registration channelType -> fallback) with behavior-faithful fallbackChannelDefaults(supportsThreads)
- new channel-defaults.ts helpers: resolveWiringDefaults ({name} substitution, sticky->mention coercion, pattern validation), resolveUnknownSenderPolicy, resolveThreadPolicy (hard-AND with capability)
- CLI adapter declares its defaults; fix stale isMention doc claiming a router text-match fallback

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PJkDAnLGWJUqjgJNXpyqGZ
2026-07-10 23:19:49 +03:00
github-actions[bot] 45f656f21c chore: bump version to 2.1.45 2026-07-10 20:03:08 +00:00
gavrielc a4513f292f Merge pull request #3009 from nanocoai/whatsapp-formatting-to-channels
Move channel formatting skills (whatsapp, slack) from trunk to the channels branch
2026-07-10 23:02:40 +03:00
gavrielc 38eb5a44df docs: scope the formatting-skills migration to installs with the channel wired
/update-nanoclaw is agent-driven; an unscoped 're-run /add-whatsapp'
would install the whole Baileys adapter on channel-free installs.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01A2YZQDTw9TQrH3m8NBtVAW
2026-07-10 23:00:57 +03:00
gavrielc 6d397fc116 docs: tighten the channel formatting skills changelog entry
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01A2YZQDTw9TQrH3m8NBtVAW
2026-07-10 22:57:28 +03:00
gavrielc cb2bbcc7aa feat!: move slack-formatting container skill to the channels branch
Same treatment as whatsapp-formatting (previous commit): slack-formatting
is channel payload, not trunk. It ships only a SKILL.md (no composed-doc
fragment) and reaches agents via ~/.claude/skills; /add-slack and the
slack setup installers now copy it from the channels branch, which
already carries an identical copy. Changelog entry combined for both.

BREAKING CHANGE: existing Slack installs lose the slack-formatting skill
from ~/.claude/skills after updating; re-run /add-slack (idempotent).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01A2YZQDTw9TQrH3m8NBtVAW
2026-07-10 22:45:23 +03:00
gavrielc b9998a4bf1 feat!: move whatsapp-formatting container skill to the channels branch
Trunk shipped WhatsApp formatting instructions to every install — the
skill-whatsapp-formatting.md fragment landed in every agent's composed
CLAUDE.md whether or not WhatsApp was wired. The skill is channel payload:
it now lives on the channels branch and is copied in by /add-whatsapp and
the setup installers (install-whatsapp.sh, add-whatsapp.sh).

BREAKING CHANGE: existing WhatsApp installs lose the fragment on their
next container spawn after updating; re-run /add-whatsapp (idempotent)
to restore it. See CHANGELOG.md for the manual copy alternative.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01A2YZQDTw9TQrH3m8NBtVAW
2026-07-10 22:43:06 +03:00
github-actions[bot] 0d41f0ef94 chore: bump version to 2.1.44 2026-07-10 18:22:21 +00:00
gavrielc 3363a0be0f Merge pull request #3003 from Shufel83/fix/agent-browser-bounded-wait
docs(agent-browser): require bounded waits for custom conditions
2026-07-10 21:22:08 +03:00
gavrielc 05cf44035e Merge branch 'main' into fix/agent-browser-bounded-wait 2026-07-10 21:21:34 +03:00
github-actions[bot] c1eb1079cc docs: update token count to 225k tokens · 113% of context window 2026-07-10 18:20:35 +00:00
gavrielc 1725d86fbd Merge pull request #3006 from nanocoai/fix/local-time-display
fix: ISO storage + local-time display for all timestamps
2026-07-10 21:20:21 +03:00
gavrielc e3bd22f96f style: prettier
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01A2YZQDTw9TQrH3m8NBtVAW
2026-07-10 21:11:38 +03:00
gavrielc c2e5b14bf8 fix: ISO storage + local-time display for all timestamps
Storage: every JS-side datetime('now') write becomes new Date().toISOString()
(messages_out all writers, processing_ack all stamps, delivered, retry
backoff, roles/members/destinations, skill SQL via strftime ISO-Z). Dormant
deliver_after raw string compares wrapped in datetime().

Display: ncl human output renders ISO instants as local "YYYY-MM-DD HH:mm"
(host + container renderers; --json stays ISO; the stamp shape round-trips
through parseZonedToUtc). Task run logs, conversation archive filenames,
clidash, the dashboard pusher, and the uninstall backup stamp all render
local; instruction texts updated in lockstep. CLAUDE.md documents the
convention.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01A2YZQDTw9TQrH3m8NBtVAW
2026-07-10 21:06:33 +03:00
github-actions[bot] 1ccef326dc chore: bump version to 2.1.43 2026-07-10 17:17:24 +00:00
gavrielc 39e7604526 Merge pull request #3005 from nanocoai/fix/task-timestamp-iso
fix: stamp task rows with ISO timestamps
2026-07-10 20:17:10 +03:00
gavrielc c0e591f2fb fix: stamp task rows with ISO timestamps
insertTaskRow used SQL datetime('now') — a naive-UTC stamp the container
formatter parses as local time, rendering <task time> hours off from the
ISO-Z chat timestamps beside it. Stamp ISO like every other messages_in
writer.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01A2YZQDTw9TQrH3m8NBtVAW
2026-07-10 19:54:35 +03:00
exe.dev user df4929d61d docs(agent-browser): require bounded waits for custom conditions
Improvised `until … do sleep; done` polling loops with no timeout can
loop forever when a page condition never becomes true (page fails to
load, selector changes, network stalls). This doesn't just fail the
command — it wedges the whole agent turn: the runner keeps the model
stream open, later messages are pushed in and silently marked complete
without a reply, and heartbeat freshness masks it from the host's
stuck-detection, so a session can hang for hours.

Add a "Waiting for a custom condition — ALWAYS bound it" section to the
agent-browser skill: prefer built-in `wait` subcommands, ban unbounded
loops, and give a ready pattern capped by both `timeout` and a
max-attempts counter that always exits. Treat a timeout as a real
failure (snapshot + report), not grounds to re-enter the wait.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-10 13:11:34 +00:00
github-actions[bot] 14f528c895 docs: update token count to 224k tokens · 112% of context window 2026-07-10 06:42:50 +00:00
github-actions[bot] 366c5f06f1 chore: bump version to 2.1.42 2026-07-10 06:42:46 +00:00
glifocat 077ce6fd9d Merge pull request #2416 from glifocat/fix/ncl-create-handlers
fix(cli): provision companion rows on `ncl groups create` and `ncl wirings create`
2026-07-10 08:42:32 +02:00
github-actions[bot] 0c0f4c2592 docs: update token count to 223k tokens · 111% of context window 2026-07-09 08:24:27 +00:00
github-actions[bot] 92635f1934 chore: bump version to 2.1.41 2026-07-09 08:24:22 +00:00
gavrielc f7a43ef881 Merge pull request #2981 from nanocoai/tasks/02-ncl-tasks-core
Scheduled tasks: ncl tasks control plane, isolated sessions, script gate
2026-07-09 11:24:07 +03:00
omri-maya 5459925c65 Apply suggestions from code review
Co-authored-by: omri-maya <omri@nanoco.ai>
2026-07-09 11:23:08 +03:00
Omri Maya b0c76ce4c5 feat(tasks): ncl tasks control plane — isolated per-series sessions, strict verb contract, script gate
Scheduled tasks move from MCP tools to a first-class ncl resource:
list / get / create / update / cancel / pause / resume / run / append-log,
with args declared on every verb (strict validation, fix-carrying errors)
and deep per-verb help with executable examples.

A series is CronJob-like: the live (pending/paused) row is the next
occurrence, completed rows are its run history. tasks list reads as a
compact run-history table, server-rendered so the container agent prints
the same aligned table. Short named ids (<slug>-<hex>) are
filesystem/thread-safe and copy-pasteable. Agents keep a per-series work
journal: the run log at tasks/<series>.md (append-log + auto-logged
final text).

Each series runs in its own isolated system session
(system:tasks:<series>); the live-task cap is dropped — isolation
replaces throttling. Spent task sessions are GC'd by the sweep.

--script on a task runs a bash gate BEFORE the agent wakes. It prints a
JSON verdict as its last stdout line: {"wakeAgent": bool, "data": {...}}.
wakeAgent:false handles the occurrence without waking the agent; data is
threaded into the fire's prompt, so a gate can fetch/inspect and hand the
agent exactly what changed. Failing scripts back off exponentially
(2·2^(n-1) min, cap 60) and auto-pause the series after 8 consecutive
failures with a host-written run-log note; manual runs are excluded from
the streak.

BREAKING: the schedule_task / list_tasks / update_task / cancel_task /
pause_task / resume_task MCP tools are removed. Agents schedule via
ncl tasks (available under cli_scope=group). Existing task rows keep
firing — storage (messages_in kind='task') is unchanged; only the
management surface moved. Migration: docs/ncl-tasks-migration.md.
Review round (gavrielc):
- Recurrence frequency guard: create/update refuse a recurrence more
  frequent than 4 fires/day with a warning steering to gate scripts;
  --dangerously-override-recurrence-limit bypasses after explicit user
  confirmation. Counted over the next 24h in the instance TZ. Guarded on
  update too, so create-slow-then-update cannot sneak past.
- ncl tasks help scripts: resource help topics (new helpTopics seam on
  registerResource) carry the full gate-script guide — contract, examples,
  state, failure/backoff semantics, testing directions.
- Agent-facing scheduling instructions slimmed: no recurring-frequency
  prose in the fragment; one pointer to the help topic.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-08 17:04:45 +03:00
Gabi Simons 87a2d3af8d Merge pull request #2978 from nanocoai/feat/core-team-pr-label
ci: auto-label PRs from core team members
2026-07-08 14:49:23 +03:00
Gabi Simons 68ebc02114 Update CONTRIBUTING.md 2026-07-08 14:48:25 +03:00
Gabi Simons d4e73c5523 Merge branch 'main' into feat/core-team-pr-label 2026-07-08 14:35:23 +03:00
github-actions[bot] 9a103f4f93 docs: update token count to 213k tokens · 106% of context window 2026-07-08 11:29:45 +00:00
github-actions[bot] a8554c6248 chore: bump version to 2.1.40 2026-07-08 11:29:40 +00:00
gavrielc 2480ae7cb8 Merge pull request #2980 from nanocoai/tasks/01-cli-verb-args-human-view
ncl CLI: verb-level args, deep help, server-rendered human view
2026-07-08 14:29:24 +03:00
Gabi Simons 8943c1cbf4 ci: auto-label PRs from core team members
Extends the existing label-pr workflow with an author allowlist that
applies a core-team label. The label is auto-provisioned on first use
so no manual repo setup is needed.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-08 10:11:43 +03:00
glifocat 1985fd6a5e fix(cli): live-refresh destinations on ncl wirings create
Closes the restart-required behavior: wirings-create's postCreate wrote
the central agent_destinations row but never projected it into running
sessions' inbound.db, so an agent with a live container dropped every
reply to the newly-wired chat as "unknown destination" until a group
restart (observed on a live instance). `ncl destinations add` already
projects for exactly this reason; wirings create now has parity.

postCreate is sync and runs inside the central-DB transaction, so it's
the wrong place for an async, cross-DB (session inbound.db) side effect.

- crud.ts: add a postCommit hook that runs after the create transaction
  commits; correct the "pure DB writes" comment.
- resources/destinations.ts: export projectDestinationsToSessions so
  wirings.ts reuses the exact same projection as destinations add/remove.
- resources/wirings.ts: project the new destination to live sessions in
  postCommit.
- crud.test.ts: cover projection with and without a running session.
2026-07-04 01:44:12 +00:00
glifocat 0820d39424 fix(cli): wrap INSERT + postCreate in db.transaction()
genericCreate previously ran the parent INSERT and the postCreate hook
as two independent statements. If postCreate throws, the parent row is
left behind without its companion — the exact partial-state class this
PR exists to close (#2415, #2389).

Wrap both in a single better-sqlite3 db.transaction() so a postCreate
throw rolls the whole thing back. postCreate is typed `(row) => void`
and the two registered hooks (container_configs insert on groups,
agent_destinations insert on wirings) are pure sync DB writes, so the
sync transaction wrapper is the right shape.
2026-07-04 01:42:29 +00:00
glifocat c752f9c065 test(cli): cover postCreate hook for groups/wirings create
Locks down both regressions: `groups-create` must result in a
`container_configs` row, `wirings-create` must result in an
`agent_destinations` row. Without these, a future refactor of
`genericCreate` that drops the `postCreate` call would silently reopen
issues #2415 and #2389.

Also drops the speculative `Promise<void>` return type on `postCreate`.
The current side effects are sync, and keeping it sync means a future
better-sqlite3 transaction wrapper around INSERT + postCreate stays
viable. If we ever need async, narrow → wide is a cheap change.
2026-07-04 01:42:29 +00:00
glifocat 0e73e4a782 fix(cli): provision companion rows on ncl groups create and ncl wirings create
`ncl groups create` only inserted into `agent_groups`, leaving no
`container_configs` row — so the first spawn for that group threw
"Container config not found" until the next host restart kicked in the
backfill.

`ncl wirings create` only inserted into `messaging_group_agents`, leaving
no companion `agent_destinations` row — so the agent could receive
messages on the wired channel but its replies got silently dropped by the
delivery ACL.

Both bugs share the same shape: the resources are declared in
`src/cli/resources/*.ts` with `create: 'approval'` and no custom
handler, so creation falls through to `genericCreate` in
`src/cli/crud.ts`, which is a raw INSERT. The domain functions
(`initGroupFilesystem`, `createMessagingGroupAgent`) wire the companion
rows, but the generic path skips them.

This adds a `postCreate` hook to `ResourceDef`. `genericCreate` calls it
after the INSERT with the values that were written, so generated fields
(`id`, `created_at`) are populated. `groups.ts` uses it to call
`initGroupFilesystem`; `wirings.ts` uses it to create the destination row.

The destination logic was extracted from `createMessagingGroupAgent`
into a new exported `ensureAgentDestinationForWiring`, so the wirings
hook doesn't have to re-INSERT the wiring row to get the destination
side effect.

Closes #2415
Closes #2389
2026-07-04 01:42:29 +00:00
167 changed files with 9740 additions and 1663 deletions
+52
View File
@@ -0,0 +1,52 @@
# Remove /add-audit
Reverses every change the skill made. Safe to re-run even if some pieces are
already gone. Run from the NanoClaw project root.
## 1. Delete the copied files
```bash
rm -rf src/audit
rm -f src/cli/dispatch.audit.ts src/cli/dispatch.audit.test.ts
rm -f src/cli/resources/audit.ts
rm -f src/audit-wiring.test.ts
```
## 2. Revert the dispatch composition
In `src/cli/dispatch.ts`, delete (not comment out) the three edits the skill
made:
1. The import line: `import { withAudit } from './dispatch.audit.js';`
2. The composition block (comment + `export const dispatch = withAudit(dispatchInner);`)
3. Rename the dispatcher back — change `async function dispatchInner(` to
`export async function dispatch(`
## 3. Unregister the resource
Delete the `import './audit.js';` line from `src/cli/resources/index.ts`.
## 4. Remove the settings
Delete the `AUDIT_ENABLED` and `AUDIT_RETENTION_DAYS` lines from `.env`.
## 5. Rebuild and restart
```bash
pnpm run build
source setup/lib/install-slug.sh
launchctl kickstart -k gui/$(id -u)/$(launchd_label) # macOS
# systemctl --user restart $(systemd_unit) # Linux
```
## Day-files
`data/audit/*.ndjson` are the operator's records and are deliberately left in
place. To purge them too:
```bash
rm -rf data/audit
```
No dependency was added, nothing under `container/` was touched, and no DB
schema changed — there is nothing else to undo.
+219
View File
@@ -0,0 +1,219 @@
---
name: add-audit
description: Add an opt-in local audit log for the ncl command surface — every dispatch outcome on both transports (host socket and container), including scope denials, approval holds, and approved replays, written as SIEM-shaped append-only NDJSON day-files under data/audit/. Read back with `ncl audit list`; plug exporters in via registerAuditHook. Off until AUDIT_ENABLED=true.
---
# /add-audit — Local Audit Log (ncl surface)
Records one canonical audit event for every command that reaches the `ncl`
dispatcher — human over the host socket or agent over the container transport
— including denials. Both transports converge on the exported `dispatch`, so
one composition covers the whole surface: there is no second door.
```
ncl (host socket) ──┐
├─→ dispatch = withAudit(dispatchInner) ─→ one NDJSON line
container cli_request ┘ │ data/audit/<UTC-day>.ndjson
approved replay (grant) ───────┘
```
What one event carries: `actor` (`host:<os-user>` or the agent group),
`origin` (transport, session, channel), dotted `action` from the guard
catalog (e.g. `groups.config.add-mcp-server`), touched/attempted `resources`,
`outcome` (`success · failure · denied · pending · approved`), `correlation_id`
(the approval id on gated chains — the pending event and the approved replay
share it, so `--correlation <id>` returns the whole chain), and `details` (the
flag names passed, plus a small allowlist of safe enum values — never raw
argument values).
Scope of this increment: the ncl dispatch surface only. Approval-lifecycle
events (request/decision as their own events), OneCLI credential holds, and
channel/sender events are later increments — they attach to `src/audit/` as
pure adds (new `*.audit.ts` adapters and observer subscriptions).
## Steps
### 0. Pre-flight (idempotency)
The apply is safe to re-run; every step below is guarded. Skip to
**Enable** if all of these already hold:
- `src/audit/` exists
- `src/cli/resources/index.ts` contains `import './audit.js';`
- `src/cli/dispatch.ts` contains `export const dispatch = withAudit(dispatchInner);`
Before editing, verify the reach-in targets still exist: `src/cli/dispatch.ts`
must contain `export async function dispatch(` (or the already-applied
composition), and `src/cli/resources/index.ts` must be the resource barrel (a
list of `import './<resource>.js';` lines). If either has moved, stop and
adapt rather than guessing.
### 1. Copy the payload
From the NanoClaw project root:
```bash
cp -R "${CLAUDE_SKILL_DIR}/add/src/." src/
```
What lands (mirrors of the destination paths):
- `src/audit/` — the domain-free leaf: event schema (`types.ts`), env config
(`config.ts`), redactor, NDJSON day-file store, emit seam, post-write hooks,
reader, boot/maintenance wiring, vocabulary — plus its tests.
- `src/cli/dispatch.audit.ts` — the CLI adapter: `withAudit` middleware and
the actor/origin/resource mapping (+ `dispatch.audit.test.ts`).
- `src/cli/resources/audit.ts` — the read-only `ncl audit` resource.
- `src/audit-wiring.test.ts` — goes red if either core edit below is deleted
or drifts.
### 2. Register the resource
Append to `src/cli/resources/index.ts` (skip if the line is already present):
```typescript
import './audit.js';
```
### 3. Compose the dispatch middleware
This is the skill's one functional reach-in, in `src/cli/dispatch.ts`. Three
small edits:
1. Add the import (next to the other `./` imports):
```typescript
import { withAudit } from './dispatch.audit.js';
```
2. Rename the dispatcher declaration — change
```typescript
export async function dispatch(
```
to
```typescript
async function dispatchInner(
```
3. Directly after that function's closing brace (before the
`registerApprovalHandler('cli_command', …)` block), add:
```typescript
// Audit middleware (installed by /add-audit): the exported dispatch is the
// wrapped function, so both transports and the approved replay below all
// pass the one composition.
export const dispatch = withAudit(dispatchInner);
```
The composition must live at the definition site (not at the import sites):
the approved-replay handler in the same file calls `dispatch(...)` too, and
only the wrapped export covers it. `src/audit-wiring.test.ts` asserts exactly
this shape via the TypeScript AST.
Loading `dispatch.audit.ts` also boots the audit log: on an enabled box it
asserts `data/audit/` is writable (refusing to start beats a silent audit
gap), runs the boot retention prune, and arms an unref'd maintenance timer.
Nothing else in core changes — no `index.ts` or `host-sweep.ts` edits.
### 4. Enable
Add the two settings to `.env` (idempotent — overwrite or append):
```bash
grep -q '^AUDIT_ENABLED=' .env && sed -i.bak 's/^AUDIT_ENABLED=.*/AUDIT_ENABLED=true/' .env && rm -f .env.bak || echo 'AUDIT_ENABLED=true' >> .env
grep -q '^AUDIT_RETENTION_DAYS=' .env || echo 'AUDIT_RETENTION_DAYS=90' >> .env
```
`AUDIT_ENABLED` is the master switch — off (or absent) means `emitAuditEvent`
is a no-op and `data/audit/` is never created. `AUDIT_RETENTION_DAYS`: day
files strictly older than the horizon are hard-deleted (unlinked) at boot and
once per UTC day; `0` = keep forever; unset = 90.
### 5. Build and test
Run `build` before the tests — it's what catches a missed copy or a drifted
import path across the whole composed tree:
```bash
pnpm run build
pnpm exec vitest run src/audit src/cli/dispatch.audit.test.ts src/audit-wiring.test.ts
pnpm test
```
### 6. Restart the service
```bash
source setup/lib/install-slug.sh
launchctl kickstart -k gui/$(id -u)/$(launchd_label) # macOS
# systemctl --user restart $(systemd_unit) # Linux
```
### 7. Verify (runtime smoke)
```bash
ncl groups list # any command — this one is now an audit event
cat data/audit/$(date -u +%F).ndjson | tail -1
ncl audit list --limit 5
```
The last line of the day-file is the `groups.list` event you just caused.
## Reading it back
```
ncl audit list [--actor <id>] [--action <name-or-prefix>] [--resource <id-or-type>]
[--outcome success|failure|denied|pending|approved]
[--since 7d|24h|30m|ISO] [--until …] [--correlation <approval-id>]
[--limit N] [--format ndjson]
```
Newest first, default limit 100. `--action groups.config` matches the whole
dotted subtree. `--format ndjson` streams the stored lines verbatim — pipe to
a file for SIEM import. On a disabled box the command errors with
`audit log is disabled — set AUDIT_ENABLED=true` rather than returning an
empty list that would read as "no actions happened".
The resource is deliberately **not** on the group-scope allowlist: audit
spans agent groups, so group-scoped agents are refused before the handler
(fails closed). Host callers and `cli_scope: global` agents only.
## Recording model, failure posture, exporters
- Events store WHO did WHICH action to WHAT target and the outcome — **never
raw argument values**. `details` carries the flag names that were passed plus
a small allowlist of governance-relevant enum fields (`role`, `mode`,
`session_mode`, `cli_scope`, `access`, `engage_mode`, `sender_scope`,
`provider`, `model`) echoed verbatim; every other value is name-only, and a
failure keeps the error *code*, never the free-text message. There is no value
redactor — a secret can't leak from a value that is never stored. Target
identifiers (ids, users, groups) surface structurally in `resources`.
- Fail-open + loud: a failed append is `log.error`'d and the audited action
proceeds — the emit brackets the dispatcher, so even a command that *throws*
still leaves a record. At boot, an enabled box refuses to start if
`data/audit/` isn't writable.
- In-process exporters register via `registerAuditHook` (from
`src/audit/index.js`) — post-write hooks that fire only after the local
append succeeds, so an external system can never be ahead of the source of
truth. Ship one as its own `/add-*` skill; declare `/add-audit` as its
dependency.
## Troubleshooting
- **Host won't start, banner names the audit directory** — `AUDIT_ENABLED=true`
but `data/audit/` isn't writable. Fix permissions or disable audit.
- **`audit log is disabled — set AUDIT_ENABLED=true`** — the read-back guard;
enable in `.env` and restart.
- **An agent gets "CLI access is scoped to this agent group" for `ncl audit`**
— by design; grant the group `cli_scope: global` only if it should read
cross-group history.
- **`pnpm test` on an enabled box adds a few events to today's day-file** —
expected: the base dispatch tests drive real dispatches. Harmless noise;
the audit test suites themselves write only to temp dirs or capture buffers.
## Removal
See [REMOVE.md](REMOVE.md) — reverses every change; existing day-files are
left in place (they're the operator's records).
@@ -0,0 +1,143 @@
/**
* Wiring tests for the /add-audit skill's two core edits — they go red if
* either edit is deleted or drifts:
*
* 1. src/cli/dispatch.ts — the exported dispatch must be the composed
* `withAudit(dispatchInner)` (AST check: only the definition-site
* composition covers both transports AND the in-module approved replay).
* 2. src/cli/resources/index.ts — the audit resource must register through
* the real barrel (behavior check), stay OFF the group-scope allowlist,
* and leave guard conformance clean.
*
* Plus the emit invariant the module's discipline rests on: emitAuditEvent
* appears only in src/audit/ and *.audit.ts adapter files.
*/
import fs from 'fs';
import path from 'path';
import ts from 'typescript';
import { describe, expect, it } from 'vitest';
// Same production barrels the guard conformance test loads (mirrors
// src/guard/conformance.test.ts).
import './cli/commands/index.js';
import './modules/index.js';
import './cli/delivery-action.js';
import './cli/dispatch.js';
import { commandGuard, GROUP_SCOPE_RESOURCES, lookup } from './cli/registry.js';
import { getApprovalHandler } from './modules/approvals/primitive.js';
import { listGuardedActions } from './guard/index.js';
describe('audit resource registration (barrel wiring)', () => {
it('registers audit-list through the real resource barrel', () => {
const cmd = lookup('audit-list');
expect(cmd).toBeDefined();
expect(cmd?.action).toBe('audit.list');
expect(cmd?.access).toBe('open');
});
it('derives a guard-catalog entry for the audit command', () => {
const guard = commandGuard('audit-list');
expect(guard.action).toBe('audit.list');
});
it('is NOT on the group-scope allowlist — group-scoped agents fail closed', () => {
expect(GROUP_SCOPE_RESOURCES.has('audit')).toBe(false);
});
it('leaves guard conformance clean with the audit resource registered', () => {
// The invariant guard conformance checks: every holding action pairs with a
// registered approve continuation. audit.list is `open` (never holds), so
// registering the resource can't introduce a gap — this pins that.
const dangling = listGuardedActions()
.filter((spec) => spec.grantActionName)
.filter((spec) => !getApprovalHandler(spec.grantActionName as string));
expect(dangling.map((s) => s.action)).toEqual([]);
});
});
describe('dispatch composition (AST wiring)', () => {
const source = fs.readFileSync(path.resolve('src/cli/dispatch.ts'), 'utf8');
const sf = ts.createSourceFile('dispatch.ts', source, ts.ScriptTarget.Latest, true);
const hasExportModifier = (node: ts.HasModifiers): boolean =>
(ts.getModifiers(node) ?? []).some((m) => m.kind === ts.SyntaxKind.ExportKeyword);
let importsWithAudit = false;
let innerDeclaredUnexported = false;
let exportsWrappedDispatch = false;
let exportsUnwrappedDispatchFn = false;
sf.forEachChild((node) => {
if (
ts.isImportDeclaration(node) &&
ts.isStringLiteral(node.moduleSpecifier) &&
node.moduleSpecifier.text === './dispatch.audit.js'
) {
const named = node.importClause?.namedBindings;
if (named && ts.isNamedImports(named) && named.elements.some((e) => e.name.text === 'withAudit')) {
importsWithAudit = true;
}
}
if (ts.isFunctionDeclaration(node) && node.name?.text === 'dispatchInner') {
innerDeclaredUnexported = !hasExportModifier(node);
}
if (ts.isFunctionDeclaration(node) && node.name?.text === 'dispatch' && hasExportModifier(node)) {
exportsUnwrappedDispatchFn = true;
}
if (ts.isVariableStatement(node) && hasExportModifier(node)) {
for (const decl of node.declarationList.declarations) {
if (
ts.isIdentifier(decl.name) &&
decl.name.text === 'dispatch' &&
decl.initializer &&
ts.isCallExpression(decl.initializer) &&
ts.isIdentifier(decl.initializer.expression) &&
decl.initializer.expression.text === 'withAudit' &&
decl.initializer.arguments.length === 1 &&
ts.isIdentifier(decl.initializer.arguments[0]) &&
decl.initializer.arguments[0].text === 'dispatchInner'
) {
exportsWrappedDispatch = true;
}
}
}
});
it('imports withAudit from the audit adapter', () => {
expect(importsWithAudit).toBe(true);
});
it('keeps the inner dispatcher unexported (the guarded path is the only path)', () => {
expect(innerDeclaredUnexported).toBe(true);
});
it('exports dispatch as withAudit(dispatchInner)', () => {
expect(exportsWrappedDispatch).toBe(true);
expect(exportsUnwrappedDispatchFn).toBe(false);
});
});
describe('emit invariant', () => {
it('emitAuditEvent appears only in src/audit/ and *.audit.ts files', () => {
const srcRoot = path.resolve('src');
const offenders: string[] = [];
const walk = (dir: string): void => {
for (const entry of fs.readdirSync(dir, { withFileTypes: true })) {
const full = path.join(dir, entry.name);
if (entry.isDirectory()) {
walk(full);
continue;
}
if (!entry.name.endsWith('.ts')) continue;
const rel = path.relative(srcRoot, full).split(path.sep).join('/');
if (rel === 'audit-wiring.test.ts') continue; // this file names the symbol
if (rel.startsWith('audit/')) continue;
if (/\.audit(\.test)?\.ts$/.test(rel)) continue;
if (fs.readFileSync(full, 'utf8').includes('emitAuditEvent')) offenders.push(rel);
}
};
walk(srcRoot);
expect(offenders).toEqual([]);
});
});
@@ -0,0 +1,30 @@
/**
* Audit env config (installed by /add-audit) — the two audit vars, read the
* same way src/config.ts reads every host setting (readEnvFile from .env,
* with process.env taking precedence). Kept audit-owned so installing the
* skill never edits core config.ts: the feature's whole footprint in core is
* the dispatch composition in dispatch.ts and the resource-barrel import.
*/
import { readEnvFile } from '../env.js';
const envConfig = readEnvFile(['AUDIT_ENABLED', 'AUDIT_RETENTION_DAYS']);
/**
* Master switch. Off by default — nothing is persisted (and data/audit/ is
* never created) until an operator sets AUDIT_ENABLED=true.
*/
export const AUDIT_ENABLED = (process.env.AUDIT_ENABLED || envConfig.AUDIT_ENABLED) === 'true';
/**
* Day-file retention horizon in days; consulted only when audit is enabled.
* Unset or unparseable → 90. An explicit 0 (or negative) = keep forever.
*/
function parseRetentionDays(raw: string | undefined): number {
if (raw === undefined || raw.trim() === '') return 90;
const n = Number.parseInt(raw, 10);
return Number.isNaN(n) ? 90 : n;
}
export const AUDIT_RETENTION_DAYS = parseRetentionDays(
process.env.AUDIT_RETENTION_DAYS || envConfig.AUDIT_RETENTION_DAYS,
);
@@ -0,0 +1,45 @@
/**
* The single emit seam. Adapters import emitAuditEvent from here directly
* (it is deliberately not re-exported by the barrel): only src/audit/ and
* `*.audit.ts` adapter files may call it — grep holds the invariant.
*
* The opt-in check lives here, so the whole feature switches at one point.
* Fail-open + loud: a failed append (or a throwing input thunk) is
* log.error'd and the audited action proceeds.
*/
import { randomUUID } from 'crypto';
import { log } from '../log.js';
import { AUDIT_ENABLED } from './config.js';
import { notifyAuditHooks } from './hooks.js';
import { appendAuditLine } from './store.js';
import type { AuditEvent, AuditEventInput } from './types.js';
export function emitAuditEvent(input: AuditEventInput | (() => AuditEventInput)): void {
if (!AUDIT_ENABLED) return; // The one opt-in check — the whole feature switches here.
try {
if (typeof input === 'function') input = input();
const event: AuditEvent = {
event_id: randomUUID(),
time: new Date().toISOString(),
schema_version: 1,
actor: { ...input.actor, email: null, user_id: null, group_ids: null },
origin: input.origin,
action: input.action,
resources: input.resources,
outcome: input.outcome,
correlation_id: input.correlationId ?? null,
// Adapters build `details` value-free (key names + an enum allowlist), so
// there is nothing to redact here — a secret can't leak from a value that
// is never stored. Written verbatim.
details: input.details ?? {},
};
const line = JSON.stringify(event);
appendAuditLine(line, event.time.slice(0, 10));
notifyAuditHooks(event, line);
// eslint-disable-next-line no-catch-all/no-catch-all -- fail-open is the posture: an audit failure must never take down the audited action
} catch (err) {
const action = typeof input === 'function' ? undefined : input.action;
log.error('Audit append failed — action proceeding (fail-open)', { action, err });
}
}
@@ -0,0 +1,240 @@
/**
* Post-write hook contract: hooks observe the LOG (fire only after a
* successful append, exported ⊆ written), failures are isolated everywhere,
* and the lifecycle (init/maintain/shutdown) behaves.
*/
import { afterEach, beforeEach, describe, expect, it, vi } from 'vitest';
const state = vi.hoisted(() => ({ enabled: true, appendThrows: false, appended: [] as string[] }));
vi.mock('../config.js', async (importOriginal) => {
const actual = await importOriginal<typeof import('../config.js')>();
return {
...actual,
DATA_DIR: '/tmp/nanoclaw-test-hooks-unused',
};
});
vi.mock('./config.js', () => ({
get AUDIT_ENABLED() {
return state.enabled;
},
AUDIT_RETENTION_DAYS: 90,
}));
vi.mock('./store.js', async (importOriginal) => {
const actual = await importOriginal<typeof import('./store.js')>();
return {
...actual,
appendAuditLine: (line: string) => {
if (state.appendThrows) throw new Error('disk full');
state.appended.push(line);
},
};
});
vi.mock('../log.js', () => ({
log: { debug: vi.fn(), info: vi.fn(), warn: vi.fn(), error: vi.fn(), fatal: vi.fn() },
}));
let hooks: typeof import('./hooks.js');
let emit: typeof import('./emit.js');
let log: (typeof import('../log.js'))['log'];
beforeEach(async () => {
state.enabled = true;
state.appendThrows = false;
state.appended.length = 0;
vi.resetModules(); // fresh hook registry per test
hooks = await import('./hooks.js');
emit = await import('./emit.js');
log = (await import('../log.js')).log;
vi.clearAllMocks();
});
afterEach(() => {
vi.clearAllMocks();
});
const EVENT_INPUT = {
actor: { type: 'human' as const, id: 'host:test' },
origin: { transport: 'socket' as const },
action: 'groups.list',
resources: [{ type: 'agent_group' }],
outcome: 'success' as const,
details: { limit: 5 },
};
describe('post-write notification', () => {
it('calls a registered hook with the parsed event and the exact stored line', () => {
const seen: Array<{ event: import('./types.js').AuditEvent; line: string }> = [];
hooks.registerAuditHook({ name: 'demo', onEvent: (event, line) => seen.push({ event, line }) });
emit.emitAuditEvent(EVENT_INPUT);
expect(state.appended).toHaveLength(1);
expect(seen).toHaveLength(1);
expect(seen[0].line).toBe(state.appended[0]);
expect(JSON.parse(seen[0].line)).toEqual(seen[0].event);
expect(seen[0].event.action).toBe('groups.list');
});
it('does NOT call hooks when the local append fails — exported ⊆ written', () => {
const onEvent = vi.fn();
hooks.registerAuditHook({ name: 'demo', onEvent });
state.appendThrows = true;
expect(() => emit.emitAuditEvent(EVENT_INPUT)).not.toThrow(); // action still proceeds
expect(onEvent).not.toHaveBeenCalled();
expect(log.error).toHaveBeenCalledWith(expect.stringContaining('Audit append failed'), expect.anything());
});
it('does NOT call hooks when audit is disabled', () => {
const onEvent = vi.fn();
hooks.registerAuditHook({ name: 'demo', onEvent });
state.enabled = false;
emit.emitAuditEvent(EVENT_INPUT);
expect(state.appended).toHaveLength(0);
expect(onEvent).not.toHaveBeenCalled();
});
it('isolates a throwing hook: the write survives, later hooks still run, the action proceeds', () => {
const second = vi.fn();
hooks.registerAuditHook({
name: 'broken',
onEvent: () => {
throw new Error('exporter exploded');
},
});
hooks.registerAuditHook({ name: 'healthy', onEvent: second });
expect(() => emit.emitAuditEvent(EVENT_INPUT)).not.toThrow();
expect(state.appended).toHaveLength(1); // the log has the event regardless
expect(second).toHaveBeenCalledTimes(1);
expect(log.error).toHaveBeenCalledWith(
expect.stringContaining('Audit hook threw'),
expect.objectContaining({ hook: 'broken', action: 'groups.list' }),
);
});
});
describe('lifecycle', () => {
it('initAuditHooks surfaces a failing init as a fatal error naming the hook', () => {
hooks.registerAuditHook({ name: 'ok', onEvent: () => {}, init: vi.fn() });
hooks.registerAuditHook({
name: 'bad-boot',
onEvent: () => {},
init: () => {
throw new Error('no route to collector');
},
});
expect(() => hooks.initAuditHooks()).toThrow(/audit hook "bad-boot" failed to initialize.*no route/);
});
it('defers init() until boot for a hook registered before initAuditHooks', () => {
const init = vi.fn();
hooks.registerAuditHook({ name: 'early', onEvent: () => {}, init });
expect(init).not.toHaveBeenCalled(); // not yet — boot hasn't run
hooks.initAuditHooks();
expect(init).toHaveBeenCalledTimes(1); // exactly once, at boot
});
it('runs init() immediately for a hook registered after boot — import-order-insensitive', () => {
hooks.initAuditHooks(); // boot completes (no hooks yet)
const init = vi.fn();
const onEvent = vi.fn();
hooks.registerAuditHook({ name: 'late', onEvent, init });
// A module that loaded after the CLI adapter still gets its one-time init...
expect(init).toHaveBeenCalledTimes(1);
// ...and still receives events (onEvent already read the live array).
emit.emitAuditEvent(EVENT_INPUT);
expect(onEvent).toHaveBeenCalledTimes(1);
});
it('a hook registered after boot whose init throws is fatal, naming the hook', () => {
hooks.initAuditHooks();
expect(() =>
hooks.registerAuditHook({
name: 'late-bad',
onEvent: () => {},
init: () => {
throw new Error('no route to collector');
},
}),
).toThrow(/audit hook "late-bad" failed to initialize.*no route/);
});
it('maintainAuditHooks calls every maintain and isolates throws', () => {
const m1 = vi.fn(() => {
throw new Error('flush failed');
});
const m2 = vi.fn();
hooks.registerAuditHook({ name: 'a', onEvent: () => {}, maintain: m1 });
hooks.registerAuditHook({ name: 'b', onEvent: () => {}, maintain: m2 });
hooks.registerAuditHook({ name: 'c', onEvent: () => {} }); // no maintain — fine
expect(() => hooks.maintainAuditHooks()).not.toThrow();
expect(m1).toHaveBeenCalledTimes(1);
expect(m2).toHaveBeenCalledTimes(1);
expect(log.error).toHaveBeenCalledWith(
expect.stringContaining('maintenance failed'),
expect.objectContaining({ hook: 'a' }),
);
});
it('shutdownAuditHooks awaits async shutdowns and isolates throws', async () => {
const order: string[] = [];
hooks.registerAuditHook({
name: 'a',
onEvent: () => {},
shutdown: async () => {
await Promise.resolve();
order.push('a');
},
});
hooks.registerAuditHook({
name: 'b',
onEvent: () => {},
shutdown: () => {
throw new Error('handle already closed');
},
});
hooks.registerAuditHook({
name: 'c',
onEvent: () => {},
shutdown: () => {
order.push('c');
},
});
await hooks.shutdownAuditHooks();
expect(order).toEqual(['a', 'c']);
expect(log.error).toHaveBeenCalledWith(
expect.stringContaining('shutdown failed'),
expect.objectContaining({ hook: 'b' }),
);
});
it('maintainAudit skips hook maintenance when audit is disabled', async () => {
const init = await import('./init.js');
const maintain = vi.fn();
hooks.registerAuditHook({ name: 'a', onEvent: () => {}, maintain });
state.enabled = false;
init.maintainAudit();
expect(maintain).not.toHaveBeenCalled();
state.enabled = true;
init.maintainAudit();
expect(maintain).toHaveBeenCalledTimes(1);
});
});
@@ -0,0 +1,111 @@
/**
* Post-write audit hooks — the in-process extension seam.
*
* A hook observes the audit LOG, not the event stream: `onEvent` fires only
* after an event has been durably appended to the local day-file, so anything
* a hook exports is guaranteed to exist in the source of truth
* (exported ⊆ written). If the local append fails, hooks are not called —
* and a hook that misses events (crash, restart) catches up by reading the
* day-files, which is the at-least-once story.
*
* Registration follows the tree's observer idiom (registerApprovalResolvedHandler,
* registerResponseHandler, …): an in-tree or skill-installed module calls
* `registerAuditHook(...)` at import time — no core edits, and credentials or
* transport for an external system live in that module, never here.
*/
import { log } from '../log.js';
import type { AuditEvent } from './types.js';
export interface AuditHook {
/** Short identifier used in logs and lifecycle errors. */
name: string;
/**
* Called after a successful local append. `line` is the exact stored bytes
* (one NDJSON line, no trailing newline); `event` is the parsed record.
* MUST be fast and non-blocking — this runs on the audited action's call
* path. A real exporter buffers here and does its IO from `maintain`/its own
* timers. Throwing is tolerated: isolated and logged, never propagated.
*/
onEvent(event: AuditEvent, line: string): void;
/**
* One-time setup, called once when audit is enabled — at boot if the hook is
* already registered, else immediately on registration (so it fires exactly
* once regardless of import order). Throw = host refuses to start.
*/
init?(): void;
/** Periodic maintenance — called from the audit maintenance timer (enabled boxes only). */
maintain?(): void;
/** Graceful-shutdown hook (flush buffers, close handles). */
shutdown?(): void | Promise<void>;
}
const hooks: AuditHook[] = [];
let hooksInitialized = false;
export function registerAuditHook(hook: AuditHook): void {
hooks.push(hook);
// onEvent/maintain/shutdown all read the live array, so they pick a hook up
// whenever it registers. init() is the exception — it runs once at boot. If
// boot already ran (a hook whose module loaded after the CLI audit adapter),
// run its init() now; otherwise it would receive events but never its boot
// hook. Same throw-is-fatal posture as boot.
if (hooksInitialized) initOneHook(hook);
}
/** Fan out one written event to every hook, isolating failures per hook. */
export function notifyAuditHooks(event: AuditEvent, line: string): void {
for (const hook of hooks) {
try {
hook.onEvent(event, line);
// eslint-disable-next-line no-catch-all/no-catch-all -- isolation is the contract: one bad hook must not affect the log, other hooks, or the audited action
} catch (err) {
log.error('Audit hook threw — event is safely in the log', { hook: hook.name, action: event.action, err });
}
}
}
/** Boot lifecycle. A hook that can't start is a silent-export-gap risk — fatal. */
export function initAuditHooks(): void {
for (const hook of hooks) initOneHook(hook);
hooksInitialized = true;
}
/**
* Run one hook's boot init with the fatal-on-throw posture. Shared by the boot
* sweep and by a post-boot registration, so every hook gets exactly one init()
* regardless of the import order its module loaded in.
*/
function initOneHook(hook: AuditHook): void {
try {
hook.init?.();
} catch (err) {
throw new Error(
`audit hook "${hook.name}" failed to initialize: ${err instanceof Error ? err.message : String(err)}`,
{ cause: err },
);
}
}
/** Periodic lifecycle — maintenance, isolated per hook. */
export function maintainAuditHooks(): void {
for (const hook of hooks) {
try {
hook.maintain?.();
// eslint-disable-next-line no-catch-all/no-catch-all -- one hook's maintenance failure must not stop the others (or the timer)
} catch (err) {
log.error('Audit hook maintenance failed', { hook: hook.name, err });
}
}
}
/** Shutdown lifecycle — awaited by the host's graceful shutdown. */
export async function shutdownAuditHooks(): Promise<void> {
for (const hook of hooks) {
try {
await hook.shutdown?.();
// eslint-disable-next-line no-catch-all/no-catch-all -- shutdown must drain every hook even when one throws
} catch (err) {
log.error('Audit hook shutdown failed', { hook: hook.name, err });
}
}
}
@@ -0,0 +1,17 @@
/**
* Opt-in local audit log — installed by /add-audit.
*
* This directory is a domain-free leaf: the event schema, the emit seam, the
* store, the reader, post-write hooks, and shared vocabulary. What gets
* audited — and how each domain describes itself — lives in the domain-owned
* `*.audit.ts` adapter files next to the code they observe. Business logic
* contains zero audit calls.
*
* emitAuditEvent is deliberately NOT re-exported here: adapters import it
* from './emit.js' directly, and only src/audit/ + `*.audit.ts` may call it.
*/
export * from './types.js';
export { AUDIT_DIR } from './store.js';
export { AUDIT_ENABLED, AUDIT_RETENTION_DAYS } from './config.js';
export { initAuditLog, maintainAudit } from './init.js';
export { type AuditHook, registerAuditHook } from './hooks.js';
@@ -0,0 +1,54 @@
/**
* Boot-time audit wiring, self-contained (installed by /add-audit).
*
* initAuditLog() runs at module load of the CLI audit adapter
* (cli/dispatch.audit.ts) — i.e. during the host's barrel phase, before the
* CLI server or any delivery poll accepts work — because dispatch.ts, which
* both transports import, composes withAudit at module scope. When enabled:
* assert data/audit/ is writable (refusing to start beats running with a
* silent audit gap), run the boot prune, start the registered post-write
* hooks, and arm the maintenance timer.
*
* The timer owns the daily cadence in-module (checked hourly; the prune
* itself fires at most once per UTC day) so installing the skill touches
* dispatch.ts and the resource barrel only — no host-sweep edit. It is
* unref'd: it never keeps the process alive.
*/
import { log } from '../log.js';
import { onShutdown } from '../response-registry.js';
import { AUDIT_ENABLED, AUDIT_RETENTION_DAYS } from './config.js';
import { initAuditHooks, maintainAuditHooks, shutdownAuditHooks } from './hooks.js';
import { assertAuditWritable, AUDIT_DIR, markPrunedToday, pruneAuditLog, pruneAuditLogIfDue } from './store.js';
const MAINTENANCE_INTERVAL_MS = 60 * 60 * 1000;
let initialized = false;
export function initAuditLog(): void {
if (!AUDIT_ENABLED || initialized) return;
initialized = true;
try {
assertAuditWritable();
} catch (err) {
throw new Error(
`AUDIT_ENABLED=true but the audit directory is not writable: ${AUDIT_DIR} (${err instanceof Error ? err.message : String(err)})`,
{ cause: err },
);
}
pruneAuditLog();
markPrunedToday();
initAuditHooks(); // throw → boot fails, same posture as the writability assert
onShutdown(() => shutdownAuditHooks());
setInterval(maintainAudit, MAINTENANCE_INTERVAL_MS).unref();
log.info('Audit log enabled', { dir: AUDIT_DIR, retentionDays: AUDIT_RETENTION_DAYS });
}
/**
* One maintenance tick: retention prune (throttled to once per UTC day
* internally) plus every hook's periodic maintenance. No-op when audit is
* disabled. Exported so a future scheduler can drive it too.
*/
export function maintainAudit(): void {
pruneAuditLogIfDue();
if (AUDIT_ENABLED) maintainAuditHooks();
}
@@ -0,0 +1,180 @@
import fs from 'fs';
import os from 'os';
import path from 'path';
import { afterEach, beforeEach, describe, expect, it, vi } from 'vitest';
const state = vi.hoisted(() => ({ dataDir: '', enabled: true }));
vi.mock('../config.js', () => ({
get DATA_DIR() {
return state.dataDir;
},
}));
vi.mock('./config.js', () => ({
get AUDIT_ENABLED() {
return state.enabled;
},
AUDIT_RETENTION_DAYS: 90,
}));
vi.mock('../log.js', () => ({
log: { debug: vi.fn(), info: vi.fn(), warn: vi.fn(), error: vi.fn(), fatal: vi.fn() },
}));
let reader: typeof import('./reader.js');
let store: typeof import('./store.js');
beforeEach(async () => {
state.dataDir = fs.mkdtempSync(path.join(os.tmpdir(), 'audit-reader-'));
state.enabled = true;
vi.resetModules();
store = await import('./store.js');
reader = await import('./reader.js');
});
afterEach(() => {
fs.rmSync(state.dataDir, { recursive: true, force: true });
});
function dayString(daysAgo: number): string {
return store.utcDay(new Date(Date.now() - daysAgo * 24 * 60 * 60 * 1000));
}
function isoAt(daysAgo: number, tag: number): string {
const base = new Date(Date.now() - daysAgo * 24 * 60 * 60 * 1000);
return `${base.toISOString().slice(0, 10)}T10:00:0${tag}.000Z`;
}
let seq = 0;
function seedEvent(daysAgo: number, overrides: Record<string, unknown> = {}): Record<string, unknown> {
seq += 1;
const event = {
event_id: `e-${seq}`,
time: isoAt(daysAgo, seq % 10),
schema_version: 1,
actor: { type: 'human', id: 'host:moshe', email: null, user_id: null, group_ids: null },
origin: { transport: 'socket' },
action: 'groups.list',
resources: [{ type: 'agent_group', id: 'ag-1' }],
outcome: 'success',
correlation_id: null,
details: {},
...overrides,
};
const dir = path.join(state.dataDir, 'audit');
fs.mkdirSync(dir, { recursive: true });
fs.appendFileSync(path.join(dir, `${dayString(daysAgo)}.ndjson`), JSON.stringify(event) + '\n');
return event;
}
describe('listAuditEvents', () => {
it('throws the disabled error rather than returning an empty list', () => {
state.enabled = false;
expect(() => reader.listAuditEvents({})).toThrow('audit log is disabled — set AUDIT_ENABLED=true');
});
it('returns flat rows newest-first across day-files, honoring --limit', () => {
seedEvent(2, { event_id: 'old' });
seedEvent(1, { event_id: 'mid-a' });
seedEvent(1, { event_id: 'mid-b' });
seedEvent(0, { event_id: 'new' });
const rows = reader.listAuditEvents({}) as Array<Record<string, unknown>>;
expect(rows.map((r) => r.event_id)).toEqual(['new', 'mid-b', 'mid-a', 'old']);
expect(rows[0]).toMatchObject({ actor: 'host:moshe', action: 'groups.list', outcome: 'success' });
expect(rows[0].resources).toBe('agent_group:ag-1');
const limited = reader.listAuditEvents({ limit: 2 }) as Array<Record<string, unknown>>;
expect(limited.map((r) => r.event_id)).toEqual(['new', 'mid-b']);
});
it('filters by actor, outcome, correlation, and resource (id or type)', () => {
seedEvent(0, { event_id: 'a', actor: { type: 'agent', id: 'ag-1' }, outcome: 'denied' });
seedEvent(0, { event_id: 'b', correlation_id: 'appr-9', resources: [{ type: 'approval', id: 'appr-9' }] });
seedEvent(0, { event_id: 'c', resources: [{ type: 'user', id: 'slack:U1' }] });
expect((reader.listAuditEvents({ actor: 'ag-1' }) as unknown[]).length).toBe(1);
expect((reader.listAuditEvents({ outcome: 'denied' }) as unknown[]).length).toBe(1);
expect((reader.listAuditEvents({ correlation: 'appr-9' }) as unknown[]).length).toBe(1);
expect((reader.listAuditEvents({ resource: 'slack:U1' }) as unknown[]).length).toBe(1);
expect((reader.listAuditEvents({ resource: 'approval' }) as unknown[]).length).toBe(1);
});
it('matches actions exactly or by dotted prefix', () => {
seedEvent(0, { event_id: 'cfg', action: 'groups.config.add-mcp-server' });
seedEvent(0, { event_id: 'list', action: 'groups.list' });
seedEvent(0, { event_id: 'other', action: 'sessions.list' });
expect((reader.listAuditEvents({ action: 'groups' }) as unknown[]).length).toBe(2);
expect((reader.listAuditEvents({ action: 'groups.config' }) as unknown[]).length).toBe(1);
expect((reader.listAuditEvents({ action: 'groups.list' }) as unknown[]).length).toBe(1);
// Prefix means dotted segments, not substrings.
expect((reader.listAuditEvents({ action: 'group' }) as unknown[]).length).toBe(0);
});
it('applies --since/--until with relative and ISO forms', () => {
seedEvent(5, { event_id: 'old' });
seedEvent(0, { event_id: 'recent' });
const relative = reader.listAuditEvents({ since: '2d' }) as Array<Record<string, unknown>>;
expect(relative.map((r) => r.event_id)).toEqual(['recent']);
const iso = reader.listAuditEvents({ until: dayString(2) }) as Array<Record<string, unknown>>;
expect(iso.map((r) => r.event_id)).toEqual(['old']);
expect(() => reader.listAuditEvents({ since: 'yesterdayish' })).toThrow('invalid --since');
});
it('--format ndjson returns the stored lines verbatim', () => {
const seeded = seedEvent(0, { event_id: 'x1' });
const out = reader.listAuditEvents({ format: 'ndjson' });
expect(typeof out).toBe('string');
expect(JSON.parse(out as string)).toEqual(seeded);
expect(() => reader.listAuditEvents({ format: 'csv' })).toThrow('invalid --format');
});
it('--format ndjson exports every event in the window (no silent 100-row cap)', () => {
for (let i = 0; i < 150; i++) seedEvent(0, { event_id: `e-${i}` });
const ndjson = reader.listAuditEvents({ format: 'ndjson' }) as string;
expect(ndjson.split('\n').length).toBe(150);
// The table view still caps at the default for readability.
expect((reader.listAuditEvents({}) as unknown[]).length).toBe(100);
});
it('honors an explicit --limit 0 as zero rows, not the default', () => {
seedEvent(0, { event_id: 'a' });
expect(reader.listAuditEvents({ limit: 0 })).toEqual([]);
});
it('rejects a negative or non-integer --limit', () => {
expect(() => reader.listAuditEvents({ limit: -1 })).toThrow('invalid --limit');
expect(() => reader.listAuditEvents({ limit: 'lots' })).toThrow('invalid --limit');
});
it('parses a timezone-less --since/--until datetime as UTC, not local', () => {
expect(reader.parseTimeFlag('2026-07-12T00:00:00', '--since')).toBe(Date.parse('2026-07-12T00:00:00Z'));
// Date-only already parses UTC; an explicit offset is respected as-is.
expect(reader.parseTimeFlag('2026-07-12', '--since')).toBe(Date.parse('2026-07-12'));
expect(reader.parseTimeFlag('2026-07-12T00:00:00+02:00', '--until')).toBe(
Date.parse('2026-07-12T00:00:00+02:00'),
);
});
it('skips malformed stored lines and still returns the rest', () => {
seedEvent(0, { event_id: 'good' });
fs.appendFileSync(path.join(state.dataDir, 'audit', `${dayString(0)}.ndjson`), 'not-json\n');
const rows = reader.listAuditEvents({}) as Array<Record<string, unknown>>;
expect(rows.map((r) => r.event_id)).toEqual(['good']);
});
it('rejects an unknown --outcome value', () => {
expect(() => reader.listAuditEvents({ outcome: 'meh' })).toThrow('invalid --outcome');
});
it('returns empty when nothing has been recorded yet', () => {
expect(reader.listAuditEvents({})).toEqual([]);
});
});
@@ -0,0 +1,178 @@
/**
* Read-back for `ncl audit list` — a newest-first stream-scan over the
* day-files. No index: fine at v1 volume, and adding one later doesn't change
* the store. NDJSON export returns the stored lines verbatim.
*/
import fs from 'fs';
import path from 'path';
import { log } from '../log.js';
import { AUDIT_ENABLED } from './config.js';
import { AUDIT_DIR, utcDay } from './store.js';
import type { AuditEvent, AuditOutcome } from './types.js';
const OUTCOMES: ReadonlySet<string> = new Set(['success', 'failure', 'denied', 'pending', 'approved']);
const DAY_FILE_RE = /^(\d{4}-\d{2}-\d{2})\.ndjson$/;
const DEFAULT_LIMIT = 100;
export interface AuditQuery {
actor?: string;
/** Exact action or dotted prefix (`groups` matches `groups.config.update`). */
action?: string;
/** Matches any resources[] entry by id or by type. */
resource?: string;
outcome?: AuditOutcome;
sinceMs?: number;
untilMs?: number;
correlation?: string;
limit: number;
}
/** `7d` / `24h` / `30m` relative to now, or an ISO date/datetime (UTC). */
export function parseTimeFlag(value: string, flag: string): number {
const rel = /^(\d+)([dhm])$/.exec(value);
if (rel) {
const n = Number(rel[1]);
const unitMs = rel[2] === 'd' ? 86_400_000 : rel[2] === 'h' ? 3_600_000 : 60_000;
return Date.now() - n * unitMs;
}
// A timezone-less ISO *datetime* parses as LOCAL per spec, but --since/--until
// are documented UTC; append 'Z' so a bare datetime reads as UTC. Date-only
// forms already parse as UTC, and an explicit offset (`+02:00`/`Z`) is left as-is.
const normalized = /^\d{4}-\d{2}-\d{2}T[\d.:]+$/.test(value) ? `${value}Z` : value;
const abs = Date.parse(normalized);
if (!Number.isNaN(abs)) return abs;
throw new Error(`invalid ${flag} value "${value}" — use e.g. 7d, 24h, 30m, or an ISO date`);
}
/** Newest first across files and within each file, up to q.limit. */
export function queryAuditEvents(q: AuditQuery): { events: AuditEvent[]; lines: string[] } {
const events: AuditEvent[] = [];
const lines: string[] = [];
let malformed = 0;
for (const { day, file } of dayFilesNewestFirst()) {
if (events.length >= q.limit) break;
// Whole-day skip: a file can't match a window its day lies outside.
if (q.sinceMs !== undefined && day < utcDay(new Date(q.sinceMs))) continue;
if (q.untilMs !== undefined && day > utcDay(new Date(q.untilMs))) continue;
let content: string;
try {
content = fs.readFileSync(file, 'utf8');
// eslint-disable-next-line no-catch-all/no-catch-all -- a torn/pruned day-file must not fail the whole query
} catch (err) {
log.warn('Audit reader failed to read day-file', { file, err });
continue;
}
const fileLines = content.split('\n').filter((l) => l.trim() !== '');
// Lines within a file are chronological — walk backwards for newest-first.
for (let i = fileLines.length - 1; i >= 0 && events.length < q.limit; i--) {
let event: AuditEvent;
try {
event = JSON.parse(fileLines[i]) as AuditEvent;
// eslint-disable-next-line no-catch-all/no-catch-all -- malformed stored lines are skipped (counted + warned below)
} catch {
malformed++;
continue;
}
if (!matches(event, q)) continue;
events.push(event);
lines.push(fileLines[i]);
}
}
if (malformed > 0) {
log.warn('Audit reader skipped malformed lines', { malformed });
}
return { events, lines };
}
function dayFilesNewestFirst(): Array<{ day: string; file: string }> {
let entries: string[];
try {
entries = fs.readdirSync(AUDIT_DIR);
// eslint-disable-next-line no-catch-all/no-catch-all -- no audit dir yet means no events, not an error
} catch {
return [];
}
return entries
.map((e) => DAY_FILE_RE.exec(e))
.filter((m): m is RegExpExecArray => m !== null)
.map((m) => ({ day: m[1], file: path.join(AUDIT_DIR, m[0]) }))
.sort((a, b) => (a.day < b.day ? 1 : -1));
}
function matches(event: AuditEvent, q: AuditQuery): boolean {
if (q.actor !== undefined && event.actor?.id !== q.actor) return false;
if (q.action !== undefined && event.action !== q.action && !event.action?.startsWith(q.action + '.')) return false;
if (q.outcome !== undefined && event.outcome !== q.outcome) return false;
if (q.correlation !== undefined && event.correlation_id !== q.correlation) return false;
if (q.resource !== undefined) {
const hit = (event.resources ?? []).some((r) => r.id === q.resource || r.type === q.resource);
if (!hit) return false;
}
const t = Date.parse(event.time ?? '');
if (q.sinceMs !== undefined && !(t >= q.sinceMs)) return false;
if (q.untilMs !== undefined && !(t <= q.untilMs)) return false;
return true;
}
/**
* `ncl audit list` handler. Disabled → an explicit error: an empty list would
* read as "no actions happened", which is a different truth than "not
* recording". `--format ndjson` returns the stored lines verbatim (the human
* formatter passes strings through); default returns flat rows for the table.
*/
export function listAuditEvents(args: Record<string, unknown>): string | Array<Record<string, unknown>> {
if (!AUDIT_ENABLED) {
throw new Error('audit log is disabled — set AUDIT_ENABLED=true');
}
const format = args.format !== undefined ? String(args.format) : '';
if (format && format !== 'ndjson') {
throw new Error(`invalid --format "${format}" — only "ndjson" is supported`);
}
const outcome = args.outcome !== undefined ? String(args.outcome) : undefined;
if (outcome !== undefined && !OUTCOMES.has(outcome)) {
throw new Error(`invalid --outcome "${outcome}" — one of: ${[...OUTCOMES].join(', ')}`);
}
// An explicit --limit is honored exactly (0 means 0, not "default"). Absent,
// the table view caps at DEFAULT_LIMIT for readability, but ndjson — the SIEM
// export path — is bounded only by --since/--until, never silently truncated.
let limit: number;
if (args.limit !== undefined) {
const n = Number(args.limit);
if (!Number.isInteger(n) || n < 0) {
throw new Error(`invalid --limit "${String(args.limit)}" — use a non-negative integer`);
}
limit = n;
} else {
limit = format === 'ndjson' ? Infinity : DEFAULT_LIMIT;
}
const q: AuditQuery = {
actor: args.actor !== undefined ? String(args.actor) : undefined,
action: args.action !== undefined ? String(args.action) : undefined,
resource: args.resource !== undefined ? String(args.resource) : undefined,
outcome: outcome as AuditOutcome | undefined,
correlation: args.correlation !== undefined ? String(args.correlation) : undefined,
sinceMs: args.since !== undefined ? parseTimeFlag(String(args.since), '--since') : undefined,
untilMs: args.until !== undefined ? parseTimeFlag(String(args.until), '--until') : undefined,
limit,
};
const { events, lines } = queryAuditEvents(q);
if (format === 'ndjson') return lines.join('\n');
return events.map((e) => ({
time: e.time,
actor: e.actor?.id ?? '',
action: e.action,
resources: (e.resources ?? []).map((r) => (r.id ? `${r.type}:${r.id}` : r.type)).join(' '),
outcome: e.outcome,
correlation: e.correlation_id ?? '',
event_id: e.event_id,
}));
}
@@ -0,0 +1,197 @@
import fs from 'fs';
import os from 'os';
import path from 'path';
import { afterEach, beforeEach, describe, expect, it, vi } from 'vitest';
// Core config (DATA_DIR) and the audit config (toggles) are mocked so
// store/emit resolve a per-test temp DATA_DIR and audit switches. Getters
// keep the values live across vi.resetModules().
const state = vi.hoisted(() => ({ dataDir: '', enabled: true, retention: 90 }));
vi.mock('../config.js', () => ({
get DATA_DIR() {
return state.dataDir;
},
}));
vi.mock('./config.js', () => ({
get AUDIT_ENABLED() {
return state.enabled;
},
get AUDIT_RETENTION_DAYS() {
return state.retention;
},
}));
vi.mock('../log.js', () => ({
log: { debug: vi.fn(), info: vi.fn(), warn: vi.fn(), error: vi.fn(), fatal: vi.fn() },
}));
let store: typeof import('./store.js');
let emit: typeof import('./emit.js');
let log: (typeof import('../log.js'))['log'];
beforeEach(async () => {
state.dataDir = fs.mkdtempSync(path.join(os.tmpdir(), 'audit-store-'));
state.enabled = true;
state.retention = 90;
vi.resetModules();
store = await import('./store.js');
emit = await import('./emit.js');
log = (await import('../log.js')).log;
vi.clearAllMocks();
});
afterEach(() => {
fs.chmodSync(path.join(state.dataDir), 0o700);
const auditDir = path.join(state.dataDir, 'audit');
if (fs.existsSync(auditDir)) fs.chmodSync(auditDir, 0o700);
fs.rmSync(state.dataDir, { recursive: true, force: true });
});
function auditDir(): string {
return path.join(state.dataDir, 'audit');
}
function writeDayFile(day: string, lines = 1): void {
fs.mkdirSync(auditDir(), { recursive: true });
fs.writeFileSync(path.join(auditDir(), `${day}.ndjson`), '{"x":1}\n'.repeat(lines));
}
function dayString(daysAgo: number): string {
return store.utcDay(new Date(Date.now() - daysAgo * 24 * 60 * 60 * 1000));
}
const EVENT_INPUT = {
actor: { type: 'human' as const, id: 'host:test' },
origin: { transport: 'socket' as const },
action: 'groups.list',
resources: [{ type: 'agent_group' }],
outcome: 'success' as const,
};
describe('appendAuditLine', () => {
it("appends one line to today's UTC day-file, creating the directory lazily", () => {
expect(fs.existsSync(auditDir())).toBe(false);
store.appendAuditLine('{"a":1}');
store.appendAuditLine('{"b":2}');
const file = path.join(auditDir(), `${store.utcDay()}.ndjson`);
expect(fs.readFileSync(file, 'utf8')).toBe('{"a":1}\n{"b":2}\n');
});
it('buckets by the given event day, not wall-clock at append', () => {
store.appendAuditLine('{"c":3}', '2020-01-02');
expect(fs.readFileSync(path.join(auditDir(), '2020-01-02.ndjson'), 'utf8')).toBe('{"c":3}\n');
// Today's file is untouched — the line filed under its own day.
expect(fs.existsSync(path.join(auditDir(), `${store.utcDay()}.ndjson`))).toBe(false);
});
});
describe('emitAuditEvent', () => {
it('writes a schema_version-1 record with envelope fields and null enrichment', () => {
emit.emitAuditEvent({ ...EVENT_INPUT, details: { limit: 100 } });
const file = path.join(auditDir(), `${store.utcDay()}.ndjson`);
const record = JSON.parse(fs.readFileSync(file, 'utf8').trim());
expect(record).toMatchObject({
schema_version: 1,
actor: { type: 'human', id: 'host:test', email: null, user_id: null, group_ids: null },
origin: { transport: 'socket' },
action: 'groups.list',
outcome: 'success',
correlation_id: null,
details: { limit: 100 },
});
expect(record.event_id).toMatch(/^[0-9a-f-]{36}$/);
expect(new Date(record.time).toISOString()).toBe(record.time);
});
it('stores details verbatim — the emit seam performs no value transformation', () => {
// emit is domain-free: adapters build `details` value-free, so there is
// nothing to redact here. Whatever is passed is written as-is.
emit.emitAuditEvent({ ...EVENT_INPUT, details: { args: ['role'], role: 'admin' } });
const file = path.join(auditDir(), `${store.utcDay()}.ndjson`);
const record = JSON.parse(fs.readFileSync(file, 'utf8').trim());
expect(record.details).toEqual({ args: ['role'], role: 'admin' });
});
it('is a no-op when audit is disabled — the directory is never created', () => {
state.enabled = false;
emit.emitAuditEvent(EVENT_INPUT);
expect(fs.existsSync(auditDir())).toBe(false);
});
it('fails open and loud when the append fails', () => {
fs.mkdirSync(auditDir(), { recursive: true });
fs.chmodSync(auditDir(), 0o500);
expect(() => emit.emitAuditEvent(EVENT_INPUT)).not.toThrow();
expect(log.error).toHaveBeenCalledWith(
expect.stringContaining('Audit append failed'),
expect.objectContaining({ action: 'groups.list' }),
);
});
});
describe('assertAuditWritable', () => {
it('creates the directory and probes it with a zero-byte append', () => {
store.assertAuditWritable();
expect(fs.existsSync(path.join(auditDir(), `${store.utcDay()}.ndjson`))).toBe(true);
});
it('throws when the directory is not writable', () => {
fs.mkdirSync(auditDir(), { recursive: true });
fs.chmodSync(auditDir(), 0o500);
expect(() => store.assertAuditWritable()).toThrow();
});
});
describe('pruneAuditLog', () => {
it('unlinks only day-files strictly older than the horizon', () => {
writeDayFile(dayString(100));
writeDayFile(dayString(91));
writeDayFile(dayString(90));
writeDayFile(dayString(1));
fs.writeFileSync(path.join(auditDir(), 'not-a-day-file.txt'), 'keep');
fs.writeFileSync(path.join(auditDir(), '2026-01-01.ndjson.bak'), 'keep');
store.pruneAuditLog(90);
const left = fs.readdirSync(auditDir()).sort();
expect(left).toEqual(
[`${dayString(90)}.ndjson`, `${dayString(1)}.ndjson`, '2026-01-01.ndjson.bak', 'not-a-day-file.txt'].sort(),
);
});
it('keeps forever when retention is 0', () => {
writeDayFile(dayString(400));
store.pruneAuditLog(0);
expect(fs.existsSync(path.join(auditDir(), `${dayString(400)}.ndjson`))).toBe(true);
});
it('is a no-op when the audit directory does not exist', () => {
expect(() => store.pruneAuditLog(90)).not.toThrow();
});
});
describe('pruneAuditLogIfDue', () => {
it('prunes at most once per UTC day', () => {
writeDayFile(dayString(100));
store.pruneAuditLogIfDue();
expect(fs.existsSync(path.join(auditDir(), `${dayString(100)}.ndjson`))).toBe(false);
writeDayFile(dayString(100));
store.pruneAuditLogIfDue();
expect(fs.existsSync(path.join(auditDir(), `${dayString(100)}.ndjson`))).toBe(true);
});
it('does nothing when audit is disabled', () => {
state.enabled = false;
writeDayFile(dayString(100));
store.pruneAuditLogIfDue();
expect(fs.existsSync(path.join(auditDir(), `${dayString(100)}.ndjson`))).toBe(true);
});
it('does nothing after markPrunedToday until the day rolls over', () => {
store.markPrunedToday();
writeDayFile(dayString(100));
store.pruneAuditLogIfDue();
expect(fs.existsSync(path.join(auditDir(), `${dayString(100)}.ndjson`))).toBe(true);
});
});
@@ -0,0 +1,87 @@
/**
* Day-file store — daily NDJSON files under data/audit/, host process is the
* single writer. Append-only is structural: nothing here can update a line,
* and retention is unlinking whole files (a literal hard delete). Both
* transports converge host-side, so single-writer holds by construction.
*
* This module throws on fs failure; the fail-open posture (log.error, action
* proceeds) lives in emit.ts, and the boot-time strictness (refuse to start)
* lives in init.ts.
*/
import fs from 'fs';
import path from 'path';
import { DATA_DIR } from '../config.js';
import { log } from '../log.js';
import { AUDIT_ENABLED, AUDIT_RETENTION_DAYS } from './config.js';
export const AUDIT_DIR = path.join(DATA_DIR, 'audit');
const DAY_FILE_RE = /^(\d{4}-\d{2}-\d{2})\.ndjson$/;
const DAY_MS = 24 * 60 * 60 * 1000;
export function utcDay(d: Date = new Date()): string {
return d.toISOString().slice(0, 10);
}
export function dayFilePath(day: string): string {
return path.join(AUDIT_DIR, `${day}.ndjson`);
}
/**
* The single append point. `day` buckets the line by the EVENT's own UTC day
* (emit passes it from event.time), not wall-clock at append — so a line whose
* emit crossed a UTC-midnight tick still files under its timestamp's day, and a
* bounded reader query can't whole-day-skip it. Throws on fs failure —
* emitAuditEvent catches.
*/
export function appendAuditLine(line: string, day: string = utcDay()): void {
fs.mkdirSync(AUDIT_DIR, { recursive: true });
fs.appendFileSync(dayFilePath(day), line + '\n');
}
/** Boot-time writability assert — a zero-byte append is a true write probe. */
export function assertAuditWritable(): void {
fs.mkdirSync(AUDIT_DIR, { recursive: true });
fs.appendFileSync(dayFilePath(utcDay()), '');
}
/** Unlink day-files strictly older than (today UTC retentionDays). 0 or negative = keep forever. */
export function pruneAuditLog(retentionDays: number = AUDIT_RETENTION_DAYS): void {
if (retentionDays <= 0) return;
let entries: string[];
try {
entries = fs.readdirSync(AUDIT_DIR);
// eslint-disable-next-line no-catch-all/no-catch-all -- no audit dir yet means nothing to prune, not an error
} catch {
return;
}
const horizon = utcDay(new Date(Date.now() - retentionDays * DAY_MS));
let pruned = 0;
for (const entry of entries) {
const m = DAY_FILE_RE.exec(entry);
if (!m || m[1] >= horizon) continue;
try {
fs.unlinkSync(path.join(AUDIT_DIR, entry));
pruned++;
// eslint-disable-next-line no-catch-all/no-catch-all -- one stuck file must not stop the prune of the others
} catch (err) {
log.error('Audit prune failed to unlink day-file', { file: entry, err });
}
}
if (pruned > 0) log.info('Audit retention pruned day-files', { pruned, retentionDays });
}
let lastPruneDay: string | null = null;
/** Retention prune, throttled to once per UTC day — safe to call every tick. */
export function pruneAuditLogIfDue(): void {
if (!AUDIT_ENABLED || AUDIT_RETENTION_DAYS <= 0) return;
const today = utcDay();
if (lastPruneDay === today) return;
lastPruneDay = today;
pruneAuditLog();
}
export function markPrunedToday(): void {
lastPruneDay = utcDay();
}
@@ -0,0 +1,57 @@
/**
* Canonical audit event — one flat, SIEM-shaped record per action.
*
* Fields are chosen so they project losslessly onto OCSF and Elastic ECS
* (the two schemas SIEMs converge on); the store keeps the neutral shape and
* a future forwarder pays the translation once, at the edge. `schema_version`
* covers evolution — additive changes only within a version.
*/
export type AuditActorType = 'human' | 'agent' | 'system';
export interface AuditActor {
type: AuditActorType;
/** `host:<os-user>` | `<channel>:<handle>` | agent group id | `host` (system). */
id: string;
}
export interface AuditOrigin {
transport: 'socket' | 'container' | 'channel';
session_id?: string;
messaging_group_id?: string;
channel?: string;
}
export interface AuditResource {
type: string;
/** Omitted when only the attempted type is known (e.g. a denied list). */
id?: string;
}
export type AuditOutcome = 'success' | 'failure' | 'denied' | 'pending' | 'approved';
/** What emit sites provide. Envelope fields are stamped by emitAuditEvent. */
export interface AuditEventInput {
actor: AuditActor;
origin: AuditOrigin;
/** Dotted namespaced verb, e.g. `groups.config.add-mcp-server`. */
action: string;
resources: AuditResource[];
outcome: AuditOutcome;
/** The approval id on gated chains; null/omitted otherwise. */
correlationId?: string | null;
details?: Record<string, unknown>;
}
export interface AuditEvent {
event_id: string;
time: string;
schema_version: 1;
actor: AuditActor & { email: null; user_id: null; group_ids: null };
origin: AuditOrigin;
action: string;
resources: AuditResource[];
outcome: AuditOutcome;
correlation_id: string | null;
details: Record<string, unknown>;
}
@@ -0,0 +1,39 @@
/**
* Domain-free event vocabulary — actor and origin constructors shared by the
* domain-owned `*.audit.ts` adapters (the CLI adapter today; approval and
* channel adapters attach here in later increments). Pure derivation; nothing
* here writes the log.
*
* Leaf rule: this module (like the rest of src/audit/) may depend on node,
* config/log, shared types, and the db read layer — never on src/cli/* or
* src/modules/*. Domain-specific mapping (CLI resources, approval payloads)
* lives in the adapter file of the domain that owns it.
*/
import os from 'os';
import { getMessagingGroup } from '../db/messaging-groups.js';
import type { AuditOrigin } from './types.js';
/**
* Host callers stamp `host:<install user>` daemon-side: the ncl socket is
* 0600 and owned by the install user, so the identity is accurate by
* construction without peer credentials.
*/
export function hostUser(): string {
try {
return os.userInfo().username;
// eslint-disable-next-line no-catch-all/no-catch-all -- os.userInfo throws on exotic hosts; a fallback actor id beats no audit event
} catch {
return process.env.USER || 'unknown';
}
}
export function containerOrigin(sessionId: string, messagingGroupId: string | null): AuditOrigin {
const origin: AuditOrigin = { transport: 'container', session_id: sessionId };
if (messagingGroupId) {
origin.messaging_group_id = messagingGroupId;
const channel = getMessagingGroup(messagingGroupId)?.channel_type;
if (channel) origin.channel = channel;
}
return origin;
}
@@ -0,0 +1,359 @@
/**
* Audit middleware behavior of the exported dispatch what gets recorded,
* for whom, and how gated chains correlate. Drives the real wrapped dispatch
* (real registry, real guard); audit is force-enabled and the store's append
* is captured. DB reads and approval delivery are mocked.
*
* Recording model under test: the log stores arg KEY NAMES plus a small
* allowlist of safe enum values never raw argument values so a
* secret-bearing arg leaves only its key behind.
*/
import os from 'os';
import { beforeEach, describe, expect, it, vi } from 'vitest';
import type { PendingApproval } from '../types.js';
const appended = vi.hoisted(() => ({ lines: [] as string[] }));
const pendingRows = vi.hoisted(() => ({ rows: [] as unknown[] }));
vi.mock('../audit/config.js', () => ({
AUDIT_ENABLED: true,
AUDIT_RETENTION_DAYS: 90,
}));
// Neutralize the adapter's module-scope boot (writability assert, prune,
// maintenance timer) — the middleware is the unit under test here.
vi.mock('../audit/init.js', () => ({
initAuditLog: vi.fn(),
maintainAudit: vi.fn(),
}));
vi.mock('../audit/store.js', async (importOriginal) => {
const actual = await importOriginal<typeof import('../audit/store.js')>();
return {
...actual,
appendAuditLine: (line: string) => {
appended.lines.push(line);
},
};
});
vi.mock('../log.js', () => ({
log: { debug: vi.fn(), info: vi.fn(), warn: vi.fn(), error: vi.fn(), fatal: vi.fn() },
}));
const mockGetContainerConfig = vi.fn();
vi.mock('../db/container-configs.js', () => ({
getContainerConfig: (...args: unknown[]) => mockGetContainerConfig(...args),
}));
vi.mock('../db/agent-groups.js', () => ({
getAgentGroup: vi.fn(() => ({ id: 'g1', name: 'Group One' })),
}));
const mockGetPendingApproval = vi.fn();
vi.mock('../db/sessions.js', () => ({
getSession: vi.fn(() => ({ id: 's1', agent_group_id: 'g1', messaging_group_id: 'mg1' })),
getPendingApproval: (...args: unknown[]) => mockGetPendingApproval(...args),
getPendingApprovalsByAction: () => pendingRows.rows,
}));
vi.mock('../db/messaging-groups.js', () => ({
getMessagingGroup: vi.fn(() => ({ channel_type: 'slack' })),
}));
const mockGetResource = vi.fn();
vi.mock('./crud.js', () => ({
getResource: (...args: unknown[]) => mockGetResource(...args),
}));
const mockRequestApproval = vi.fn();
vi.mock('../modules/approvals/index.js', () => ({
registerApprovalHandler: vi.fn(),
requestApproval: (...args: unknown[]) => mockRequestApproval(...args),
}));
import { register } from './registry.js';
register({
name: 'groups-test',
description: 'echo command on the groups resource',
action: 'groups.test',
resource: 'groups',
access: 'open',
parseArgs: (raw) => raw,
handler: async (args) => ({ echo: args }),
});
register({
name: 'groups-get',
description: 'echo command for dash-joined id resolution',
action: 'groups.get',
resource: 'groups',
access: 'open',
parseArgs: (raw) => raw,
handler: async (args) => ({ echo: args }),
});
register({
name: 'wirings-list',
description: 'not on the group-scope allowlist',
action: 'wirings.list',
resource: 'wirings',
access: 'open',
parseArgs: (raw) => raw,
handler: async () => [],
});
register({
name: 'groups-fail',
description: 'handler that throws',
action: 'groups.fail',
resource: 'groups',
access: 'open',
parseArgs: (raw) => raw,
handler: async () => {
throw new Error('boom');
},
});
register({
name: 'groups-gated',
description: 'approval-gated command',
action: 'groups.gated',
resource: 'groups',
access: 'approval',
parseArgs: (raw) => raw,
handler: async () => 'ran',
});
register({
name: 'roles-grant',
description: 'command carrying an allowlisted --role value',
action: 'roles.grant',
resource: 'roles',
access: 'open',
parseArgs: (raw) => raw,
handler: async () => ({ granted: true }),
});
import { dispatch } from './dispatch.js';
import type { CallerContext } from './frame.js';
const AGENT_CTX: CallerContext = { caller: 'agent', sessionId: 's1', agentGroupId: 'g1', messagingGroupId: 'mg1' };
function grantRow(frameId: string, command: string): PendingApproval {
return {
approval_id: 'appr-123-abc',
session_id: 's1',
request_id: 'appr-123-abc',
action: 'cli_command',
payload: JSON.stringify({ frame: { id: frameId, command, args: {} }, callerContext: AGENT_CTX }),
created_at: new Date().toISOString(),
agent_group_id: 'g1',
channel_type: null,
platform_id: null,
platform_message_id: null,
expires_at: null,
status: 'pending',
title: 'CLI: groups-gated',
options_json: '[]',
approver_user_id: null,
};
}
function events(): Array<Record<string, any>> {
return appended.lines.map((l) => JSON.parse(l));
}
beforeEach(() => {
vi.clearAllMocks();
appended.lines.length = 0;
pendingRows.rows = [];
mockGetContainerConfig.mockReturnValue({ cli_scope: 'group' });
mockGetResource.mockImplementation((plural: string) => (plural === 'groups' ? { scopeField: 'id' } : undefined));
mockRequestApproval.mockResolvedValue(undefined);
});
describe('withAudit(dispatch)', () => {
it('records a success event for a host caller with socket origin and host actor', async () => {
const resp = await dispatch({ id: '1', command: 'groups-test', args: { foo: 'bar' } }, { caller: 'host' });
expect(resp.ok).toBe(true);
const [event] = events();
expect(event).toMatchObject({
schema_version: 1,
actor: { type: 'human', id: `host:${os.userInfo().username}`, email: null },
origin: { transport: 'socket' },
action: 'groups.test',
outcome: 'success',
correlation_id: null,
details: { args: ['foo'] },
});
// The value 'bar' is never stored — only the key name.
expect(event.details.foo).toBeUndefined();
});
it('records an agent command value-free, with container origin and channel', async () => {
await dispatch({ id: '1', command: 'groups-test', args: {} }, AGENT_CTX);
const [event] = events();
expect(event.actor).toMatchObject({ type: 'agent', id: 'g1' });
expect(event.origin).toEqual({
transport: 'container',
session_id: 's1',
messaging_group_id: 'mg1',
channel: 'slack',
});
expect(event.details).toEqual({ args: [] });
expect(event.resources).toContainEqual({ type: 'agent_group' });
});
it('records the passed flag names and echoes allowlisted safe values verbatim', async () => {
await dispatch(
{ id: '1', command: 'roles-grant', args: { role: 'admin', user: 'slack:U1' } },
{ caller: 'host' },
);
const [event] = events();
expect(event.action).toBe('roles.grant');
// Both flag names recorded; only the allowlisted `role` keeps its value.
expect(event.details.args).toEqual(['role', 'user']);
expect(event.details.role).toBe('admin');
expect(event.details.user).toBeUndefined();
// The user id still surfaces structurally, in resources.
expect(event.resources).toContainEqual({ type: 'user', id: 'slack:U1' });
});
it('records a denied event for a scope denial, naming the attempted resource type, with no free-text reason', async () => {
const resp = await dispatch({ id: '1', command: 'wirings-list', args: {} }, AGENT_CTX);
expect(resp.ok).toBe(false);
const [event] = events();
expect(event).toMatchObject({
action: 'wirings.list',
outcome: 'denied',
resources: [{ type: 'wirings' }],
details: { args: [], error: 'forbidden' },
});
// The denial message (which can echo caller input) is never stored.
expect(event.details.reason).toBeUndefined();
});
it('records a failure event with the error code only when the handler throws', async () => {
await dispatch({ id: '1', command: 'groups-fail', args: {} }, { caller: 'host' });
const [event] = events();
expect(event).toMatchObject({ action: 'groups.fail', outcome: 'failure', details: { error: 'handler-error' } });
expect(event.details.reason).toBeUndefined();
});
it('records a failure event and re-throws when the dispatcher itself throws', async () => {
mockRequestApproval.mockRejectedValueOnce(new Error('pending_approvals insert failed'));
await expect(dispatch({ id: '1', command: 'groups-gated', args: {} }, AGENT_CTX)).rejects.toThrow(
'pending_approvals insert failed',
);
const [event] = events();
expect(event).toMatchObject({ action: 'groups.gated', outcome: 'failure', details: { error: 'exception' } });
});
it('records a hold as a pending event correlated to the approval row it created', async () => {
pendingRows.rows = [grantRow('1', 'groups-gated')];
const resp = await dispatch({ id: '1', command: 'groups-gated', args: {} }, AGENT_CTX);
expect(resp.ok).toBe(false);
if (!resp.ok) expect(resp.error.code).toBe('approval-pending');
const [event] = events();
expect(event).toMatchObject({
action: 'groups.gated',
outcome: 'pending',
correlation_id: 'appr-123-abc',
});
expect(event.resources).toContainEqual({ type: 'approval', id: 'appr-123-abc' });
expect(event.details.error).toBeUndefined();
});
it('records an uncorrelated pending event when no approval row was created (no approver)', async () => {
pendingRows.rows = [];
await dispatch({ id: '1', command: 'groups-gated', args: {} }, AGENT_CTX);
const [event] = events();
expect(event).toMatchObject({ outcome: 'pending', correlation_id: null });
});
it('records an approved replay as an `approved` event with the grant approval id', async () => {
const grant = grantRow('9', 'groups-gated');
mockGetPendingApproval.mockReturnValue(grant);
const resp = await dispatch({ id: '9', command: 'groups-gated', args: {} }, AGENT_CTX, { grant });
expect(resp.ok).toBe(true);
const [event] = events();
expect(event).toMatchObject({
action: 'groups.gated',
outcome: 'approved',
correlation_id: 'appr-123-abc',
});
expect(event.resources).toContainEqual({ type: 'approval', id: 'appr-123-abc' });
});
it('records a --help probe under a neutral cli.help action, never the real verb', async () => {
const resp = await dispatch({ id: '1', command: 'groups-gated', args: { help: true } }, AGENT_CTX);
expect(resp.ok).toBe(true);
const [event] = events();
expect(event.action).toBe('cli.help');
expect(event.outcome).toBe('success');
expect(event.resources).toEqual([]);
});
it('records unknown commands as cli.unknown-command with the raw name in details', async () => {
await dispatch({ id: '1', command: 'nope-nothing', args: {} }, { caller: 'host' });
const [event] = events();
expect(event).toMatchObject({
action: 'cli.unknown-command',
outcome: 'failure',
resources: [],
details: { args: [], command: 'nope-nothing', error: 'unknown-command' },
});
});
it('records the resolved command and target id for dash-joined positional ids', async () => {
const uuid = '550e8400-e29b-41d4-a716-446655440000';
await dispatch({ id: '1', command: `groups-get-${uuid}`, args: {} }, { caller: 'host' });
const [event] = events();
expect(event).toMatchObject({ action: 'groups.get', outcome: 'success' });
expect(event.resources).toContainEqual({ type: 'agent_group', id: uuid });
// The id surfaces in resources; details keeps only the flag name.
expect(event.details.args).toContain('id');
expect(event.details.id).toBeUndefined();
});
it('normalizes hyphenated arg key names in details', async () => {
await dispatch({ id: '1', command: 'groups-test', args: { 'dry-run': 'true' } }, { caller: 'host' });
const [event] = events();
expect(event.details.args).toContain('dry_run');
expect(event.details.dry_run).toBeUndefined();
});
it('never stores arg values — a secret-bearing arg leaves only its key', async () => {
await dispatch(
{ id: '1', command: 'groups-test', args: { env: '{"NOTION_TOKEN":"tok-123","SAFE":"ok"}' } },
{ caller: 'host' },
);
const [event] = events();
expect(event.details.args).toContain('env');
expect(event.details.env).toBeUndefined();
// The secret never reaches the stored bytes.
expect(appended.lines[0]).not.toContain('NOTION_TOKEN');
expect(appended.lines[0]).not.toContain('tok-123');
});
});
@@ -0,0 +1,263 @@
/**
* CLI audit adapter (installed by /add-audit) owns how the dispatcher
* describes itself to the audit log: the dispatch middleware plus the
* CLI-specific actor/origin/resource mapping. Composed in dispatch.ts as
* `export const dispatch = withAudit(dispatchInner)`; business logic there
* contains zero audit calls.
*
* Recording model: the log stores WHO did WHICH action to WHAT target and the
* outcome never raw argument VALUES. `details` carries the arg key names
* that were passed plus a small allowlist of governance-relevant enum fields
* (role, mode, ) echoed verbatim; everything else is name-only. There is no
* value redactor because no free-form value is ever written a secret can't
* leak from a field that was never stored. Target identifiers (ids, users,
* groups) are structured and surface separately in `resources`.
*
* Loading this module also boots the audit log (writability assert, boot
* prune, hook lifecycle, maintenance timer): dispatch.ts is imported by both
* transports during the host's barrel phase, so initAuditLog() runs before
* any command is accepted and an enabled box with an unwritable
* data/audit/ refuses to start.
*/
import { emitAuditEvent } from '../audit/emit.js';
import { initAuditLog } from '../audit/init.js';
import {
type AuditActor,
type AuditEventInput,
type AuditOrigin,
type AuditOutcome,
type AuditResource,
} from '../audit/types.js';
import { containerOrigin, hostUser } from '../audit/vocab.js';
import { getPendingApprovalsByAction } from '../db/sessions.js';
import type { PendingApproval } from '../types.js';
import { getResource } from './crud.js';
import type { CallerContext, RequestFrame, ResponseFrame } from './frame.js';
import { commandGuardAction } from './guard.js';
import { type CommandDef, lookup } from './registry.js';
initAuditLog();
// ── CLI mapping ──
/**
* Host callers stamp `host:<install user>` daemon-side (the ncl socket is
* 0600 and owned by the install user); container callers are their agent group.
*/
export function actorForCaller(ctx: CallerContext): AuditActor {
return ctx.caller === 'host' ? { type: 'human', id: `host:${hostUser()}` } : { type: 'agent', id: ctx.agentGroupId };
}
export function originForCaller(ctx: CallerContext): AuditOrigin {
if (ctx.caller === 'host') return { transport: 'socket' };
return containerOrigin(ctx.sessionId, ctx.messagingGroupId || null);
}
/**
* Frame-level args use `--hyphen-keys`; recorded key names use the same
* underscore form the parsed handlers see. Mirrors crud's normalizeArgs
* (kept local so audit doesn't depend on a module tests commonly mock).
*/
export function normalizeArgKeys(raw: Record<string, unknown>): Record<string, unknown> {
const out: Record<string, unknown> = {};
for (const [k, v] of Object.entries(raw)) {
out[k.replace(/-/g, '_')] = v;
}
return out;
}
/**
* Arg values safe to record verbatim: governance-relevant enums that are never
* secrets and whose value IS the audit-worthy detail (which role was granted,
* which scope/mode was set). Every other value is dropped only the presence
* of the flag (its key) is kept so no free-form or secret-bearing value ever
* reaches disk.
*/
const SAFE_VALUE_FIELDS: ReadonlySet<string> = new Set([
'role',
'mode',
'session_mode',
'cli_scope',
'access',
'engage_mode',
'sender_scope',
'provider',
'model',
]);
/** CLI resource plural → audit resource type, where the singular isn't it. */
const RESOURCE_TYPE_OVERRIDES: Record<string, string> = {
groups: 'agent_group',
'messaging-groups': 'messaging_group',
'dropped-messages': 'dropped_message',
'user-dms': 'user_dm',
};
/**
* Derive touched/attempted resources from a command's args. Generic by design:
* `id` the command's own resource, group/user args their types, and a bare
* `{type}` entry when nothing else is known (a denied `users list` still names
* what was attempted). Ids are structured identifiers, not secrets.
*/
export function resourcesForCli(cmd: CommandDef, args: Record<string, unknown>): AuditResource[] {
if (!cmd.resource) return [];
const type = RESOURCE_TYPE_OVERRIDES[cmd.resource] ?? getResource(cmd.resource)?.name ?? cmd.resource;
const out: AuditResource[] = [];
const push = (t: string, id: unknown): void => {
if (typeof id !== 'string' || !id) return;
if (!out.some((r) => r.type === t && r.id === id)) out.push({ type: t, id });
};
push(type, args.id);
push('agent_group', args.agent_group_id ?? args.group);
push('user', args.user);
if (out.length === 0) out.push({ type });
return out;
}
// ── Command resolution, mirrored for the record ──
// Dispatch resolves the command on a local copy of the frame that never leaves
// it, so the middleware mirrors the one documented mechanic below. The mirror
// is mechanical, and drift only ever degrades a record's detail (a fallback
// action name) — never dispatch behavior, and never an outcome.
/**
* Mirror of dispatch's command resolution: exact lookup, then the longest
* registered dash-prefix with the remainder recorded as --id.
*/
function resolveForRecord(req: RequestFrame): { cmd?: CommandDef; args: Record<string, unknown> } {
const direct = lookup(req.command);
if (direct) return { cmd: direct, args: req.args };
let shortened = req.command;
let idx: number;
while ((idx = shortened.lastIndexOf('-')) > 0) {
shortened = shortened.slice(0, idx);
const fallback = lookup(shortened);
if (fallback) {
const tail = req.command.slice(shortened.length + 1);
return { cmd: fallback, args: { ...req.args, id: req.args.id ?? tail } };
}
}
return { args: req.args };
}
/**
* The approval row a hold just created for this frame it gives the pending
* event the same correlation_id the approved replay will carry as its guard
* grant. requestApproval keeps the minted id internal, so the row is
* recovered by the frame id it stored in its payload; no row (e.g. no
* configured approver) the hold is still recorded, uncorrelated.
*/
function holdApprovalIdFor(frameId: string): string | null {
const rows = getPendingApprovalsByAction('cli_command');
for (let i = rows.length - 1; i >= 0; i--) {
try {
const payload = JSON.parse(rows[i].payload) as { frame?: { id?: string } };
if (payload.frame?.id === frameId) return rows[i].approval_id;
// eslint-disable-next-line no-catch-all/no-catch-all -- a row with an unparseable payload is simply not this frame's hold
} catch {
continue;
}
}
return null;
}
// ── The dispatch middleware ──
type DispatchInner = (
req: RequestFrame,
ctx: CallerContext,
opts?: { grant?: PendingApproval },
) => Promise<ResponseFrame>;
/**
* Build the audit record for one dispatch. `res` is the response frame, or
* null when `inner` threw (`err` set) a crash still leaves a record.
*
* Outcome: ok success (or `approved` when a grant drove the replay),
* forbidden denied (captures pre-handler scope denials), approval-pending
* pending (the record of a hold), a thrown/other error failure. A `--help`
* probe is introspection, not the verb, so it records under a neutral
* `cli.help` action with no target never as the real command succeeding.
* Correlation is the approval id: a replay carries the row as its grant, and a
* fresh hold recovers the row it just created.
*/
function buildEvent(
req: RequestFrame,
ctx: CallerContext,
opts: { grant?: PendingApproval },
res: ResponseFrame | null,
err: unknown,
): AuditEventInput {
const resolved = resolveForRecord(req);
const cmd = resolved.cmd;
const normArgs = normalizeArgKeys(resolved.args);
const isHelp = req.args.help === true && !!res && res.ok;
const pending = !!res && !res.ok && res.error.code === 'approval-pending';
const approved = !!res && res.ok && !!opts.grant && !isHelp;
const outcome: AuditOutcome = !res
? 'failure' // inner threw
: res.ok
? approved
? 'approved'
: 'success'
: res.error.code === 'forbidden'
? 'denied'
: pending
? 'pending'
: 'failure';
// details: arg key names + allowlisted safe values only. No free-form value
// is ever stored, so nothing needs redacting and nothing can leak.
const details: Record<string, unknown> = { args: Object.keys(normArgs).sort() };
for (const f of SAFE_VALUE_FIELDS) {
if (normArgs[f] !== undefined) details[f] = normArgs[f];
}
if (err) {
details.error = 'exception'; // the throw's message is never stored
} else if (res && !res.ok && !pending) {
details.error = res.error.code; // the error CODE (a safe enum), never the free-text message
}
if (!cmd) details.command = req.command;
const correlationId = opts.grant?.approval_id ?? (pending ? holdApprovalIdFor(req.id) : null);
const action = isHelp ? 'cli.help' : cmd ? commandGuardAction(cmd) : 'cli.unknown-command';
const resources: AuditResource[] = isHelp ? [] : cmd ? resourcesForCli(cmd, normArgs) : [];
if (correlationId) resources.push({ type: 'approval', id: correlationId });
return {
actor: actorForCaller(ctx),
origin: originForCaller(ctx),
action,
resources,
outcome,
correlationId,
details,
};
}
/**
* Dispatch middleware the exported `dispatch` is the wrapped function, so
* the socket server, the container delivery-action, and the in-module
* approved replay are all covered by the one composition.
*
* The emit brackets `inner` so a thrown dispatcher still leaves a record: on
* throw we emit a `failure` event and re-raise unchanged, so an approval-gated
* command whose hold crashes on the DB write is not a silent governance gap.
*/
export function withAudit(inner: DispatchInner): DispatchInner {
return async (req, ctx, opts = {}) => {
let res: ResponseFrame;
try {
res = await inner(req, ctx, opts);
} catch (err) {
emitAuditEvent(() => buildEvent(req, ctx, opts, null