Compare commits

...

1 Commits

Author SHA1 Message Date
Gabi Simons d548980ddf fix(agent-runner): reconcile Claude tool allowlist with pinned CLI, add drift guard
TOOL_ALLOWLIST named five tools that don't exist on claude-code 2.1.197
(Task was renamed Agent upstream; TodoWrite, TeamCreate, TeamDelete, and
ToolSearch are gone). Remove the phantoms and correct the comment (and
the stale claim in docs/agent-runner-details.md): allowedTools is a
permission auto-approve list, not an availability filter — wire captures
show no allowlist effect on the stable tool surface, and this runner's
bypassPermissions moots its permission role. No wire-visible change.

Wire measurement on this pin shows PER-QUERY nondeterminism in the
offered surface: conditional tools (Glob/Grep) flicker, and
disallowedTools stripping of flag-gated tools is best-effort. The
fixture therefore captures N interleaved rounds per mode and asserts
equality on the STABLE cores, with the flicker set recorded separately
(variantTools) so regeneration cannot flake and a real allowlist-
semantics change fails deterministically. A disallow probe is recorded
as a diagnostic; the deterministic tool block remains the runner's
PreToolUse hook.

claude.tools.test.ts fails on any future claude-code CLI or Agent SDK
bump until the fixture is regenerated with dump-sdk-tools.ts and the
lists re-verified. The SDK dep is pinned exactly (0.3.197, was caret)
so drift can only enter deliberately.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-12 14:45:09 +03:00
8 changed files with 392 additions and 25 deletions
+1
View File
@@ -4,6 +4,7 @@ All notable changes to NanoClaw will be documented in this file.
## [Unreleased]
- **Claude tool allowlist reconciled with the pinned CLI, plus a tool-surface drift guard.** `TOOL_ALLOWLIST` in the agent runner named five tools that don't exist on claude-code 2.1.197 (`Task` — renamed `Agent` upstream — plus `TodoWrite`, `TeamCreate`, `TeamDelete`, `ToolSearch`); the phantom entries are removed and the comment corrected: `allowedTools` is a permission auto-approve list, **not an availability filter** — interleaved wire captures show no allowlist effect on the stable tool surface, and `bypassPermissions` moots its permission role. (The surface itself shows per-query variance in conditional tools on this CLI pin — the drift fixture records the flicker set separately so regeneration cannot flake on it — so neither `allowedTools` nor `disallowedTools` alone should be relied on to shape what the model sees — the runner's PreToolUse hook is the deterministic block.) No wire-visible behavior change. A new wire-captured fixture (`container/agent-runner/src/providers/sdk-tools-baseline.json`) and `claude.tools.test.ts` now fail on any future claude-code CLI **or SDK** pin bump until the fixture is regenerated with `dump-sdk-tools.ts` and the lists re-verified.
- [BREAKING] **`whatsapp-formatting` and `slack-formatting` container skills moved from trunk to the `channels` branch.** They now install with their channel — `/add-whatsapp` / `/add-slack` and the setup installers copy them in — so installs without those channels stop carrying channel-specific formatting instructions in every agent's context. **Migration — only if this install has the channel wired** (check `src/channels/whatsapp.ts` / `src/channels/slack.ts`): updating removes both skills from the working tree — WhatsApp agents lose the formatting fragment from their composed CLAUDE.md on next spawn, Slack agents lose the mrkdwn skill from `~/.claude/skills`. Re-run the matching skill, `/add-whatsapp` or `/add-slack` (idempotent), to restore. Installs without the channel need nothing — do NOT run the add-skill just in case; it installs the full channel adapter.
- **Pre-task script failures back their series off instead of spinning.** A `--script` that errors lands the occurrence as a failed run (`script-skip:error` ack → `failed` status); recurrence reads the series' trailing failed streak and re-arms at `max(cron next, now + 2·2^(n1) min, cap 60)`; after 8 consecutive failures the series is auto-paused with a host-written note in its run log (`ncl tasks resume` revives it). A deliberate `wakeAgent:false` gate is a normal run and never backs off. Also fixed: an explicitly-addressed `<message to>` in a task fire's final text now delivers as a deliberate send (previously suppressed as a turn-reply echo → zero delivery when the agent skipped the MCP tool); identical echoes of an MCP send are dropped in the runner, where the duplication originates.
- [BREAKING] **Scheduled tasks moved from MCP tools to `ncl tasks`.** The six scheduling MCP tools are no longer exposed to agent containers; agents and operators manage tasks with `ncl tasks list/get/create/update/cancel/pause/resume/delete`. New tasks run from a per-agent-group system session rather than waking the chat session that created them, and task writes are not approval-gated inside the owning group. **Migration:** [docs/ncl-tasks-migration.md](docs/ncl-tasks-migration.md).
+1 -1
View File
@@ -5,7 +5,7 @@
"": {
"name": "nanoclaw-agent-runner",
"dependencies": {
"@anthropic-ai/claude-agent-sdk": "^0.3.197",
"@anthropic-ai/claude-agent-sdk": "0.3.197",
"@anthropic-ai/sdk": "^0.108.0",
"@modelcontextprotocol/sdk": "^1.29.0",
"cron-parser": "^5.0.0",
+1 -1
View File
@@ -9,7 +9,7 @@
"test": "bun test"
},
"dependencies": {
"@anthropic-ai/claude-agent-sdk": "^0.3.197",
"@anthropic-ai/claude-agent-sdk": "0.3.197",
"@anthropic-ai/sdk": "^0.108.0",
"@modelcontextprotocol/sdk": "^1.29.0",
"cron-parser": "^5.0.0",
@@ -0,0 +1,82 @@
/**
* Drift guard for the harness tool surface. sdk-tools-baseline.json is a wire
* capture of every tool the pinned CLI can offer under our configuration
* (regenerate with dump-sdk-tools.ts — instructions in its header). These
* tests catch upstream renames/removals when the claude-code pin moves:
* bumping container/cli-tools.json fails the version assertion until the
* fixture is regenerated and the lists below are re-verified.
*/
import fs from 'fs';
import { describe, expect, it } from 'bun:test';
import cliTools from '../../../cli-tools.json';
import { SDK_DISALLOWED_TOOLS, TOOL_ALLOWLIST } from './claude.js';
import baseline from './sdk-tools-baseline.json';
/**
* Disallow entries that do NOT exist on the pinned CLI in headless SDK mode
* (wire-verified: never offered, in both string and streaming input modes).
* Kept in SDK_DISALLOWED_TOOLS as drift insurance — if an upstream version
* starts offering one, the fixture regeneration surfaces it here and the
* entry moves out of this set.
*/
const KNOWN_ABSENT_DISALLOWED = ['AskUserQuestion', 'EnterPlanMode', 'ExitPlanMode'];
const installedSdkVersion = (
JSON.parse(
fs.readFileSync(new URL('../../node_modules/@anthropic-ai/claude-agent-sdk/package.json', import.meta.url), 'utf8'),
) as { version: string }
).version;
// Membership checks run against stable variant: a tool that flickers on
// this pin (see dump-sdk-tools.ts header) is still a real tool.
const baselineTools = new Set<string>([...baseline.tools, ...baseline.variantTools]);
describe('sdk tool-surface drift guard', () => {
it('fixture matches the pinned claude-code CLI version', () => {
const pin = cliTools.find((t) => t.name === '@anthropic-ai/claude-code')?.version;
expect(baseline.cliVersion).toBe(pin);
});
it('fixture matches the installed Agent SDK version', () => {
// The SDK is a caret-free pinned dep; a bump must be captured deliberately.
expect(baseline.sdkVersion).toBe(installedSdkVersion);
});
it('allowedTools has no surface effect: the stable cores match across interleaved rounds', () => {
// `tools`/`toolsBare` are STABLE cores over N interleaved captures per
// mode; per-query flicker (see the dump-sdk-tools.ts header) lands in
// `variantTools` instead of here. Passing TOOL_ALLOWLIST has never been
// observed to filter or promote a stable tool under bypassPermissions —
// if a future CLI makes allowedTools an availability filter, the cores
// diverge in every round and this fails deterministically.
expect([...baseline.tools].sort()).toEqual([...baseline.toolsBare].sort());
});
it('every allowlist entry names a real tool on this surface', () => {
for (const name of TOOL_ALLOWLIST) {
expect(baselineTools.has(name), `allowlist entry '${name}' not in captured surface`).toBe(true);
}
});
it('every disallow entry is either a real tool or documented drift insurance', () => {
for (const name of SDK_DISALLOWED_TOOLS) {
const real = baselineTools.has(name);
const insurance = KNOWN_ABSENT_DISALLOWED.includes(name);
expect(
real || insurance,
`disallow entry '${name}' is neither on the surface nor in KNOWN_ABSENT_DISALLOWED`,
).toBe(true);
}
});
it('drift-insurance entries are still absent from the surface', () => {
for (const name of KNOWN_ABSENT_DISALLOWED) {
expect(
baselineTools.has(name),
`'${name}' now exists on the surface — move it out of KNOWN_ABSENT_DISALLOWED and re-verify its disposition`,
).toBe(false);
}
});
});
+22 -19
View File
@@ -24,7 +24,7 @@ function log(msg: string): void {
// the question and blocks on the real reply.
// - EnterPlanMode / ExitPlanMode / EnterWorktree / ExitWorktree: Claude
// Code UI affordances; in a headless container they'd appear stuck.
const SDK_DISALLOWED_TOOLS = [
export const SDK_DISALLOWED_TOOLS = [
'CronCreate',
'CronDelete',
'CronList',
@@ -36,30 +36,33 @@ const SDK_DISALLOWED_TOOLS = [
'ExitWorktree',
];
// Tool allowlist for NanoClaw agent containers. MCP-tool entries are derived
// at the call site from the registered `mcpServers` map so that any server
// added via `add_mcp_server` (or wired in container.json directly) is
// reachable to the agent — without this, the SDK's allowedTools filter
// silently drops every MCP namespace not listed here.
const TOOL_ALLOWLIST = [
// Pre-approved tool set for NanoClaw agent containers. `allowedTools` is a
// permission auto-approve list, NOT an availability filter: interleaved wire
// captures show no allowlist effect on the stable tool surface (the fixture's
// stable cores are equal; claude.tools.test.ts asserts it), and its permission
// role is moot under this runner's `bypassPermissions`. The surface itself has
// per-query variance in conditional tools (see dump-sdk-tools.ts header), so
// never rely on this list — or on `disallowedTools` alone — to shape what the
// model sees. Kept as the accurate pre-approval set for a
// hypothetical non-bypass mode, and because the per-server `mcpAllowPattern`
// entries derived at the call site are retained (MCP invocation-gating under
// non-bypass modes is unverified). Exported for the fixture regenerator and
// tests.
export const TOOL_ALLOWLIST = [
'Agent',
'Bash',
'Read',
'Write',
'Edit',
'Glob',
'Grep',
'WebSearch',
'WebFetch',
'Task',
'NotebookEdit',
'Read',
'SendMessage',
'Skill',
'TaskOutput',
'TaskStop',
'TeamCreate',
'TeamDelete',
'SendMessage',
'TodoWrite',
'ToolSearch',
'Skill',
'NotebookEdit',
'WebFetch',
'WebSearch',
'Write',
];
// MCP server names are sanitized by the SDK when forming tool prefixes:
@@ -0,0 +1,182 @@
/**
* Regenerates sdk-tools-baseline.json — the bare SDK tool-surface fixture
* asserted by claude.tools.test.ts.
*
* Must run INSIDE the agent container image (the pinned CLI binary only
* exists there). From the repo root:
*
* docker run --rm --network none \
* -v "$PWD/container/agent-runner/src":/app/src:ro \
* --entrypoint bun <nanoclaw-agent image> /app/src/providers/dump-sdk-tools.ts \
* > container/agent-runner/src/providers/sdk-tools-baseline.json
*
* MEASURED NONDETERMINISM on the pinned CLI (2.1.197): across query
* invocations with byte-identical options — even inside one process — (a)
* conditional tools (Glob, Grep) flicker in and out of the surface, and (b)
* `disallowedTools` sometimes strips flag-gated tools (Workflow, DesignSync,
* EnterWorktree) and sometimes ignores them entirely; each single query is
* internally coherent. Consequences: schema-stripping via disallowedTools is
* BEST-EFFORT — the deterministic enforcement is the runner's PreToolUse
* hook, which blocks the invocation regardless of whether the schema shipped.
*
* Because the variance is per-query, a single capture pair cannot isolate the
* allowlist's effect. So: ROUNDS interleaved captures per mode (allowlist /
* bare), then split each mode into its STABLE core (tools present in every
* capture of that mode) and the VARIANT set (present in some but not all
* captures across both modes):
* - `tools` : stable core WITH the production TOOL_ALLOWLIST.
* - `toolsBare` : stable core with no allowedTools.
* - `variantTools` : the flicker set, recorded for membership checks.
* - `toolsDisallowProbe` : one capture with disallowedTools=[DISALLOW_PROBE]
* — a diagnostic of which disallow behavior this regen observed (no test
* asserts stripping; a fixed assertion on a nondeterministic mechanism
* would be a coin-flip).
* The drift test asserts tools === toolsBare on the STABLE cores: no allowlist
* effect has ever been observed there, so the list is permission-layer only
* (moot under bypassPermissions). If a CLI/SDK bump makes the allowlist shape
* the surface, the stable cores diverge across every round by construction
* and the assertion fails deterministically instead of flaking.
*
* Agent-teams is enabled via a temp settings.json (wire-verified: settings
* env strictly beats SDK options env).
*
* Zero API traffic: ANTHROPIC_BASE_URL points at an in-process stub answering
* 401; the full tools array rides on the first /v1/messages request, captured
* before the run dies on the auth error. The fixture records WIRE tool names
* (the SDK init message reports legacy aliases, e.g. `Task` for wire `Agent` —
* do not swap this to an init capture).
*/
import { execFileSync } from 'child_process';
import fs from 'fs';
import { query } from '@anthropic-ai/claude-agent-sdk';
import { TOOL_ALLOWLIST } from './claude.js';
let requests: string[] = [];
let captured: (() => void) | null = null;
const server = Bun.serve({
hostname: '127.0.0.1',
port: 0,
async fetch(req) {
const url = new URL(req.url);
const body = await req.text();
if (url.pathname.includes('/messages')) {
requests.push(body);
captured?.();
}
return new Response(
JSON.stringify({ type: 'error', error: { type: 'authentication_error', message: 'fixture-capture-stub' } }),
{ status: 401, headers: { 'content-type': 'application/json' } },
);
},
});
const HOME = '/tmp/dump-sdk-tools-home';
const CWD = '/tmp/dump-sdk-tools-ws';
fs.mkdirSync(`${HOME}/.claude`, { recursive: true });
fs.mkdirSync(CWD, { recursive: true });
fs.writeFileSync(
`${HOME}/.claude/settings.json`,
JSON.stringify({ env: { CLAUDE_CODE_EXPERIMENTAL_AGENT_TEAMS: '1' } }, null, 2),
);
/**
* Diagnostic probe for disallowedTools: a flag-gated tool whose stripping is
* nondeterministic on the current pin (see header). The fixture records what
* this regen run observed; future pins can be compared against it.
*/
export const DISALLOW_PROBE = 'Workflow';
/** Run one capture and return the sorted wire tool names. */
async function capture(opts?: { allowedTools?: string[]; disallowedTools?: string[] }): Promise<string[]> {
requests = [];
const firstRequest = new Promise<void>((resolve) => {
captured = resolve;
});
const q = query({
prompt: 'fixture capture: reply with one word',
options: {
cwd: CWD,
pathToClaudeCodeExecutable: '/pnpm/claude',
systemPrompt: { type: 'preset' as const, preset: 'claude_code' as const },
env: {
...process.env,
HOME,
ANTHROPIC_BASE_URL: `http://127.0.0.1:${server.port}`,
ANTHROPIC_API_KEY: 'fixture-dummy-key',
ANTHROPIC_AUTH_TOKEN: undefined,
},
permissionMode: 'bypassPermissions',
allowDangerouslySkipPermissions: true,
settingSources: ['user'],
...(opts?.allowedTools ? { allowedTools: opts.allowedTools } : {}),
...(opts?.disallowedTools ? { disallowedTools: opts.disallowedTools } : {}),
},
});
void (async () => {
try {
for await (const _m of q) {
/* drain until the auth error kills the run */
}
} catch {
/* expected: 401 from the stub */
}
})();
await Promise.race([firstRequest, Bun.sleep(75_000)]);
await Bun.sleep(1_500); // let retries land so we can pick the largest body
if (requests.length === 0) {
console.error('[dump-sdk-tools] no /v1/messages request captured');
process.exit(1);
}
const biggest = requests.reduce((a, b) => (b.length > a.length ? b : a));
const parsed = JSON.parse(biggest) as { tools?: Array<{ name: string }> };
return [...new Set((parsed.tools ?? []).map((t) => t.name))].sort();
}
// Interleaved so a drifting gate state biases both modes equally (see header).
const ROUNDS = 3;
const withRuns: string[][] = [];
const bareRuns: string[][] = [];
for (let i = 0; i < ROUNDS; i++) {
withRuns.push(await capture({ allowedTools: TOOL_ALLOWLIST }));
bareRuns.push(await capture());
}
const toolsDisallowProbe = await capture({ disallowedTools: [DISALLOW_PROBE] });
const stable = (runs: string[][]): string[] => runs[0].filter((t) => runs.every((r) => r.includes(t))).sort();
const union = (runs: string[][]): Set<string> => new Set(runs.flat());
const tools = stable(withRuns);
const toolsBare = stable(bareRuns);
const stableSet = new Set([...tools, ...toolsBare]);
const variantTools = [...union([...withRuns, ...bareRuns])].filter((t) => !stableSet.has(t)).sort();
const cliVersionRaw = execFileSync('/pnpm/claude', ['--version'], { encoding: 'utf8' }).trim();
const cliVersion = cliVersionRaw.split(/\s+/)[0];
const sdkVersion = (
JSON.parse(fs.readFileSync('/app/node_modules/@anthropic-ai/claude-agent-sdk/package.json', 'utf8')) as {
version: string;
}
).version;
console.log(
JSON.stringify(
{
cliVersion,
sdkVersion,
capturedAt: new Date().toISOString(),
capture: `wire names; stable cores over ${ROUNDS} interleaved rounds per mode (tools=production allowlist, toolsBare=no allowedTools); variantTools=flicker set; toolsDisallowProbe=single capture with disallowedTools:[probe]; teams on`,
rounds: ROUNDS,
disallowProbe: DISALLOW_PROBE,
tools,
toolsBare,
variantTools,
toolsDisallowProbe,
},
null,
2,
),
);
process.exit(0);
@@ -0,0 +1,96 @@
{
"cliVersion": "2.1.197",
"sdkVersion": "0.3.197",
"capturedAt": "2026-07-12T11:43:58.342Z",
"capture": "wire names; stable cores over 3 interleaved rounds per mode (tools=production allowlist, toolsBare=no allowedTools); variantTools=flicker set; toolsDisallowProbe=single capture with disallowedTools:[probe]; teams on",
"rounds": 3,
"disallowProbe": "Workflow",
"tools": [
"Agent",
"Bash",
"CronCreate",
"CronDelete",
"CronList",
"DesignSync",
"Edit",
"EnterWorktree",
"ExitWorktree",
"Glob",
"Grep",
"NotebookEdit",
"Read",
"ReportFindings",
"ScheduleWakeup",
"SendMessage",
"Skill",
"TaskCreate",
"TaskGet",
"TaskList",
"TaskOutput",
"TaskStop",
"TaskUpdate",
"WebFetch",
"WebSearch",
"Workflow",
"Write"
],
"toolsBare": [
"Agent",
"Bash",
"CronCreate",
"CronDelete",
"CronList",
"DesignSync",
"Edit",
"EnterWorktree",
"ExitWorktree",
"Glob",
"Grep",
"NotebookEdit",
"Read",
"ReportFindings",
"ScheduleWakeup",
"SendMessage",
"Skill",
"TaskCreate",
"TaskGet",
"TaskList",
"TaskOutput",
"TaskStop",
"TaskUpdate",
"WebFetch",
"WebSearch",
"Workflow",
"Write"
],
"variantTools": [],
"toolsDisallowProbe": [
"Agent",
"Bash",
"CronCreate",
"CronDelete",
"CronList",
"DesignSync",
"Edit",
"EnterWorktree",
"ExitWorktree",
"Glob",
"Grep",
"NotebookEdit",
"Read",
"ReportFindings",
"ScheduleWakeup",
"SendMessage",
"Skill",
"TaskCreate",
"TaskGet",
"TaskList",
"TaskOutput",
"TaskStop",
"TaskUpdate",
"WebFetch",
"WebSearch",
"Workflow",
"Write"
]
}
+7 -4
View File
@@ -148,9 +148,12 @@ class ClaudeProvider implements AgentProvider {
systemPrompt: input.systemContext?.instructions
? { type: 'preset', preset: 'claude_code', append: input.systemContext.instructions }
: undefined,
// Base tools plus one `mcp__<server>__*` pattern per registered MCP
// server — without the explicit MCP patterns the SDK's allowedTools
// filter silently drops every MCP namespace.
// Permission auto-approve list (NOT an availability filter — wire
// captures show no allowlist effect on the offered tool surface, and
// bypassPermissions moots its permission role). Kept accurate for a
// hypothetical non-bypass mode; the `mcp__<server>__*` patterns are
// retained because MCP invocation-gating under non-bypass modes is
// unverified. See sdk-tools-baseline.json + claude.tools.test.ts.
allowedTools: [...TOOL_ALLOWLIST, ...Object.keys(this.mcpServers).map(mcpAllowPattern)],
disallowedTools: SDK_DISALLOWED_TOOLS,
env: this.env,
@@ -195,7 +198,7 @@ SDK message (so the idle timer stays honest) and maps recognized messages to `Pr
**Claude-specific behavior inside the provider:**
- `MessageStream` for async iterable input (push-based follow-ups)
- Resume via the SDK `resume` option keyed on the stored `continuation` (the SDK session ID) — no separate resume-at cursor
- `TOOL_ALLOWLIST` (Bash, Read, Write, Edit, Glob, Grep, WebSearch, WebFetch, Task, Skill, …) extended at the call site with a `mcp__<server>__*` pattern per registered MCP server; `SDK_DISALLOWED_TOOLS` blocks SDK builtins that collide with NanoClaw's own scheduling/interaction model (CronCreate/Delete/List, ScheduleWakeup, AskUserQuestion, Enter/ExitPlanMode, Enter/ExitWorktree)
- `TOOL_ALLOWLIST` (Agent, Bash, Read, Write, Edit, Glob, Grep, WebSearch, WebFetch, Skill, …) — a permission auto-approve list, not an availability filter (moot under `bypassPermissions`; wire-verified, see `claude.tools.test.ts`) — extended at the call site with a `mcp__<server>__*` pattern per registered MCP server; `SDK_DISALLOWED_TOOLS` blocks SDK builtins that collide with NanoClaw's own scheduling/interaction model (CronCreate/Delete/List, ScheduleWakeup, AskUserQuestion, Enter/ExitPlanMode, Enter/ExitWorktree)
- **PreToolUse hook** records the current tool + its declared timeout to `container_state` (so the host sweep widens its stuck tolerance while a long Bash runs) and, as defense-in-depth, blocks any `SDK_DISALLOWED_TOOLS` call that slips through. It does **not** sanitize bash env vars — there is no such hook.
- **PostToolUse / PostToolUseFailure** hooks clear the in-flight tool
- **PreCompact** hook archives the transcript to `conversations/` before compaction