AIStoreHub Package Vault: Pi
Package Slug: pi
Archetype: agent
Upstream Repository: https://github.com/earendil-works/pi
Workspace Files & Specifications (68 files)
- AGENTS.md (14535 bytes)
# Development Rules ## Conversational Style - Keep answers short and concise - No emojis in commits, issues, PR comments, or code - No fluff or cheerful filler text (e.g., "Thanks @user" not "Thanks so much @user!") - Technical prose only, be direct - Use concise, clear, simple language. Define unavoidable jargon before using it. - Explain non-trivial designs and problems as: problem, concrete example or short trace, then solution. State why the solution is necessary and distinguish it from optional complexity. - Prefer concrete behavior and small illustrations over abstract summaries, dense terminology, or unexplained lists of changes. - When the user asks a question, answer it first before making edits or running implementation commands. - When responding to user feedback or an analysis, explicitly say whether you agree or disagree before saying what you changed. ## Code Quality - Read files in full before wide-ranging changes, before editing files you have not fully inspected, and when asked to investigate or audit. Do not rely on search snippets for broad changes. - Before recommending a fix design, check how sibling implementations (other adapters, providers, commands) handle the same concern, and follow that pattern. If you deviate, say why. - No `any` unless absolutely necessary. - Never tell the members of a typed union apart with `in` (`"report" in entry`): make it a tagged union (a `kind` field) and switch on the tag. `in` is only for values of unknown type at a trust boundary (wire input, files, Node errors), for feature detection, and for stored or frozen wire formats that a version bump would have to change (say so in a comment). - No runtime checks of values whose types TypeScript already guarantees (no `typeof`/`instanceof`/`Array.isArray`/shape guards on typed parameters, typed callback results or typed declarations). Validate only at decided trust boundaries (wire input, schema-checked arguments and params). - Inline single-line helpers that have only one call site. - Check node_modules for external API types; don't guess. - **No inline imports** (`await import()`, `import("pkg").Type`, dynamic type imports). Top-level imports only. - In `packages/coding-agent`, resolve package assets through helpers in `src/config.ts`. Do not use `__dirname` directly; the helpers account for source checkouts, npm installations, and standalone binaries. - Never remove or downgrade code to fix type errors from outdated deps; upgrade the dep instead. - Use only erasable TypeScript syntax (Node strip-only mode) in code checked by the root config (`packages/*/src`, `packages/*/test`, `packages/coding-agent/examples`): no parameter properties, `enum`, `namespace`/`module`, `import =`, `export =`, or other constructs needing JS emit. Use explicit fields with constructor assignments. - Always ask before removing functionality or code that appears intentional. - Do not preserve backward compatibility unless the user asks for it. - Never hardcode key checks (e.g. `matchesKey(keyData, "ctrl+x")`). Add defaults to `DEFAULT_EDITOR_KEYBINDINGS` or `DEFAULT_APP_KEYBINDINGS` so they stay configurable. - Never modify `packages/ai/src/models.generated.ts` directly; update the catalog definitions in `packages/ai/catalog/` instead (see its README), then regenerate. Including the resulting `models.generated.ts` diff is always OK, even if regeneration includes unrelated upstream model metadata changes. The generated model data in `packages/ai/src/providers/data/` is gitignored: never stage it, and don't look for it in git history. ## Chord v2 Code Style Binding for Chord v2 (`packages/chord/src/`) and its tests, when writing and when reviewing. The model is pi-durable's source. - Files: one concept per file, named after it; no one-helper files, no grab-bag files. First line: a comment saying what the file is for. Order: imports, exported types, then the public entry point first and its helpers below it in the order they are used; constants right above their first user. A blank line between top-level declarations. Split files by concept, never by size: one concept may take a long file (up to about 1,000 lines is fine); a second concept in a file is the reason to split. - Functions: every export has a one-line doc comment saying what it does or guarantees. Inline comments explain a non-obvious why (cost, race, decided rule), never what the next line does. Short functions with one job (about 40 lines needs a reason; one `switch` over a tagged union is fine). No boolean flag parameters that switch behaviour: two functions or a tagged value. Never write the same lines twice. Context last: required args, then `context`/`ctx` (always required, as in Durable; optional only on error-reporting paths), then optional args. - Types and checks: tagged unions for every state and result. Casts only where TypeScript cannot express a decided invariant, each with a short why. Every remaining runtime check names the trust boundary it serves (wire input, schema-checked params and arguments, declaration policy the types cannot express). Same-host values are plain data, immutable by convention: never copied, frozen or walked to enforce it; freeze only Chord-owned metadata when a decision says so. No defensive code against broken adapters, callers or our own modules. - Cost and memory: no hot-path work a decision did not ask for (walks, copies, compilations, per-call allocations). The function doing counted work counts it through its context; call sites never count for others. Caches are `WeakMap`s keyed by their owner, each with a GC test; no module-level strong caches. - Names and API: the design's vocabulary (`params`, `binding`, `principal key`), no invented synonyms; decided `ChordError` codes and reasons. Export only what a later layer or app needs. No dev or test modes, no unrequested options. - Tests: one test file per source file, one `describe` per mechanism, the layer spec's test list case by case; cost guards through the in-memory telemetry adapter; GC tests for every cache and binding. Every slice also adds one or more runnable examples in `packages/chord/test/examples/NN-topic.ts`, one topic each, numbered across slices, written as references for Chord's users (Durable's `test/examples/` format: header with what it shows and how to run it, commented steps, printed results). - Slices: Chord v2 is built in vertical slices (`packages/chord/docs/slices.md`). Implementers and reviewers read the slice spec, the feature map in `slices.md` and the whole of `packages/chord/docs/design.md` in full, and check the work against all three; slice specs cite design sections for orientation, not as a reading limit. - Deferred work: anything left to a later slice goes into the section "Carry-over" of `packages/chord/docs/slices.md` under that slice, in the same change; a slice's spec and build include every entry of its section. - Before reporting: re-read every changed file top to bottom as a stranger would; each file must be understandable without jumping around. Then `npm run check` and the layer's tests. ## Reviews - When reviewers (subagents or other sessions) report while others are still reviewing, do not summarize or relay their findings piecemeal. Wait until every report is in, then give one report that covers all of them. ## Commands - After code changes (not docs): `npm run check` (full output, no tail). Fix all errors, warnings, and infos before committing. Does not run tests. - `apps/p2` is outside the root check and the root tests. After p2 changes also run `cd apps/p2 && npm run check`; p2's suite is `cd apps/p2 && npm test` (TUI tests need Node 26, browser tests need `CHROMIUM_BIN`). - Run `npm run build` whenever something needs built output (`dist/`): tests or benchmarks that import built packages, Durable's workerd suite. The full suites of Chord and Durable need it: Chord's tests that start Node processes or bundle with esbuild load workspace packages from `dist/` (the rest run from source by its vitest config), and Durable imports pi-ai's `utils/*`, which have no source export. Never use it to type check or lint: that is `npm run check`, the preferred check. - Never run `npm test` unless requested by the user. - Never run the full vitest suite directly: it includes e2e tests that activate when endpoint/auth env vars are present. For all non-e2e tests, run `./test.sh` from the repo root. It runs `packages/*` only. Otherwise run specific tests from the package root: - Vitest: `node "$(git rev-parse --show-toplevel)/node_modules/vitest/dist/cli.js" --run test/specific.test.ts` - `packages/tui` (`node:test`): `node --test test/specific.test.ts` - If you create or modify a test file, run it and iterate on test or implementation until it passes. - For `packages/coding-agent/test/suite/`, use `test/suite/harness.ts` + the faux provider. No real provider APIs, keys, or paid tokens. - When regressions tests for fixing a github issue, add a comment with the github issue number next to the test. - For ad-hoc scripts, `write` them to a temp file (e.g. `/tmp`), run, edit if needed, remove when done. Don't embed multi-line scripts in `bash` commands. - Never commit unless the user asks. ## Dependency and Install Security - Treat npm dep and lockfile changes as reviewed code. Direct registry deps, including internal workspace deps, stay pinned to exact versions. - When updating `undici`, you MUST read its changelog/release notes for the target version and evaluate whether any changes may affect functionality before applying the update. - Hydrate/update locally with `npm install --ignore-scripts`; clean/CI-style with `npm ci --ignore-scripts`. Don't run lifecycle scripts unless the user asks. - If dep metadata changes, refresh `package-lock.json` with `npm install --package-lock-only --ignore-scripts`. - If `packages/coding-agent/install-lock/` needs regen, run `node scripts/generate-coding-agent-install-lock.mjs` (verify with `--check` or `npm run check`). New deps with lifecycle scripts require review and an explicit allowlist entry in that script; never add one silently. - Pre-commit blocks lockfile commits unless `PI_ALLOW_LOCKFILE_CHANGE=1`. Don't bypass unless the user wants the lockfile change committed. ## Git Multiple pi sessions may be running in this cwd at the same time, each modifying different files. Git operations that touch unstaged, staged, or untracked files outside your own changes will stomp on other sessions' work. Follow these rules: Committing: - Only commit files YOU changed in THIS session. - Stage explicit paths (`git add <path1> <path2>`); never `git add -A` / `git add .`. - Before committing, run `git status` and verify you are only staging your files. - `packages/ai/src/models.generated.ts` may always be included alongside your files. - Message format: `{feat,fix,docs}[(ai,tui,agent,coding-agent)]: <commit message> (optionally multiple lines)`. Message is informative and concise. Never run (destroys other agents' work or bypasses checks): - `git reset --hard`, `git checkout .`, `git clean -fd`, `git stash`, `git add -A`, `git add .`, `git commit --no-verify`. If rebase conflicts occur: - Resolve conflicts only in files you modified. - If a conflict is in a file you did not modify, abort and ask the user. - Never force push. ## Issues and PRs See `CONTRIBUTING.md` for the contributor gate (auto-close workflows, `lgtm`/`lgtmi`, quality bar). When reviewing PRs: - Do not run `gh pr checkout`, `git switch`, or otherwise move the worktree to the PR branch unless the user explicitly asks. - Use `gh pr view`, `gh pr diff`, `gh api`, and local `git show`/`git diff` against fetched refs to inspect PR metadata, commits, and patches without changing branches. - If you need PR file contents, fetch/read them into temporary files or use `git show <ref>:<path>` without switching branches. When creating issues: - Add `pkg:*` labels for affected packages (`pkg:agent`, `pkg:ai`, `pkg:coding-agent`, `pkg:tui`); use all that apply. When posting issue/PR comments: - Write the comment to a temp file and post with `gh issue/pr comment --body-file` (never multi-line markdown via `--body`). - Keep comments concise, technical, in the user's tone. - End every AI-posted comment with the AI-generated disclaimer line specified by the originating prompt (e.g. `This comment is AI-generated by `/wr``). When closing issues via commit: - Include `fixes #<number>` or `closes #<number>` in the message so merging auto-closes the issue. For multiple issues, repeat the keyword per issue (`closes #1, closes #2`); a shared keyword (`closes #1, #2`) only closes the first. ## Writing p2 plugin code Before writing or reviewing plugin code in `apps/p2` (server and worker facets, contracts, stores), read [apps/p2/docs/plugin-practices.md](apps/p2/docs/plugin-practices.md) in full and follow it. ## Working on p3 (Pi Roam) packages Work on a p3 (Pi Roam) package in `apps/p3` starts by reading [apps/p3/docs/README.md](apps/p3/docs/README.md), then the guides it lists, in its order, in full. Follow them. ## Building p2's web UI For building or changing p2's web UI (`apps/p2`), load and follow [.pi/skills/p2-ui.md](.pi/skills/p2-ui.md). ## Testing pi Interactive Mode with tmux For testing pi's interactive mode, load and follow [.pi/skills/interactive-testing.md](.pi/skills/interactive-testing.md). ## Changelog Location: `packages/*/CHANGELOG.md` (one per package). Sections under `## [Unreleased]`: `### Breaking Changes` (API changes requiring migration), `### Added`, `### Changed`, `### Fixed`, `### Removed`. Rules: - All new entries go under `## [Unreleased]`. Read the full section first and append to existing subsections; never duplicate them. - Released version sections (e.g. `## [0.12.2]`) are immutable; never modify them. - Do not create changelog entries when the work targets a branch other than `main` Attribution: - Internal (from issues): `Fixed foo bar ([#123](https://github.com/earendil-works/pi/issues/123))` - External contributions: `Added feature X ([#456](https://github.com/earendil-works/pi/pull/456) by [@username](https://github.com/username))` ## Releasing For release preparation, publishing, verification, or recovery, load and follow [.pi/skills/release.md](.pi/skills/release.md). ## User Override If the user's instructions conflict with any rule in this document, ask for explicit confirmation before overriding. Only then execute their instructions. - aistorehub.json (426 bytes)
{ "name": "Pi", "slug": "pi", "version": "1.0.0", "type": "agent", "description": "AI agent toolkit: unified LLM API, agent loop, TUI, coding agent CLI", "owner": "community", "repository": "catalog", "github_repo_url": "https://github.com/earendil-works/pi", "tags": [ "trending", "earendil-works", "typescript", "github", "open-source" ], "imported_at": "2026-11-21T23:56:28.453Z" } - apps/p2-mac/Makefile (2545 bytes)
# Builds and runs Pi.app in the foreground (Ctrl-C quits it and stops a server it started). See README.md. # # make run own server, local only: port 8798, data in ~/.pi/p2-mac/local (FAKE=1: faux provider) # make run-radius own server connected to Radius: port 8799, data in ~/.pi/p2-mac/radius (signs in on first use) # make attach the p2 already running on 127.0.0.1:8788 with ~/.pi/p2 (ATTACH_PORT, ATTACH_DATA_DIR) # # p2 runs one server per data directory and Radius is on exactly when <data>/radius.json exists, so each own-server # target gets its own data directory and port and leaves the server on 8788 alone. CHECKOUT ?= $(abspath ../p2) APP_BIN := build/Pi.app/Contents/MacOS/Pi LOCAL_PORT ?= 8798 LOCAL_DATA_DIR ?= $(HOME)/.pi/p2-mac/local RADIUS_PORT ?= 8799 RADIUS_DATA_DIR ?= $(HOME)/.pi/p2-mac/radius ATTACH_PORT ?= 8788 ATTACH_DATA_DIR ?= $(HOME)/.pi/p2 FAKE ?= # `-source own` overrides a server chosen in the menu (saved as `source=attached`, which wins over P2MAC_PORT) for this # launch only: it lands in the argument domain of the user defaults, which is never persisted. OWN_ENV = P2MAC_CHECKOUT="$(CHECKOUT)" P2MAC_FAKE="$(FAKE)" .PHONY: build run run-radius radius-login attach clean checkout-ready build: ./scripts/build-app.sh # A fresh worktree has no dependencies: its server would exit at once (ERR_MODULE_NOT_FOUND) behind "Pi is not running". REPO_ROOT = $(abspath $(CHECKOUT)/../..) checkout-ready: @test -d "$(REPO_ROOT)/node_modules" && test -d "$(REPO_ROOT)/packages/ai/src/providers/data" || { \ echo "$(REPO_ROOT) is not ready to run p2. Run there:"; \ echo " npm install --ignore-scripts && npm run hydrate:model-data"; exit 1; } run: build checkout-ready mkdir -p -m 700 "$(LOCAL_DATA_DIR)" $(OWN_ENV) P2MAC_PORT=$(LOCAL_PORT) P2MAC_DATA_DIR="$(LOCAL_DATA_DIR)" $(APP_BIN) -source own run-radius: build checkout-ready @test -f "$(RADIUS_DATA_DIR)/radius.json" || $(MAKE) radius-login $(OWN_ENV) P2MAC_PORT=$(RADIUS_PORT) P2MAC_DATA_DIR="$(RADIUS_DATA_DIR)" $(APP_BIN) -source own # Device-code sign-in to Radius for the radius data directory (PI_RADIUS_GATEWAY picks another gateway). radius-login: mkdir -p -m 700 "$(RADIUS_DATA_DIR)" cd "$(CHECKOUT)" && P2_DATA_DIR="$(RADIUS_DATA_DIR)" npm run radius:login attach: build @curl -fsS "http://127.0.0.1:$(ATTACH_PORT)/health" >/dev/null || { echo "No p2 answers on 127.0.0.1:$(ATTACH_PORT)"; exit 1; } P2MAC_ATTACH=$(ATTACH_PORT) P2MAC_DATA_DIR="$(ATTACH_DATA_DIR)" $(APP_BIN) clean: rm -rf build .build - apps/p2-mac/README.md (17418 bytes)
# p2-mac (spike) A native macOS app for p2. It has two parts: - **A menu bar item** that finds or runs the local p2 server. - **Windows** that show p2's web UI in a WKWebView, signed in as the local owner without a token to paste. It is AppKit with WKWebView, macOS 14 or later, and has no dependencies. The plan is the session artifact `pi-desktop-plan`. ## Build and run ```sh cd apps/p2-mac ./scripts/build-app.sh # build/Pi.app (debug, ad-hoc signed); CONFIG=release for an optimized build open build/Pi.app ``` Or use the Makefile, which builds and runs the app in the foreground against this repository's `apps/p2`: ```sh make run # its own server, local only: port 8798, data in ~/.pi/p2-mac/local (FAKE=1: faux provider) make run-radius # its own server connected to Radius: port 8799, data in ~/.pi/p2-mac/radius (signs in on first use) make attach # the p2 already running on 127.0.0.1:8788 with ~/.pi/p2 ``` A development build carries no p2. On first launch, choose a checkout in the menu bar item (**Choose p2 Checkout…**). It can be a pi-mono root or its `apps/p2`, with dependencies installed. The app then runs `node --import ./src/source.ts src/hosts/server.ts` there. If a p2 already answers on the port, the app adopts it instead: it opens that server and can restart it when it runs in dev mode. **Server** in the menu bar item lists the p2 servers running on this Mac, found by their loopback listeners, `/health` and the process's `P2_DATA_DIR` and working directory. Choosing one (your dev checkout's on 8788, say) makes the app use it: it never starts or stops that server, and it restarts it through `dev.sock` when it runs in dev mode. **Run Pi from …** switches back to the app's own server. ### Nightly builds `.github/workflows/p2-mac-nightly.yml` builds the app every night from the newest commit on main that passed CI and publishes it on pkg.earendil.com, the app `pi-mac`, channel `nightly`. Anyone can download it, no login needed: ``` https://pkg.earendil.com/-/apps/pi-mac/nightly/download ``` Open the DMG and drag Pi to Applications. The build needs nothing installed: - **p2 and Node inside:** `scripts/package-app.sh` puts the commit's p2 into `Contents/Resources/p2`. That copy holds the root tsconfig, the workspace packages p2 links (`src`, `package.json`, their own `node_modules`), `apps/p2` and production `node_modules`, with model data hydrated. The official Node build, checked against its pinned hash, goes into `Contents/Resources/node`. Unless a checkout is chosen, the app runs that copy with that Node (`ServerConfig.bundled`). **Use This App's Pi Instead of the Checkout** goes back to it. The server never writes into the bundle. - **Updates:** Sparkle (`Updater.swift`) checks the channel's appcast daily and installs a new build when Pi quits. **Check for Updates…** checks now. The DMG is signed with Sparkle's EdDSA key, whose public half is in Info.plist. The appcast comes from pkg.earendil.com (its `src/apps.ts`). - **Data backup:** before a new build first starts its p2 on a data directory, the app clones the directory to `<data>.backups/<previous build>`, since p2's data formats change without notice. It keeps three (`DataBackup.swift`). - **Signing:** with `SIGN_IDENTITY`, every Mach-O file in the bundle (Node, esbuild, Sparkle's helpers, the app) is signed with the hardened runtime. Node gets the JIT entitlements (`scripts/node.entitlements`), the app the microphone (`scripts/app.entitlements`). The DMG is notarized and stapled. Without `SIGN_IDENTITY`, the build is ad-hoc signed, without the hardened runtime. - **Versions:** `CFBundleVersion` is the commit's UTC minute (`202611180023`), which Sparkle orders by. The short version is its day (`2026.11.18`), and `PiCommit` holds the SHA. - **Smoke test:** `scripts/smoke-app.sh` starts the signed bundle's server in a bare environment and waits for `/health`. The workflow's signing credentials live in the GitHub environment `p2-mac-release`. `scripts/release-secrets.sh` sets them: it makes the Developer ID key and its CSR, turns Apple's certificate into a .p12, checks the App Store Connect API key with notarytool, and makes Sparkle's key in the login keychain (account `pi-mac`). Each one goes straight into `gh secret set`, never to the terminal. Run it yourself, not through an agent. To build one locally (ad-hoc signed, no updater): `./scripts/package-app.sh`, which writes `build/dist/Pi-<day>-<version>.dmg`. With `SPARKLE_PUBLIC_KEY`, the app gets the feed, and with `SPARKLE_KEY_FILE` the DMG gets signed for it. The script's header lists every variable. ### Settings Each setting is a user default (`defaults write com.earendil.pi <key> <value>`). An environment variable overrides most of them for test runs: | Default | Variable | Default value | |---|---|---| | `checkout` | `P2MAC_CHECKOUT` | the app's own copy of p2 (packaged builds), else none | | `node` | `P2MAC_NODE` | the app's own Node with its own p2, else `node` on the login shell's PATH | | `port` | `P2MAC_PORT` | 8788 | | `dataDir` | `P2MAC_DATA_DIR` | `~/.pi/p2` | | `mode` | `P2MAC_MODE` | `start` (`dev`: the dev supervisor) | | `fake` | `P2MAC_FAKE` | off (`1`: the faux provider) | | `source`, `attachedPort`, `attachedDataDir` | `P2MAC_ATTACH` (a port; `P2MAC_DATA_DIR` is then its data directory) | the app's own server | | `appearance` | none | `system` (`light`, `dark`) | `P2MAC_DIAG_DIR` makes every finished page load write `page-N.json` (what the page sees, its theme included) and snapshots of the page and the window (`page-N.png`, `window-N.png`) into that directory. With `P2MAC_DIAG_BRIDGE=<preference>` as well, the first page drives the host bridge once (`bridge-N.json`): it reads and sets the preference, checks the event, has an invalid value, a frame inside the page and a page on another origin refused, picks System in View > Appearance, clicks p2's theme button twice and then double-clicks it, changes the theme four more times (p2's button and the bridge, each way) while grabbing the windows with `screencapture` 0, 80, 160 and 300 ms after each change (`fade-N-*.png`, with a millisecond clock stamped into the page, and the page's view transition timing in `bridge-N.json`), and reloads over a stale stored theme, which the reload's record removes again. The grabs bring the window in front of other apps' windows and need the terminal's Screen Recording permission. The server's output goes to `~/Library/Logs/Pi/server-<port>.log`, which the menu bar item opens with **Show Server Log**. ### A trial run beside a live p2 This run leaves your real data and port alone: ```sh mkdir -m 700 /tmp/p2mac-data P2MAC_CHECKOUT=~/Development/pi-mono/apps/p2 P2MAC_PORT=8798 P2MAC_DATA_DIR=/tmp/p2mac-data P2MAC_FAKE=1 \ build/Pi.app/Contents/MacOS/Pi ``` Ctrl-C quits it like the Quit command, which also stops its server. A checkout used only for running p2 needs `npm install --ignore-scripts` and `npm run hydrate:model-data` at its root. ## How it works - **Windows and brain tabs** (`PiWindowController.swift`, `BrainTabBar.swift`, `BrainDirectory.swift`): - While the app is signed in to Radius, every window has a native tab bar in its title bar row. It shows the Radius shell's brain tabs: the same set, order, labels, organization prefixes, "Shared · …" and dimming of away brains, styled like Elanor's tabs. - The bar belongs to the window and never reloads; under it, each brain has its own page (`BrainPage.swift`), and a window keeps its recent ones alive. - A click switches in place, and so do ⌘1–9 (Window menu). Cmd-click, the middle button or dragging a tab out of the bar opens that brain in a new window. - This Mac's brain (matched by the server ID in its data directory's `radius.json`) loads directly and is marked with a laptop. Other brains load Radius's page for them. - Not signed in: no tab bar; the window shows this Mac's brain under the window buttons. - **Radius** (`RadiusAccount.swift`, `OAuthLoopback.swift`): Radius provides the list of Pis and the grants that open remote ones; the app loads no Radius pages. - **One credential, shared 1:1 with this Mac's Pi:** `<P2_DATA_DIR>/radius.json` on disk, no Keychain. The Pi connects to Radius with it; the app lists the tabs with it. - **Sign In to Radius…** in the menu bar item signs in as the Pi's own `radius:login` does (client `pi-gateway`, scope `gateway offline_access`), but through the system browser (PKCE S256, a loopback redirect on a free port; never in a webview). The credential goes into radius.json, keeping the Pi's server ID for the same gateway, and the Pi connects within 30 seconds. - **Refresh:** Radius does not rotate refresh tokens, so the app and the Pi refresh independently. The app reads the file before every call and writes a refreshed token back. - **The tabs:** the app polls `GET /v1/pi/servers` every 5 seconds (any gateway bearer; the token's organization only). `404 pi_shell_not_enabled` means no tab bar. Cloud Pis get no tab yet. - **Other Pis** load from their forwarded origin (`https://<label>.rdtun.com`) as the tab's page. A one-minute grant from `POST /v1/pi/servers/{org}/{server}/open?path=` (requested from Radius, not built there yet) turns into a view cookie, which lasts 12 hours. While the page is open, the app renews it every 6 hours (retrying after 5 minutes when that fails) with a new grant, exchanged without its path (the origin then answers 204 with the cookie instead of redirecting to a page), and puts the new cookie into the page's store (`ViewCookie.swift`). Without that, a page open longer kept its live connection but got the origin's 401 page for every image, file and artifact. A navigation after the cookie expired anyway goes to Radius's `/pi/forward`, and the app gets a fresh grant instead of loading that page. - **Sign Out of Radius** deletes radius.json, which also disconnects this Mac's Pi. - The gateway is the credential's (new sign-ins: `PI_RADIUS_GATEWAY`, default `https://radius.pi.dev`). `/v1/pi/servers` is new in Radius and not deployed yet. - **Development without Radius:** - `P2MAC_BRAINS=<file>` names a JSON file in the shape of `/v1/pi/servers` (`{ "servers": [...] }`) and counts as signed in. - `P2MAC_SIGN_IN=headless`, for a fake gateway, signs in at launch without opening the browser. - **Sign-in** (`LocalSignIn.swift`): the app reads `<data>/owner-key` and answers p2's local owner challenge (`/owner/local/challenge`, then `/owner/local/claim`), as the TUI does. It puts the `p2_owner_<port>` cookie into the brain's own `WKWebsiteDataStore(forIdentifier:)`, then loads `http://127.0.0.1:<port>`. The result is a full local owner, with nothing to paste. - **Server** (`ServerController.swift`): - It probes `/health` every 1.5 s. - It runs the server as a child process with the login shell's environment (`LoginEnvironment.swift`), so Homebrew and nvm tools are on PATH. - It stops the server with SIGTERM, and SIGKILL after 15 s. - It restarts its own server directly, and a server run elsewhere through `<data>/dev.sock` (`DevControl.swift`), when that server runs in dev mode. - Quitting the app stops its own server and leaves a server run elsewhere running. - **Chrome** (`WindowChrome.swift`): the webview fills the window under a transparent title bar, so p2's own bars are the title bar and its background runs to the top edge. - An empty unified toolbar gives the window the system's toolbar-window buttons, centred 26 points from the top: the middle of p2's 52-pixel bars. The build script records the real SDK in the binary (`vtool`). SwiftPM records the deployment target instead, and AppKit then draws the older, smaller window buttons. - The page gets `data-shell="macos"` and `--shell-inset-start` (the window buttons' width). The script marks the bars at the window's top-left corner `data-shell-corner`, found by geometry because the layout decides which bars are there: the sidebar header, the top bar, a full-window viewer, a views pane. p2's CSS starts a marked bar's content after the buttons (`apps/p2/src/web/style.css`, end; on main). In full screen nothing is marked. - The marks are set in an animation-frame callback, so a bar is indented before it is painted. WebKit pauses those callbacks in a hidden window, where a timer does the marking instead. Dragging covers p2's bar height (52 points), not the whole title bar, which is taller. - WebKit has no `app-region`, so a script in its own content world samples the title bar strip for interactive elements and reports their rectangles. A mouse-down anywhere else in the strip drags the window, and a double click does what the title bar's would. - Native macOS window tabs are off: brain tabs replace them. - **Pages** (`BrainPage.swift`): - **Links:** Cmd-click, middle-click and `target=_blank` on the brain's origin open a new window. Every other origin opens in the default browser. - **Native handling:** downloads go to ~/Downloads; `<input type=file>` gets a file panel; `alert`, `confirm` and `prompt` are sheets; the microphone is allowed for the brain origin only; the page reloads when the web process crashes. - **Placeholder:** while there is no page, a native view shows the server's state with Start Pi or Try Again. - **Appearance** (`Appearance.swift`, `HostBridge.swift`): the app owns the light/dark preference for every brain page, as the Radius shell does for the pages it frames. - The preference (System, Light, Dark) is the user default `appearance`. The app sets `NSApp.appearance` from it before the first window, so the tab bar, the placeholder and every page in "system" mode follow without a flash: WKWebView's `prefers-color-scheme` follows the app's appearance. - **View > Appearance** changes it, with a checkmark on the current one. - **Fade:** a change the app makes (the menu, the bridge) cross-fades in about 250 ms. Each document and the app fade their own pixels: the page runs a view transition when its scheme changes, and the app adds a Core Animation fade only to the views it draws itself (the tab bar, the placeholder), never to the content view or the webview's layer, which would stack a second fade over the page's. No fade at launch, with Reduce Motion, or for the system's own switch (macOS fades the screen). - **Page bridge:** every brain page, direct and remote, has `window.webkit.messageHandlers.piHost` in its own world, so p2 (`apps/p2/src/web/theme.ts`) knows at startup that the app owns the preference and ignores the one stored in its origin. `postMessage({ type: "theme" })` resolves to `{ preference }`; `postMessage({ type: "set-theme", preference })` stores and applies it and resolves to `{ preference }`. After every change, from either side, all pages of all windows get the event `pi-host-theme` with `detail: { preference }`. The resolved scheme is never sent: the page reads it from `matchMedia`. - We - apps/p2-mac/scripts/build-app.sh (1570 bytes)
#!/bin/sh # Builds build/Pi.app from the Swift package: the binary, Sparkle.framework, Info.plist, the icon, an ad-hoc # signature. CONFIG=release for an optimized build. scripts/package-app.sh turns it into a release. # Without p2 in its resources and without SUFeedURL, the app runs a checkout and never updates itself. set -eu cd "$(dirname "$0")/.." config="${CONFIG:-debug}" swift build -c "$config" bin="$(swift build -c "$config" --show-bin-path)/P2Mac" if [ ! -f build/AppIcon.icns ] || [ scripts/make-icon.swift -nt build/AppIcon.icns ]; then rm -rf build/AppIcon.iconset mkdir -p build swift scripts/make-icon.swift build/AppIcon.iconset iconutil -c icns build/AppIcon.iconset -o build/AppIcon.icns fi app=build/Pi.app rm -rf "$app" build/P2.app mkdir -p "$app/Contents/MacOS" "$app/Contents/Resources" "$app/Contents/Frameworks" # ditto keeps the framework's symlinks (Versions/Current), which codesign requires. ditto "$(dirname "$bin")/Sparkle.framework" "$app/Contents/Frameworks/Sparkle.framework" # SwiftPM records the deployment target (14.0) as the SDK the binary was built with, and AppKit then draws the # window chrome of that release: small, closely spaced window buttons and a 52-point toolbar instead of the current # ones. Record the real SDK. vtool -set-build-version macos 14.0 "$(xcrun --sdk macosx --show-sdk-version)" -replace -output "$app/Contents/MacOS/Pi" "$bin" 2>/dev/null cp Resources/Info.plist "$app/Contents/Info.plist" cp build/AppIcon.icns "$app/Contents/Resources/AppIcon.icns" codesign --force --sign - "$app" echo "$app" - apps/p2-mac/scripts/package-app.sh (7070 bytes)
#!/usr/bin/env bash # Packages Pi.app as a release that needs nothing installed: this commit's p2 with its production dependencies and # the Node it runs on go into Contents/Resources (ServerConfig.bundled finds them there). Then it signs the app, # checks that the signed server starts, builds a DMG, notarizes it and signs it for Sparkle. Output in build/dist: # the DMG and build.json, the fields pkg.earendil.com's app upload takes. # # ./scripts/package-app.sh ad-hoc signed, no notarization, no updater: a local test of the bundle # # Environment (all optional; CI sets them, .github/workflows/p2-mac-nightly.yml): # SIGN_IDENTITY codesign identity, e.g. "Developer ID Application: … (TEAMID)" # NOTARY_KEY, NOTARY_KEY_ID, NOTARY_ISSUER App Store Connect API key (.p8 file) for notarytool # SPARKLE_PUBLIC_KEY Sparkle's EdDSA public key: the app gets the feed and updates itself # SPARKLE_KEY_FILE its private key file, to sign the DMG for Sparkle # FEED_URL the appcast; default: pkg.earendil.com's nightly channel set -euo pipefail cd "$(dirname "$0")/.." root="$(cd ../.. && pwd)" NODE_VERSION=v26.13.1 NODE_SHA256=af6202f5370a3a5209e85dc543758dfd9f44c8726626a236730cd603a67a8559 MINIMUM_SYSTEM_VERSION=14.0 FEED_URL="${FEED_URL:-https://pkg.earendil.com/-/apps/pi-mac/nightly/appcast.xml}" SIGN_IDENTITY="${SIGN_IDENTITY:--}" work=build/package app=build/Pi.app dist=build/dist rm -rf "$work" "$dist" mkdir -p "$work" "$dist" build/cache # Versions come from the commit: Sparkle orders builds by CFBundleVersion (the commit's UTC minute), people read the # short version (its day). sha="$(git -C "$root" rev-parse HEAD)" version="$(TZ=UTC git -C "$root" log -1 --format=%cd --date=format-local:%Y%m%d%H%M)" short_version="$(TZ=UTC git -C "$root" log -1 --format=%cd --date=format-local:%Y.%m.%d)" echo "packaging $sha as $short_version ($version)" # Node: the official build, checked against its pinned hash. node_archive="build/cache/node-$NODE_VERSION-darwin-arm64.tar.xz" if [ ! -f "$node_archive" ] || ! echo "$NODE_SHA256 $node_archive" | shasum -a 256 -c - >/dev/null 2>&1; then curl -fsSL -o "$node_archive" "https://nodejs.org/dist/$NODE_VERSION/node-$NODE_VERSION-darwin-arm64.tar.xz" echo "$NODE_SHA256 $node_archive" | shasum -a 256 -c - fi tar -xf "$node_archive" -C "$work" node_dir="$PWD/$work/node-$NODE_VERSION-darwin-arm64" # p2: a clean copy of the commit (uncommitted changes stay out), with p2's production dependencies, installed and # hydrated by the Node it will run on. src="$work/src" mkdir -p "$src" git -C "$root" archive HEAD | tar -x -C "$src" ( cd "$src" export PATH="$node_dir/bin:$PATH" npm ci --omit=dev --ignore-scripts --no-audit --no-fund -w apps/p2 npm run hydrate:model-data >/dev/null ) CONFIG=release ./scripts/build-app.sh resources="$app/Contents/Resources" mkdir -p "$resources/node/bin" "$resources/p2/apps" cp "$node_dir/bin/node" "$resources/node/bin/node" cp "$node_dir/LICENSE" "$resources/node/LICENSE" # The source resolver (apps/p2/src/source-resolver.ts) reads the root tsconfig's paths; packages resolve through it. cp "$src/package.json" "$src/tsconfig.json" "$src/tsconfig.base.json" "$resources/p2/" cp -R "$src/node_modules" "$resources/p2/node_modules" # The workspace packages p2 uses are the ones npm linked: node_modules/@earendil-works/x -> ../../packages/x. for link in "$src"/node_modules/@earendil-works/*; do [ -L "$link" ] || continue package="$(readlink "$link")" package="${package#../../}" mkdir -p "$resources/p2/$package" for part in package.json src node_modules; do if [ -e "$src/$package/$part" ]; then cp -R "$src/$package/$part" "$resources/p2/$package/"; fi done done rsync -a --exclude /p2/test --exclude /p2/docs "$src/apps/p2" "$resources/p2/apps/" # Links to workspaces left out (none today) would break the signature. find "$resources/p2" -type l ! -exec test -e {} \; -print -delete plist="$app/Contents/Info.plist" set_plist() { /usr/libexec/PlistBuddy -c "Delete :$1" "$plist" >/dev/null 2>&1 || true; /usr/libexec/PlistBuddy -c "Add :$1 $2 $3" "$plist"; } set_plist CFBundleVersion string "$version" set_plist CFBundleShortVersionString string "$short_version" set_plist LSMinimumSystemVersion string "$MINIMUM_SYSTEM_VERSION" set_plist PiCommit string "$sha" if [ -n "${SPARKLE_PUBLIC_KEY:-}" ]; then set_plist SUFeedURL string "$FEED_URL" set_plist SUPublicEDKey string "$SPARKLE_PUBLIC_KEY" set_plist SUEnableAutomaticChecks bool true # Downloads in the background and installs when Pi quits. set_plist SUAutomaticallyUpdate bool true fi # Signing, inside out: every Mach-O file in the resources (Node, esbuild, native addons), Sparkle's helpers, the app. # Notarization needs the hardened runtime and a secure timestamp on all of them. An ad-hoc build goes without the # hardened runtime: its library validation needs a Team ID, which ad-hoc signatures lack, so Sparkle would not load. sign=(codesign --force --sign "$SIGN_IDENTITY") if [ "$SIGN_IDENTITY" != "-" ]; then sign+=(--options runtime --timestamp); fi find "$resources" -type f \( -perm -u+x -o -name '*.node' -o -name '*.dylib' \) -print0 | while IFS= read -r -d '' file; do if file -b "$file" | grep -q '^Mach-O'; then if [ "$file" = "$resources/node/bin/node" ]; then "${sign[@]}" --entitlements scripts/node.entitlements "$file" else "${sign[@]}" "$file" fi fi done sparkle="$app/Contents/Frameworks/Sparkle.framework/Versions/B" "${sign[@]}" "$sparkle/XPCServices/Installer.xpc" "${sign[@]}" --preserve-metadata=entitlements "$sparkle/XPCServices/Downloader.xpc" "${sign[@]}" "$sparkle/Autoupdate" "${sign[@]}" "$sparkle/Updater.app" "${sign[@]}" "$app/Contents/Frameworks/Sparkle.framework" "${sign[@]}" --entitlements scripts/app.entitlements "$app" codesign --verify --deep --strict "$app" ./scripts/smoke-app.sh "$app" file="Pi-$short_version-$version.dmg" dmg="$dist/$file" staging="$work/dmg" mkdir -p "$staging" ditto "$app" "$staging/Pi.app" ln -s /Applications "$staging/Applications" # HFS+: an APFS image of the same app compresses to more than twice the size. hdiutil create -quiet -volname Pi -srcfolder "$staging" -fs HFS+ -format ULMO -ov "$dmg" if [ "$SIGN_IDENTITY" != "-" ]; then codesign --force --sign "$SIGN_IDENTITY" --timestamp "$dmg"; fi if [ -n "${NOTARY_KEY:-}" ]; then xcrun notarytool submit "$dmg" --key "$NOTARY_KEY" --key-id "$NOTARY_KEY_ID" --issuer "$NOTARY_ISSUER" --wait xcrun stapler staple "$dmg" fi # Sparkle signs the final bytes, so after stapling. ed_signature="" if [ -n "${SPARKLE_KEY_FILE:-}" ]; then ed_signature="$(.build/artifacts/sparkle/Sparkle/bin/sign_update -p --ed-key-file "$SPARKLE_KEY_FILE" "$dmg")" fi cat >"$dist/build.json" <<EOF { "sha": "$sha", "file": "$file", "version": "$version", "shortVersion": "$short_version", "minimumSystemVersion": "$MINIMUM_SYSTEM_VERSION", "edSignature": "$ed_signature" } EOF du -h "$dmg" cat "$dist/build.json" - apps/p2-mac/scripts/release-secrets.sh (7618 bytes)
#!/usr/bin/env bash # Sets up the signing credentials of .github/workflows/p2-mac-nightly.yml in the GitHub environment `p2-mac-release`. # Run it yourself in a terminal, not through an agent: keys never reach stdout. They go from files and keychain items # straight into `gh secret set`, which encrypts them on this machine before upload. # # ./scripts/release-secrets.sh environment create the environment, limited to main # ./scripts/release-secrets.sh certificate-request make the signing key and a CSR for Apple # ./scripts/release-secrets.sh certificate <file.cer> turn Apple's certificate into the signing secrets # ./scripts/release-secrets.sh notary <AuthKey_ID.p8> <issuer ID> # check an App Store Connect API key, store it for notarytool # ./scripts/release-secrets.sh sparkle make (or reuse) Sparkle's update key, store it # ./scripts/release-secrets.sh status list what the environment holds (names only) # # REPO overrides the repository (default earendil-works/pi). set -euo pipefail cd "$(dirname "$0")/.." REPO="${REPO:-earendil-works/pi}" ENVIRONMENT=p2-mac-release # Between certificate-request and certificate: the private key waits here, readable by you only. PENDING="$HOME/.p2-mac-signing" SPARKLE_ACCOUNT=pi-mac OPENSSL=/usr/bin/openssl secret() { gh secret set "$1" --env "$ENVIRONMENT" --repo "$REPO" >/dev/null; echo " secret $1 set"; } variable() { gh variable set "$1" --env "$ENVIRONMENT" --repo "$REPO" --body "$2" >/dev/null; echo " variable $1 = $2"; } # A directory for this run's intermediate files, removed on exit. scratch() { work="$(mktemp -d)" chmod 700 "$work" trap 'rm -rf "$work"' EXIT } environment() { echo "Creating $ENVIRONMENT in $REPO (deployments from main only)" gh api -X PUT "repos/$REPO/environments/$ENVIRONMENT" --input - >/dev/null <<-EOF { "deployment_branch_policy": { "protected_branches": false, "custom_branch_policies": true } } EOF if ! gh api "repos/$REPO/environments/$ENVIRONMENT/deployment-branch-policies" --jq '.branch_policies[].name' | grep -qx main; then gh api -X POST "repos/$REPO/environments/$ENVIRONMENT/deployment-branch-policies" -f name=main -f type=branch >/dev/null fi echo " done" } certificate_request() { if [ -e "$PENDING/key.pem" ]; then echo "$PENDING/key.pem exists: a request is pending. Finish it with 'certificate <file.cer>', or delete $PENDING." exit 1 fi read -r -p "Your name (as on the Apple developer account): " name read -r -p "Your Apple ID email: " email mkdir -p "$PENDING" chmod 700 "$PENDING" (umask 077 && "$OPENSSL" genrsa -out "$PENDING/key.pem" 2048 2>/dev/null) "$OPENSSL" req -new -key "$PENDING/key.pem" -out "$PENDING/request.csr" -subj "/emailAddress=$email/CN=$name/C=AT" cat <<-EOF Wrote $PENDING/request.csr (the private key stays next to it). 1. Open https://developer.apple.com/account/resources/certificates/add (as the team's Account Holder). 2. Choose "Developer ID Application", profile type "G2 Sub-CA", and upload request.csr. 3. Download the certificate, then run: ./scripts/release-secrets.sh certificate ~/Downloads/developerID_application.cer EOF } certificate() { local cer="${1:?usage: certificate <file.cer>}" [ -e "$PENDING/key.pem" ] || { echo "No pending key in $PENDING; run certificate-request first."; exit 1; } scratch "$OPENSSL" x509 -inform DER -in "$cer" -out "$work/cert.pem" # The certificate must be for the pending key, and a Developer ID Application certificate. local key_hash cert_hash identity issuer key_hash="$("$OPENSSL" rsa -in "$PENDING/key.pem" -noout -modulus 2>/dev/null | shasum -a 256)" cert_hash="$("$OPENSSL" x509 -in "$work/cert.pem" -noout -modulus | shasum -a 256)" [ "$key_hash" = "$cert_hash" ] || { echo "$cer is not for the key in $PENDING."; exit 1; } identity="$("$OPENSSL" x509 -in "$work/cert.pem" -noout -subject -nameopt multiline | sed -n 's/^ *commonName *= //p')" issuer="$("$OPENSSL" x509 -in "$work/cert.pem" -noout -issuer -nameopt multiline | sed -n 's/^ *commonName *= //p')" case "$identity" in "Developer ID Application: "*) ;; *) echo "Not a Developer ID Application certificate: $identity"; exit 1 ;; esac [ "$issuer" = "Developer ID Certification Authority" ] || { echo "Unexpected issuer: $issuer"; exit 1; } echo "Certificate: $identity" # The intermediate goes into the .p12, so the runner's keychain has the whole chain. curl -fsSL https://www.apple.com/certificateauthority/DeveloperIDG2CA.cer | "$OPENSSL" x509 -inform DER -out "$work/ca.pem" local password password="$("$OPENSSL" rand -hex 24)" # SHA1/3DES: the formats `security import` reads; it rejects newer OpenSSL defaults. printf '%s' "$password" | "$OPENSSL" pkcs12 -export -inkey "$PENDING/key.pem" -in "$work/cert.pem" \ -certfile "$work/ca.pem" -name "$identity" -keypbe PBE-SHA1-3DES -certpbe PBE-SHA1-3DES -macalg sha1 \ -passout stdin -out "$work/identity.p12" echo "Uploading to $ENVIRONMENT" base64 -i "$work/identity.p12" | secret MACOS_CERTIFICATE_P12 printf '%s' "$password" | secret MACOS_CERTIFICATE_PASSWORD variable MACOS_SIGN_IDENTITY "$identity" # The login keychain keeps the key: your backup, and local signing with the same identity. security import "$work/identity.p12" -k "$HOME/Library/Keychains/login.keychain-db" -P "$password" -T /usr/bin/codesign >/dev/null echo " imported into your login keychain" rm -rf "$PENDING" echo " removed $PENDING" } notary() { local key="${1:?usage: notary <AuthKey_ID.p8> <issuer ID>}" local issuer="${2:?usage: notary <AuthKey_ID.p8> <issuer ID>}" local key_id key_id="$(basename "$key" .p8)" key_id="${key_id#AuthKey_}" echo "Checking key $key_id with notarytool" if ! xcrun notarytool history --key "$key" --key-id "$key_id" --issuer "$issuer" >/dev/null 2>&1; then echo "notarytool rejected the key; check the key file, its ID ($key_id) and the issuer ID." exit 1 fi echo "Uploading to $ENVIRONMENT" base64 -i "$key" | secret APPLE_NOTARY_KEY printf '%s' "$key_id" | secret APPLE_NOTARY_KEY_ID printf '%s' "$issuer" | secret APPLE_NOTARY_ISSUER echo "Apple lets you download $key once: keep it in a password manager, then delete it here." } sparkle() { local tool=.build/artifacts/sparkle/Sparkle/bin/generate_keys [ -x "$tool" ] || swift package resolve scratch # Creates the key in your login keychain under its own account, or reuses the one there. It prints the public key. "$tool" --account "$SPARKLE_ACCOUNT" >/dev/null local public public="$("$tool" --account "$SPARKLE_ACCOUNT" -p)" "$tool" --account "$SPARKLE_ACCOUNT" -x "$work/sparkle.key" >/dev/null echo "Uploading to $ENVIRONMENT" secret SPARKLE_PRIVATE_KEY <"$work/sparkle.key" variable SPARKLE_PUBLIC_KEY "$public" cat <<-EOF The private key stays in your login keychain (account $SPARKLE_ACCOUNT): every installed app trusts only it, so back it up (generate_keys --account $SPARKLE_ACCOUNT -x <file> into a password manager). Never replace it while apps are installed: they would stop updating. EOF } status() { echo "$REPO, environment $ENVIRONMENT" echo "secrets:" gh secret list --env "$ENVIRONMENT" --repo "$REPO" | sed 's/^/ /' echo "variables:" gh variable list --env "$ENVIRONMENT" --repo "$REPO" | sed 's/^/ /' } case "${1:-}" in environment) environment ;; certificate-request) certificate_request ;; certificate) certificate "${2:-}" ;; notary) notary "${2:-}" "${3:-}" ;; sparkle) sparkle ;; status) status ;; *) sed -n '2,14p' "$0" | sed 's/^# \{0,1\}//'; exit 1 ;; esac - apps/p2-mac/scripts/smoke-app.sh (1140 bytes)
#!/usr/bin/env bash # Checks that a packaged Pi.app's own p2 starts: its signed Node runs the server from the app's resources, with a # throwaway data directory and the faux provider, and the server answers /health. # # ./scripts/smoke-app.sh build/Pi.app set -euo pipefail app="${1:?usage: smoke-app.sh <Pi.app>}" resources="$(cd "$app/Contents/Resources" && pwd)" data="$(mktemp -d)" port=18797 log="$data.log" trap 'kill "$pid" 2>/dev/null || true; wait "$pid" 2>/dev/null || true; rm -rf "$data" "$log"' EXIT # A bare environment: the server must not depend on a Node or tools on the builder's PATH. (cd "$resources/p2/apps/p2" && exec env -i HOME="$HOME" PATH=/usr/bin:/bin:/usr/sbin:/sbin PORT=$port \ P2_DATA_DIR="$data" P2_FAKE=1 "$resources/node/bin/node" --import ./src/source.ts src/hosts/server.ts) >"$log" 2>&1 & pid=$! for _ in $(seq 1 120); do if curl -fsS "http://127.0.0.1:$port/health" 2>/dev/null | grep -q '"name":"p2"'; then echo "smoke test: p2 answers on port $port" exit 0 fi if ! kill -0 "$pid" 2>/dev/null; then break; fi sleep 1 done echo "smoke test failed; server output:" >&2 cat "$log" >&2 exit 1 - apps/p2/biome.json (540 bytes)
{ "$schema": "https://biomejs.dev/schemas/2.3.5/schema.json", "formatter": { "enabled": true, "indentStyle": "tab", "indentWidth": 3, "lineWidth": 120 }, "linter": { "enabled": true, "domains": { "react": "recommended" }, "rules": { "recommended": true, "style": { "noNonNullAssertion": "off" } } }, "files": { "includes": [ "src/**", "plugins/**", "!plugins/terminals/vendor", "test/**", "!test/fixtures/v1-data/data", "scripts/**", "*.json", "!package-lock.json" ] } } - apps/p2/DEV.md (4076 bytes)
# p2 development See [p2 notes](docs/p2-notes.md) for architecture, reloads and testing; [README](README.md#tui-isolation) defines the TUI threat model. Production/data directories must not be used for tests. Commit nothing without approval. ## Server and TUI The server supports Node 22.19+. The launcher requires Node >=26.5.1; slayer has Node 26.11.1 at `~/.local/bin/node26`. This wrapper supplies the libraries required by that binary. From `apps/p2`: ```sh ~/.local/bin/node26 --import ./src/source.ts src/hosts/tui.ts ~/.local/bin/node26 --import ./src/source.ts src/hosts/tui.ts https://your-domain npm run check ~/.local/bin/node26 --import ./src/source.ts --test --test-concurrency=1 test/p2/tui*.test.ts ``` To use `npm run tui`/`npm test` with this Node, put its binary directory first in PATH and supply its library path: ```sh PATH="$HOME/node-v26.11.1-linux-x64/bin:$PATH" LD_LIBRARY_PATH="$HOME/node26-lib" npm run tui PATH="$HOME/node-v26.11.1-linux-x64/bin:$PATH" LD_LIBRARY_PATH="$HOME/node26-lib" npm test ``` The server builds one ESM bundle through esbuild and serves it only to web-admitted owners/share guests. Facets are never imported from the visitor's checkout. Successful server plugin reloads rebuild/publish the bundle; the launcher swaps to a fresh worker only after readiness and terminates its predecessor. TUI-only source/import changes count as dirty plugins. TUI imports gate facet inclusion without expanding server/worker selection. Keybindings, theme and display settings are guest-local: the launcher reads pi's `~/.pi/agent` files (`PI_CODING_AGENT_DIR`) and `~/.pi/p2-tui/prefs.json`, validates them and posts data to the worker, watching for changes; `<P2_DATA_DIR>/tui.json` (`TuiPreferences`) holds only server defaults below them. Launcher `config.json` holds only per-origin cookies. ## Security boundary Do not move authentication, sockets, raw terminal management or machine access into downloaded code. The launcher recognizes a small message protocol; unknown capabilities are denied. Only opaque chord frames use its fixed-origin socket. Download manifests/paths, byte bounds and version hashes are validated; redirects are refused. The trusted Worker bootstrap must drop `fs.read`, `fs.write`, `worker`, `net`, `child`, `addon`, `wasi` and `inspector` before ANY downloaded module evaluation. Node's permission-mode workers have implicit module-read grants until explicitly dropped. Bundle import is from an in-memory data URL. Empty env, captured/discarded stdout/stderr, V8 limits and cheap process/V8/worker-surface taming are required. Never expose parentPort or the Worker handle. Disable the entire fs/fs-promises function facade too: inherited numeric descriptors can bypass path-based permission checks, including raw fd reads/writes. BroadcastChannel and environment-data APIs must not communicate with other host workers. These restrictions precede the bundle import and are tested through built-in aliases. All downloaded output is untrusted: it must pass the launcher's bounded, complete-sequence terminal allowlist. Never forward raw worker output, logs, exceptions, OSC/clipboard/title/hyperlinks or image protocols. Local file, clipboard, browser, editor and job-control capabilities exist only as launcher actions (`src/hosts/tui-actions.ts`): each needs an explicit, launcher-owned user grant (the visitor's own key, or a confirmation in the launcher-drawn consent screen of the exact data) with validated data, never a server-provided path, a credentialed URL or a simulated prompt. Adding one follows p2-tui-port.md sections 3.1-3.5. Denial of service is owner-accepted. The watchdog samples RSS/ArrayBuffers and terminates the worker over its caps, but sampling/V8 limits cannot enforce a hard native-memory quota; memory exhaustion can affect the host. There is no OS sandbox. Runtime vulnerabilities remain out of scope. Keep Node patched and rerun the escape tests after runtime changes. Tests use fake providers, disposable directories and port 0; no production or paid provider calls.
- apps/p2/docs/agent-plugins.md (6512 bytes)
# Agent-written plugins The agent writes real p2 plugins (multiplayer or persistent views, server objects, tools) and applies them itself. Html artifacts (docs/artifacts.md) stay for single-player pages. ## Problem An html artifact is one page per viewer. A shared counter, a game with rules or a board several people edit needs state on the server, rules applied there and events between viewers: a plugin. Before this, only a p2 checkout's built-in plugins could be changed, only by owner-started turns, and only in dev mode. ## How it works 1. **The user plugin root**, `<data>/plugins/` (`userPluginsDirectory`, `src/runtime/host-dirs.ts`), is a second Chord plugins root after `apps/p2/plugins/` (`createP2Plugins`, `src/hosts/plugins.ts`). It is loaded and watched like the built-in root on the server, every worker, the web and the TUI bundles. A built-in wins a name clash (`listRoots`: the first root wins; discovery reports the clash). The server creates the directory at start (mode 0700). User plugins import p2 only through `@p2/app/v1` and its per-kind modules; the bundles resolve `react` and the other host packages from the app's and the repository's `node_modules` (`common.nodePaths`, `src/hosts/bundles.ts`). `pluginFile(path)` from `@earendil-works/chord/plugins` resolves against the plugin's own directory in whichever root it lives (the facet's `plugin.dir`). 2. **Writing** uses the ordinary coding tools, no plugin file tools: the agent works in the server's environment. A session working on this machine writes there directly; any other session spawns a subagent with `env: null` and `cwd: <data>/plugins` (location `local:<data>/plugins`), whose tools then act on the server's files. 3. **Applying** is `reload_plugins` (plugins/dev-tools): the worker calls the server object `PluginReload` (worker principals only), which calls `ServerParts.plugins.reload(requested?)` (`applyPlugins` in `src/hosts/server.ts`): - `requested`, else every dirty plugin plus every plugin of the user root (an unchanged one keeps its digest, so its hosts keep it as it is: an apply never depends on the watcher having seen the last write; a directory not discovered yet is waited for once); - one `rebuild`: Chord's build (web facets included), then the TUI bundle. Any failure stops before the rollout, so the old code stays; the rollout ends with the web surface's step, which pages apply in place; - per plugin: `applied`, `removed`, `failed` (the build's esbuild errors as `<kind>: file:line:column: message`, or a host's load error), `refused` (unknown, or a discovery problem: an invalid `package.json`, a built-in's name), plus the server's and workers' errors logged since the apply began (`processLogs`); - in dev mode the dev status is notified like the Reload button. 4. **Showing**: `view_open` (code mode) opens any kind a web facet registered in `ViewKinds` (`{ <kind>: { Pane } }`, docs/views.md), with `item` naming what it shows. A plugin keeping several items per session also contributes an artifact kind (`ArtifactKinds`), so the artifacts side panel lists them and opens them as views. 5. **Guidance** is a skill, `plugins/agent-plugins/skills/writing-p2-plugins/SKILL.md`, with a complete, tested example next to it (`example/counter/`: a shared counter with presence, a wave event, a pane and a code-mode tool). Plugins' skills come from the worker registry `PluginSkills` (`src/runtime/plugin-skills.ts`, exported by `@p2/app/v1/worker`), keyed by plugin, `{ dir }` (a facet passes `pluginFile("skills")`). The prompt plugin lists them in `<available_skills>` after the environment's and the project's skills (the first skill of a name wins), counted as global skills (a locked session gets them only with its global instructions), and **only for conversations working on the server's machine**: the listing points at the plugin's files, never copied to another environment. Its local watches rewatch when the registry's directories change. A conversation on another environment gets instead the agent-plugins plugin's `p2-plugins` prompt section: multiplayer or persistent views are plugins; start a subagent at `local:<data>/plugins`, whose skills include the guide. 6. **Plugin tools** are code-mode only by default (`callers: ["tools"]`), grouped by the code-mode namespace their extension declares (`namespace: { name, description, instructions? }`, Durable's `ToolNamespace`): the prompt lists the namespace and its description from the extensions present, so it stays stable while tools come and go, and `describeNamespace(name)` gives the instructions and tools. The skill and the example say so; leaving `callers` out offers a tool in the model's main list. ## Decisions - **No per-turn authority.** `origin` is gone from p2 (tool requirements, `originOf`, `answerOrigin`, the `origin` fields of agent-written entries and of `Conversations`, `PeerConversation`, `ConversationBroker` and `AgentLocations`, each bumped a version). Whoever may send input in a session may have its agent use every tool it has. What a session's agents reach is decided by its tool selection, its location (a sandbox) and its lock. `reload_plugins` has no requirements, like bash: subagents can be given it. - **A failed build applies nothing**, bundles included. - **`Views` evolves by versions** (`evolve` with `accepts`) instead of changing in place: version 2 added plugin kinds, version 3 `item` and dropped view state and the library. - **Kind names are not checked** against what pages register: the pane shows "Unknown view kind". ## Tests `test/p2/agent-plugins.test.ts`: a subagent at `local:<root>` copies the example with `bash` and applies it with `reload_plugins` (server object, event, presence, the code-mode tool and its namespace in the prompt, the view kind, the web script in the pages' combined file); an edit reloads in place; build errors with file and line, refusals (a built-in's name, an unknown plugin); `pluginFile` in the user root and its skills listed after the project's; a subagent given other tools cannot reload; a guest's turn uses `reload_plugins` and the view tools; in Chromium, an unknown kind's pane becomes the plugin's after the reload. `envs.test.ts`: a remote conversation lists no plugin skill and gets the hint. `dev-tools.test.ts`: the tool's results. `tool-access.test.ts`: `scope` and `locked` without origin. - apps/p2/docs/artifacts.md (7373 bytes)
# Artifacts Artifacts are pages and documents the agent writes for the person: an html page (a chart, a tool, a game, an editor such as Max's glyph editor) or a Markdown document. They belong to the session, keep every version, show in the transcript where the agent shows them, open as views beside the chat (docs/views.md), and are listed in the artifacts side panel. Plugin: `plugins/artifacts`. Decision: "Artifacts and views" in `apps/p3/docs/decisions.md`. ## What it does - **The agent** works with them through code-mode tools in the namespace `artifact` (`tool.ts`, extension `p2-artifacts`): `artifact_list`, `artifact_read` (a version, default the newest), `artifact_write` (a new artifact, its ID made from the title unless given, or the next version of `id`), `artifact_edit` (exact replacements, each `old` once), `artifact_show` (in the transcript, right after the call), `artifact_open` (a view of its kind), `artifact_delete`, and for html pages `artifact_html_errors`, `artifact_html_logs`, `artifact_html_eval` (code run in a page open on some device), `artifact_html_state` (the whole state as an object) and `artifact_html_set_state`. Each returns a structured value. The prompt lists the namespace's description; `describeNamespace("artifact")` gives the instructions and the page API. - **New page errors ride along**: errors the open pages report and no message carried yet go with the person's next message as an `<artifact-errors>` block (the views plugin's `MessageContexts`), once. - **In the transcript** (`activity.tsx`): `artifact_show`, called from a script, places the artifact's pane in place after the script (`hasResult` "inline"; nested code-mode calls place results as top-level calls do, `src/web/activity/model.ts`). It works as beside the chat: people who may send change it (an html page saves its state) and select in it (the selection goes with their next message through `Views.selectInline`); viewers read it. The pane mounts only near the screen, with hysteresis so a card at the edge never flips: it mounts within one viewport of the transcript's scroller and unmounts only past two (two `IntersectionObserver`s rooted at the scroller). The card keeps the height its pane last had (`inline-size.ts`, remembered per artifact for the page's host) while the pane is away and while a remounted pane settles: an html frame starts at that height, and the reserve holds until the pane reports its real one (the page's size message, a document drawn) or 1.5 s pass, so scrolling back never shrinks the card under the reader. A card shows the artifact's newest version; a deleted artifact says so. - **Beside the chat**: view kinds `html` and `md` (`panes.tsx`), with `item` the artifact's ID. - **The side panel** (`web.tsx`): the top bar's button counts every artifact kind's items (`ArtifactKinds`, this plugin's html and md and any plugin's), the side panel lists them, newest first. A row opens the item as a view of its kind; for a reader who may not change the session's views (a read-only guest) it shows the item's pane in the side panel on their device only, not editable (`local-item.ts`). People who may send can remove items (`useRemove`). The command menu's "Open artifact…" and its root search do the same (`commands.tsx`). - **TUI** (`tui.ts`): `/artifacts` lists them; Enter shows one (Markdown rendered, a page as its source) over the chat or in the side panel, following new versions; Ctrl+B opens a page in the browser. ## html pages `/artifact-frame?session&id&version` (`server.ts`) serves an html version into its frame: the viewer is authorized (`canView`, no cross-site requests), the page gets the CSP sandbox (`frame.ts`: an opaque origin, scripts and https CDN resources, `eval` for the agent's code, no frames, forms or app identity; `frame-ancestors` names the Radius shell when relayed). Two scripts go in front of the page: the diagnostics (errors, CSP violations, its content height for inline cards) and the bridge (`bridge.ts`), which defines `window.p2view`: - `onState(fn)`: calls `fn(state, from)` now (`from` is `init`) and at each change made elsewhere (`update`: the agent or another device). - `state`, `readOnly` (people who may not send: their changes are not saved). - `setState(next | fn)`: the page calls it on each edit; the pane saves it after 300 ms of rest. - `select(summary, data)` and `clearSelection()`: the view's selection (beside the transcript only). - `onDescribeRegion(fn)`: says what lies under a drawing. Without it, the bridge collects the labels and texts of the elements there. The bridge also reports the page's text selection (for comments), applies highlights, posts console lines, and runs the agent's code (an async function body; its JSON result or error goes back). The page's posts are untrusted. The pane (`HtmlPane`) accepts them only from its own frame, saves only that artifact's state, and checks the state's size (256 KiB of JSON). Each page holds `ArtifactPage {session, id}` (variants viewer, editor): its summary, its state (`value`), the agent's requests (event `request`, answered with `answer`; the first open page gets them; 30 s timeout), and `report` for its errors and logs (kept in memory per artifact, the newest 50 errors and 500 lines, `pages.ts`). Results that come from pages are flagged to the agent as the page's, not instructions. ## Who owns what | Fact | Owner | Where | | --- | --- | --- | | A session's artifacts (kind, title, newest version, times) | the server | document `p2.artifacts`, keyed by the session's host address (`keyed: Session`) | | Each version's content (at most 2 MiB) | the server | table `artifact_versions` with a `host(Session)` column, written once | | An html artifact's state | the server | document `p2.artifact-states`, keyed by the session | | What open pages reported | the server, in memory | `PageRegistry` (`pages.ts`) | All writes go through one local object, `ArtifactStoreHost` (`server.ts`, `store.ts`); `Artifacts {session}` (variants reader, editor, worker) and `ArtifactPage` derive from its documents. Being keyed by the session's address, the data goes with the session's data export and import and is removed by its delete (`exportHostData`, `deleteHostData`); a clone or fork copies it (`SessionCopied`). An archived session's pages still serve (no worker needed). There is no migration: artifacts written before 2026-11-12 stay in the session's files (space `artifacts`), unused. ## Tests - `test/p2/artifacts.test.ts`: the tools; the frame route; a page's state whole for page and agent; code run in an open page; errors with the next message, once; a read-only guest reads; export and delete carry the data. - `test/p2/ui-inline-artifacts.test.ts`: Chromium. Markdown and html in place, fitted and capped at 70vh, for a guest too; a page scrolled out and back in keeps its height and the reader's place; an inline counter page changed and selected by the owner (saved, carried) and read-only for a guest; a deleted artifact. - `test/p2/ui-views.test.ts`: the glyph-editor-style page's state and selection round trip. - `test/p2/ui-viewers-search.test.ts`, `ui-command-menu-sharing.test.ts`: the side panel, the commands, a reader opening one in the side panel. - `test/p2/fork-clone.test.ts`, `tui-p11.test.ts`. - apps/p2/docs/changes.md (41035 bytes)
# Chord and p2: decided changes Changes decided while walking through Chord's concepts. Each entry says what changes, why, and what it touches. The loader and reload work is in `plugin-loading.md` (12.1, 14); the open redesign of resources, listeners and lifecycles is still being discussed and is not listed here until decided. ## 1. Terminology: facet > object type > object, reference - **facet:** a plugin's code for one surface (`server.ts`, `worker.ts`, `web.tsx`, `tui.ts`); it serves object types. - **object type:** declared by a contract: name, input (the address), members. - **object:** one instance of an object type (Chord's code calls it a record). - **reference:** what `get` returns (Chord's code: handle; the bookkeeping behind it: hold). Code and docs (packages/chord, apps/p2) are renamed to these words: `record` → object, `Handle`/`Hold` → reference, behavior → the object's code, resource spec → resource declaration, epoch → code version. **An object's variant is named `variant`, not `kind`.** Factories return `{ variant: "editor", … }`, callers check `ref.variant === "editor"`. Other `kind`s stay: member descriptions (`kind: "method" | "state" | "opaque"`, `contract.ts:58-60`, part of the fingerprint), the link peer kind (`"host" | "client"`, hello frame), principals, `Author.kind`, app types. Sites: `RESERVED_MEMBERS` (`contract.ts:98`: `kind` → `variant`), `kindOf` (`record.ts`) and `VariantBehavior` (`contract.ts:253`), the shape frame's `kind` (`wire.ts:79`, read at `connection.ts:864,990-994`), `hosts/plugins.ts:295`, and every `ref.kind` / `{ kind: "<variant>" }` in plugin factories and UI code. **Wire.** Renaming a frame field breaks old peers: a stale browser tab or TUI client would read no variant and fall back to `LOCAL_VARIANT` (`connection.ts:990-994`), so every call fails with `not_in_variant`. The link hello (`connection.ts:332`) gets a protocol version; on a mismatch the client is told to reload instead of continuing. ## 2. The object type says where it lives; callers stop passing `at` **Today.** A caller names the host with every `get`: `get(Workspace, { session, conversation }, { at: \`session:${session}\` })`. p2 has 59 literal `at: "server"` and 26 `` at: `session:${id}` `` (built by hand from strings), plus `at` passed along as a value: the `at` option of `useObject` and `holdLingering` (`src/runtime/web.tsx:174,178`, `src/runtime/linger.ts:63`), the positional `at` of the TUI `watch` (`src/runtime/tui.ts:84-109`), `src/web/file-completion.tsx:59`, and several hundred in tests. **Problems.** - Stringly typed: a typo (`"sever"`, `"sesion:"`) shows only at runtime. - Where an object lives is knowledge every caller must carry. Moving a type from the server to the worker breaks every caller, at runtime (`unknown type`). - Redundant: the session is usually already in the input; callers write it twice. **Change.** The contract declares its host declaratively, through typed helpers. A helper carries a static pattern (`server`, `session:*`) and a resolver from the input to the host name: ```ts export const Workspace = remote("workspace", { input: Type.Object({ session: Type.String(), conversation: Type.Number() }), host: sessionHost((input) => input.session), // pattern "session:*" … }); export const Sessions = remote("sessions", { input: Type.Object({}), host: serverHost, … }); // pattern "server" useObject(Workspace, { session, conversation }); // no `at` ctx.get(Sessions, {}); ``` - One place knows where a type lives: its definition. Host names come from the helpers, never from free strings. - Local types (same host only) have no `host`. - The resolver runs on the normalized input (Chord normalizes before resolving), so defaults count. - `serve` checks the pattern against `host.name` ("Workspace is declared for `session:*`, but `server` serves it"). This covers types served outside facets too (`WorkerBoundary`, `PluginRemoval`, `Harness`), so a misplaced type fails at startup, not at the first call. - The serving host checks, at acquisition, that the normalized input resolves to itself. A destination chosen by a client is never trusted. - Placement is part of the contract fingerprint. Moving a type is a breaking contract change: existing references get `upgrade_required` and clients reload. References are not relocated live. Types served on session workers with an empty input get a `session` input (version bump) and check `host.name === \`session:${session}\`` in the factory: `WorkerBoundary` (`src/runtime/worker.ts:174`, caller `plugins/sessions/server.ts:173`), `SessionTransfer` (`plugins/sessions/contract.ts:207`, callers `plugins/sessions/server.ts:291,300`), `PluginRemoval` (`src/hosts/plugins.ts:107`, callers `src/hosts/server.ts:301,333`). **Touches.** Chord (via pi first): `host` helpers with pattern and resolver in `remote(...)` contracts, address resolution without `at`, the check at `serve` and at acquisition, placement in the fingerprint; `at` stays only for the transition. p2: `host` on every remote contract, the three input changes above, removing the 85 literal options and the plumbed `at` (`useObject`, `holdLingering`, `linger.ts`, TUI `watch`, `file-completion.tsx`), tests. ## 3. Typed principals, all with a `kind` **Today.** Chord types a principal as `unknown` (`packages/chord/src/old/types.ts:5`); factories receive `unknown` and p2 narrows it with helpers (`isOwner`, `isGuest`, `isWorker` in `src/runtime/principals.ts`). Chord also invents principals of another shape, without `kind`, outside the app's type: `{ host: true }` for a host's own code (`host.ts:37`), `{ host: parentName }` for the parent's own requests as a child sees them (`connection.ts:651`; in a worker: `{ host: "server" }`), `{ host: hostName }` as the default for a routed host calling back (`host.ts:870`). Chord recognizes the host principal by object identity (`principal === this.principal`, `host.ts:367,375,411`). Six places in p2 cast by hand: five check `(principal as { host?: unknown }).host !== "server"` (`plugins/search/worker.ts:167`, `plugins/conversations/worker.ts:124`, `plugins/sessions/worker.ts:67`, `src/runtime/worker.ts:192`, `src/hosts/plugins.ts:222`), one checks `.host === true` (`mayReach`, `src/runtime/server.ts:159`). p2's owner principal is confusing: `via: "device"` means the local machine, `via: "relay"` covers both Radius and remote cookie devices, `device` is optional, and the owner has no name (`authorOf` returns "Owner", the web prints the literal "Owner": `src/web/Chat.tsx:742,772`). **Change, Chord (via pi first).** The principal type is a type parameter of the host (`createHost<P>(...)`); factories, `ctx`, `principalKey` and `invalidatePrincipal` receive `P` and TypeScript narrows on `kind`. Chord invents no principals: the app supplies the principal of the host's own code (`createHost({ self: … })`) and the one the parent's own requests carry (`host.connect(link, { principal: … })`); relayed requests keep the caller's principal; `route.principal` is mandatory (no `{ host }` default). Chord compares the host principal by `principalKey`, not by object identity, so an equal principal never creates a second object. **Change, p2.** ```ts /** The person running this p2 (one today). */ export type OwnerPrincipal = | { readonly kind: "owner"; readonly access: "local" } // a browser on this machine (loopback), no owner cookie | { readonly kind: "owner"; readonly access: "relay" } // via Radius, authenticated by the relay | { readonly kind: "owner"; readonly access: "device"; readonly device: string }; // owner cookie, also on loopback /** Someone who joined one session through a share link. */ export type GuestPrincipal = { readonly kind: "guest"; readonly id: string; readonly name: string; // self-chosen, unverified readonly session: string; readonly mode: "view" | "full"; readonly revision: number; // grant revision: removal, link reset or archive invalidates older ones }; /** A session's worker calling the server. */ export type WorkerPrincipal = { readonly kind: "worker"; readonly session: string; readonly generation: number }; /** The server's own code calling into a session worker (fork, transfers, the conversation broker). */ export type ServerPrincipal = { readonly kind: "server" }; /** A host's own code calling objects on the same host. */ export type HostPrincipal = { readonly kind: "host" }; export type P2Principal = OwnerPrincipal | GuestPrincipal | WorkerPrincipal | ServerPrincipal | HostPrincipal; ``` - The five `host !== "server"` casts become `principal.kind !== "server"`, keeping their extra checks (`host.name`, session); `mayReach`'s `.host === true` becomes `principal.kind === "host"`. - `principalKey`: `owner:local`, `owner:relay`, `owner:device:<id>`, `guest:<id>:r<revision>`, `worker:<session>:<generation>`, `server`, `host`. - **Owner name.** Not in the principal: a link's principal is stamped once at accept, and the key decides which tabs share an object, so a name in it would go stale. A setting `ownerName` (default "Owner", validated) is read when writing: `authorOf` gets it from the worker's init values or a server value. The web renders `author.name` instead of the literal "Owner" (`src/web/Chat.tsx:742,772`). Names in Durable entries already written stay as written. - "Revoke all remote owners" (today one pattern `{ kind: "owner", via: "relay" }`, `src/runtime/server.ts:72`) becomes two propagating pattern calls, `access: "relay"` and `access: "device"` (patterns match equal fields; predicates do not propagate to child hosts, `host.ts:125`). - Call sites of `via`: stamping in `src/hosts/server.ts:418,426,427` (a device-cookie owner stays `access: "device"` on loopback), the loopback owner cookie (`src/hosts/server.ts:537`), Radius (`plugins/radius/server.ts:54`), owner cookies (`plugins/devices/server.ts:56`), revocation (`src/runtime/server.ts:72`). Checks of `via === "device"` become `access === "local"`: sharing join (`plugins/sharing/server.ts:260`), login `remote` (`plugins/login/server.ts:169`), owner-link minting and `canMint` (`plugins/devices/server.ts:115,170,175`). **Later, not planned.** Several people (individuals, teams, orgs): users with IDs and roles instead of one owner, sessions owned by a user or team, `canView`/`canSend` by membership. That touches every `isOwner` check (some mean "admin", some "may use this session") and the single-tenant data directory (`models.json`, `auth.json`). ## 4. A `files` tool: send and open (low priority) **Today.** `send_file` (`plugins/send-file/tool.ts`) copies a snapshot and shows a download card (`src/web/Chat.tsx:504`). The `read`, `write` and `edit` cards already link into the workspace ("Open in workspace", `src/web/Chat.tsx:335-342,388,399,543`). What is missing is a tool that only points the user at a file or directory, without reading or writing it. The workspace route can show one: `{ workspace: { tab: "files", directory, file: { path, view: "file", line } } }` (`src/web/navigation.ts`). **Change.** One tool, `files`, with an `action`: - `send` (path, optional name): today's `send_file`. - `open` (path, optional line): resolves the path with `env.canonicalPath` through the session's environment, then checks that it is a file or directory inside the conversation's canonical cwd (the workspace shows nothing else). It copies nothing and returns `details: { action: "open", path (canonical, cwd-relative), kind, line? }`; the web derives `directory` from `path`. Replay: Durable's replay policy is per registration, so the whole `files` tool stays `replay: "unsafe"` like `send_file` (`plugins/send-file/tool.ts:20`), unless `send` becomes idempotent. Web: the card shows the file with "Open in workspace →", navigating to the workspace route above; the link is hidden for view-only guests, whom the workspace refuses (`plugins/workspace/worker.ts:177`, `canSend`). Undecided: also open it automatically when the call finishes while the conversation is open in that tab (client-side only, never when loading history; only for owner and full guests). TUI: no docked workspace; open it in the file view or show the path. `/downloads` lists `files` send calls as well as old `send_file` calls. **Open question: old names.** Durable stores tool names in the `pi.agent` selections (`tools`, `packages/durable/src/harness/agent.ts`), and pending calls resolve by exact name (`packages/durable/src/harness/tool.ts:210`). Subagents store `"send_file"` in their tool list (`CODING_TOOLS`, `plugins/subagents/tools.ts:31`), and models copy calls from their history. An alias in the subagent registry is therefore not enough. Either keep `send_file` registered as a hidden compatibility tool (old arguments, unsafe replay, not offered to new configurations), or migrate the `pi.agent` selections. **Touches.** Old Durable entries are named `send_file`: the web card, the TUI renderers and `/downloads` (`toolViews("send_file")`, `plugins/send-file/tui.ts:99`) keep rendering that name. Rename in `CODING_TOOLS` and the tool descriptions (`plugins/subagents/tools.ts:31,279`, `plugins/agent/tool.ts:49`), the fake provider (`plugins/models/fake.ts:22`) and tests. ## 5. `lifetime` instead of `eager` **Today.** `host.serve(Type, factory, { eager: true })`: the host creates the object when the facet loads, `load` waits until it is live unless `awaitEager: false`, and the host holds it until it closes. Only such objects may register into registries (`record.ts:1505`). `eager: { input }` allows another input (`record.ts:50`), and any host-principal `get` of an eager type with another input also gets the eager flag (`host.ts:410-411`). The name says when the object is created, not what matters: how long it lives. **Change.** An explicit lifetime: | `lifetime` | Created | Ends | `linger` | | --- | --- | --- | --- | | `"referenced"` (default, when omitted) | at the first `get` of its address | after the last reference is released, plus `linger` | allowed (default 0) | | `"host"` | when its facet loads; loading waits until it is live (see below) | when the host closes, or a reload removes its facet | not allowed | - `"host"` is allowed only for local types, and the object has exactly one address: an empty input (or only defaults), the host principal. Other inputs are rejected; `eager: { input }` is dropped. Others reach it with `get`; their references do not change its lifetime. - Loading waits for `"host"` objects by default. An explicit option (today's `awaitEager: false`, renamed) defers this: a worker uses it to avoid the startup deadlock with the server, whose link is admitted only after the worker reports ready (`src/hosts/worker.ts:49-62,86-93`). - A `"host"` object whose activation fails is retried at the next reload, as today (`reload.ts:586`). - Registrations stay limited to `"host"` objects. - Most of today's `eager: true` objects are `contrib - apps/p2/docs/chord-next-walkthrough.md (105759 bytes)
# Chord v2 and its plugins: a guided walkthrough A step-by-step introduction to how plugins work on Chord v2 (`packages/chord/src`, used by the playground and p3), for someone who knows nothing about either. It teaches the **design**: the target design `packages/chord/docs/design.md` together with the decision log `packages/chord/docs/decisions.md` (the log is newer and wins where they differ). It does not describe the state of the code. API names are the design's working names; where the kernel already has a name (`packages/chord/src`), the examples use it. Points the design has not settled are marked **Open** and list their options. Nothing in an Open block is decided. --- ## For the agent guiding the user You walk a user through this document. Follow these rules: 1. **One concept at a time.** Explain it in your own words, in the user's language, with the document's example. Keep it concrete; don't paste whole sections. Explain in full sentences in this document's tone, not as terse notes; define every word the user may not know when it first appears (e.g. IPC, relay). 2. **Then ask the check questions** of that concept and **wait** for the answers. 3. **Judge the answers** against the model answers (they are for you). If an answer is wrong or half right, say precisely what is missing, explain that part differently, and ask again. Don't move on while a concept is unclear. 4. **Counter-questions come first.** Answer a user's question completely before anything else. If the answer lies in a later concept, give a short answer and say where it continues. 5. **Move on only when the user says so.** 6. **No terms before they are introduced.** If you need one, introduce it first. 7. **Teach the design, not the implementation status.** For questions beyond this document, answer from, in this order: `redesign-decisions.md` (newest decisions, wins on conflicts), `packages/chord/docs/design.md` (the target design), `packages/chord/docs/kernel.md` and `step-*.md` (names the kernel already uses), `plugin-loading.md` (loading, project plugins), `questions.md` (the Questions service), `research/chord-on-durable-objects.md`. `target-design.md` and `p3-walkthrough.md` are superseded. 8. **Open is open.** Present every Open point with its options; never present one option, or your own idea, as decided. If a question has no answer in the sources, say so plainly. 9. If you find a gap or a contradiction in the sources while teaching, tell the user and note it for whoever maintains the design. 10. Keep a short list of what the user found hard; come back to it at the end. Start by telling the user the note below, then begin with "The setting". The setting has no check questions: ask whether the user has questions, then move on when they say so. ## For the user **Keep asking until you really understand a thing.** There are no silly questions. The concepts build on each other; a gap early on makes everything after it harder. If an example doesn't click, ask for another one. If a word is unclear, ask what it means. --- ## The setting and the running example **Chord** is the runtime under an app: it lets plugins define **objects** that live in one place and lets every part of the app use them, also across process boundaries. An app on Chord runs as several participants. Chord calls each participant a **host**. In a Node app (the playground, p3) there are three roles: ``` browser browser TUI clients: any number, not addressable └─────────┼───────┘ server exactly one ┌─────────┼─────────┐ worker S1 worker S2 worker S3 one per session, started on demand ``` - **The server** (exactly one) serves the UI, holds data shared by all sessions and starts the workers. - **A worker** (one per session) runs the agent and works on that session's files. The server starts it as a child process when the session is first needed. - **Clients** are browser tabs and TUIs (terminal UIs). They connect by themselves; nobody can send a request *to* a client, so clients are not addressable. A client's own objects are therefore only ones nobody outside it needs (concept 6 names them: `local` types). **How they connect.** Each connection between two hosts is a **link**: - Clients connect only to the server, over a WebSocket. - The server talks to each worker over **IPC** (inter-process communication: the pipe between a parent process and the child it started). - Clients never talk to a worker directly; the server **relays** (passes requests on). A browser call to something in session S1 travels browser → server → worker S1, and the answer comes back the same way. Workers reach each other through the server too. - When a client connects, the server checks who it is (for example its cookie) and attaches that to the whole link. Concept 3 builds on this. **The goal that shapes everything: reload.** You change a plugin's code and the new code takes over inside the running app. Two words matter: - a **reload** swaps a plugin's code while the hosts keep running: data, open connections and running work survive where they can; - a **restart** ends a process and starts it again: everything it held in memory is gone. Reloads are always explicit (a button or a command), never on save. Changes to Chord itself or to the app's runtime are **base code**: they restart the hosts. Much of the design exists to make reloads safe; concept 23 shows the reload step by step. **Base code and plugins.** Whether something is "core" or a plugin is decided by where its code lives: - **base code** (the app's `src/`): imported directly by the start code under a fixed name such as `@app/core`; always there, not switchable; a change restarts the hosts. Core is what the app always needs and everything builds on (the agent harness, the session list, access, the session location). - **a plugin** (its own directory): found by listing, loaded by the plugin loader, switchable; a change reloads it. Optional features are plugins. Both use exactly the same building blocks; base code is not special to Chord. **A plugin** is a folder with a `package.json`. Its `name` (globally unique and stable, like an npm package name, e.g. `@playground/notes`) is the plugin's identity: Chord identifies the plugin's facets by it (their stored documents, reloads, error messages), never by a directory name or load order. Base code's core facet gets a fixed name the same way (`@app/core`). Each other file is meant for certain hosts; the part of a plugin for one kind of host is a **facet**: | File | Runs on | Holds | | --- | --- | --- | | `package.json` | (read by the loader) | the plugin's `name` | | `contract.ts` | every host that imports it | descriptions only: names, shapes, method signatures | | `server.ts` | the server | the server facet | | `worker.ts` | every session worker | the worker facet | | `web.tsx` | the browser | the web facet (UI) | | `tui.ts` | the TUI | the TUI facet | **The running example: a notes plugin.** The whole walkthrough builds this one plugin, piece by piece: - notes per session, kept across restarts; - a web panel shows the notes; the owner and guests with write access can add and remove notes; read-only guests can only read, and don't see who wrote a note; - a text file dropped into the session's `inbox/` folder becomes a note; - notes older than 30 days are deleted every hour; - importing a large file runs in the background and can be cancelled; - `GET /s/:session/notes.txt` downloads all notes; - the panel shows how many people are looking at the notes. --- ## Concept 1: Object types and objects **The problem.** Several places need "the notes of session S1": the owner's tab, a guest's tab, the download route. If each built its own copy, the copies would drift apart, and nobody would know when to clean them up. **The idea.** There is **one object** for "the notes of S1". Everyone asks Chord for it by its **address** and gets that same object. - An **object type** describes what such an object is: its name, what its address looks like (the **input**), where it lives, what you can do with it (its **members**: data you can read and methods you can call) and a version. It contains no code that runs; every host can share it. Another word for it is **contract**. - A **factory** is the code that builds an object of that type, on the host where it lives. - An **object** is one concrete instance: "Notes for `{ session: "S1" }`". The three pieces live in three files of the notes plugin: ```ts // contract.ts: the object type export const Notes = remote("notes", { version: 1, host: /* where it lives: concept 2 */, input: Type.Object({ session: Type.String() }), // the address: { session: "S1" } (a TypeBox schema) variants: { /* the members: concept 3 */ }, }); ``` ```ts // worker.ts: the factory, on every session worker export default defineFacet((host, self) => { host.serve(Notes, (input, principal, self) => ({ /* members */ })); }); ``` ```ts // web.tsx: a caller in the browser const notes = await host.get(Notes, { session: "S1" }); // browser → server → worker S1 ``` Read `host.serve` as "this host builds `Notes` objects like this". `principal` comes in concept 3, `self` in concept 7. - **Who builds it:** the host where the object lives runs the factory. That is why the `Notes` factory is in `worker.ts`. - **One object per address:** the first `get` for `{ session: "S1" }` runs the factory; every later `get` for the same address gets the same object. `{ session: "S2" }` is another object, on another worker. (Concept 3 refines this.) - **Methods return data, never objects.** If a method wants to point you to an object, it returns its address (`{ session: "S1" }`) and you `get` it yourself. **Check questions** 1. Two tabs ask for the notes of S1. How often does the factory run? 2. What is the difference between `Notes` and "Notes for S1"? 3. Why does the factory live in `worker.ts`, not in `server.ts`? *Model answers:* (1) Once: same address, same object (concept 3 refines this when the callers are different people). (2) `Notes` is the description (the type); "Notes for S1" is one instance (an object). (3) Notes of a session live on that session's worker, and the host where an object lives runs its factory. --- ## Concept 2: Where objects live: host kinds and placement **The problem.** A caller asks for "Notes for S1". Chord must find the host that holds it, without the caller knowing host names, and the worker must be sure the object really belongs to it. **The idea.** The app describes its addressable hosts as typed **host kinds**; one concrete host of a kind is an **address**. A type's `host` field is a **placement function**: input in, address out. ```ts // the app's runtime (base code, not a plugin) export const Server = hostKind("server"); // one host export const Session = hostKind("session", Type.Object({ id: Type.String() })); // one host per id Session({ id: "S1" }) // an address: the worker of session S1 Server() // the address of the server ``` ```ts // notes/contract.ts export const Notes = remote("notes", { version: 1, host: ({ session }) => Session({ id: session }), // typed: `session` must exist in the input input: Type.Object({ session: Type.String() }), variants: { … }, }); ``` **A call, step by step:** ``` browser get(Notes, { session: "S1" }) runs the placement function → Session({ id: "S1" }) has only a link to the server → sends the request there, with the address server the address is not mine; the app told me how to reach Session hosts (concept 4) → link to worker S1 (started if needed), forward worker S1 runs the placement function too → Session({ id: "S1" }) = me → runs the factory ``` - **The caller computes, the provider checks.** The caller runs the function on the normalized input and sends the result. The provider runs it too and refuses an object that is not its own with the error `invalid/placement`; a worker for S2 never builds "Notes for S1". (During a rollout, concept 24, such a refusal is handled like a mismatch: a provider on another revision makes the call park; on the same revision it fails.) - **No host-name strings** in app code. Addresses are typed values; Chord serializes them internally. - **Placement is not part of the type's fingerprint** (the hash Chord uses to compare two versions of a type, concept 26). If the rule changes between versions, the provider's refusal surfaces it. - A function is type-checked: a `Session` placement on a type without a `session` field does not compile. **Check questions** 1. A tab asks for `Notes { session: "S2" }`. Which host builds it, and who computed that? 2. Why does the worker run the placement function again, if the caller already did? 3. What does app code never write when it names a host? *Model answers:* (1) The worker `Session({ id: "S2" })`; the caller computed the address with the type's placement function, the server routed by it. (2) To refuse objects that are not its own (a wrong or outdated rule at the caller must not create an object in the wrong place). (3) A host-name string; it calls the host kind (`Session({ id })`). --- ## Concept 3: Who is calling: principals and variants **The problem.** The owner may add and remove notes; a read-only guest may only read and should not even see who wrote what. The object must know who calls, and the caller must not be able to lie about it. **The idea.** Every request carries a **principal**: who calls. The app defines its principal type; every variant has a `kind`. p3's principals: | `kind` | Who | | --- | --- | | `owner` | the person running the app (`access: "local" \| "relay" \| "device"`; a paired device has its `device` id) | | `guest` | someone invited into one session (`session`, `mode`, its `grant` and the grant's revision) | | `server` | the server calling a worker | | `worker` | a session worker calling the server (`session`) | - **Links stamp principals.** A principal is never taken from a request's data: the server authenticates a client when it connects and stamps that principal on the whole link; the server tells Chord who each worker is when it connects it (concept 4). A caller cannot claim to be someone else. **Where principals come from.** Chord never sees cookies, share links or grants; that is app code (in p3 the sharing feature, `p3-functions/sessions-access.md`). Chord only receives the finished principal and stamps it on the link: ``` owner: "copy share link" for S1, read/write sharing (app code on the server) creates a link token for (S1, write) guest opens the link → join page → enters a name sharing stores a guest record (grant g1) and sets a guest cookie guest's tab connects (WebSocket) entry point (app code): authenticate(req): cookie → gu - apps/p2/docs/chord-walkthrough.md (45280 bytes)
# Chord and p2: a guided walkthrough A step-by-step introduction to how p2 plugins work on top of Chord, for someone who knows nothing about either. It describes the design (`target-design.md`), not the state of the implementation. --- ## For the agent guiding the user You walk a user through this document. Follow these rules: 1. **One concept at a time.** Explain it in your own words, in the user's language, with the example from the document. Don't paste the whole section; keep it short and concrete. 2. **Then ask the check questions** of that concept (2–3) and **wait** for the answers. 3. **Judge the answers** against the model answers (they are for you, not for the user). If an answer is wrong or half right, say precisely what is missing, explain that part again differently, and ask again. Don't move on while a concept is not understood. 4. **Counter-questions come first.** If the user asks something, answer it completely before anything else. If the answer lies in a later concept, say so, give a short answer, and offer to continue there. 5. **Move on only when the user says so.** 6. **No terms before they are introduced.** If you need one, introduce it first. 7. **Teach the design, not the implementation status.** Don't say what exists today and what is planned. When a question goes beyond this document, answer from these sources, in this order: `target-design.md` (the design; it wins on conflicts), `changes.md` (the reasoning behind each change), `packages/chord/docs/tutorial/` (Chord's tutorial: plugin files, the app's API, how an app loads plugins), `packages/chord/docs/design.md` (Chord's API and links). The code lives in `packages/chord` and `apps/p2`; it may lag behind the design. 8. Keep a short list of what the user found hard, and come back to it at the end. Start by telling the user the note below, then begin with concept 1. ## For the user **Keep asking until you really understand a thing.** There are no silly questions. The concepts build on each other: a gap early on makes everything after it harder. If an example doesn't click, ask for another one. If a word is unclear, ask what it means. --- ## The setting and the running example **p2** is a coding-agent app built from plugins. It runs as several processes. Chord calls each process a **host**, and every backend host has a name: - the **server** (name `server`): exactly one; serves the web UI, holds data shared by all sessions, starts the workers; - a **worker** per session (name `session:S1` for session S1): runs the agent and works on the session's files; - **clients**: browsers (the web UI) and TUIs (a terminal UI), any number. **How they connect:** ``` browser ──┐ WebSocket IPC ┌── worker session:S1 browser ──┼───────────── server ─────┤ TUI ──────┘ └── worker session:S2 ``` - Each connection is a **link**. Clients connect only to the server, over a WebSocket. The server starts a worker as a child process when its session is first needed and talks to it over IPC (the pipe between a parent process and its child). - Clients never talk to a worker directly; the server **relays**. A browser call to something in session S1 travels browser → server → worker `session:S1`, and the answer comes back the same way. Workers reach each other through the server too. - When a client connects, the server checks who it is (its cookie) and remembers that for the whole link. Concept 2 builds on this. **Chord** is the runtime under p2: it lets plugins define **objects** that live on one host and lets any host use them across links. **The goal that shapes everything: reload.** You change a plugin's code, and the new code takes over inside the running app. No host restarts: the server, the workers and the browsers keep running, and so do open terminals, running scripts, the agent's current answer and connected browsers. Only the changed plugin's code is swapped. This is a **reload**; the opposite is a **restart**, where a process ends and starts again and everything it held in memory is lost. Much of what follows exists to make reloads safe: what must survive a reload (data, open files), what must be replaced (behaviour), and how running work is handed over. Concept 11 shows how a reload works step by step; until then it is enough to know: **on a reload, the plugin's factories run again with the new code, in the same running process.** **A plugin** is a folder under `plugins/`. Each file is meant for certain hosts: | File | Runs on | Holds | | --- | --- | --- | | `contract.ts` | every host that imports it | descriptions only: names, shapes, method signatures | | `server.ts` | the server | code for the server | | `worker.ts` | every session worker | code that needs the agent or the session's files | | `web.tsx` | the browser | UI components | | `tui.ts` | the TUI | terminal UI | The running example is a **notes** plugin: - notes per session, stored in SQLite; - a text file dropped into an inbox folder becomes a note; - notes older than 30 days are deleted every hour; - a web panel shows the notes; the owner and invited guests with write access can add notes, read-only guests can only read. --- ## Concept 1: Object types and objects **The problem.** Several places need "the notes of session S1": the owner's browser tab, a guest's tab, an HTTP export handler. If each built its own copy, they would drift apart, and nobody would know when to clean up. **The idea.** There is **one object** "notes of S1". Everyone asks Chord for it by its **address** and gets the same one. - An **object type** describes what such an object is: its name, what its address looks like (the **input**), on which host it lives, and what you can do with it (its **members**: data you can read and methods you can call). It contains no code that runs; it is a description every host can share. Another word for it is **contract**. - A **factory** is the code that builds an object of that type, on the host where it lives. - An **object** is one concrete instance: "Notes for `{ session: "S1" }`". The three pieces live in three files of the notes plugin: ```ts // plugins/notes/contract.ts: the object type, imported by the worker and the browser export const Notes = remote("notes", { // a type usable across links, named "notes" version: 1, // the contract's version (concept 12) host: sessionHost, // lives on the worker "session:" + input.session input: Type.Object({ session: Type.String() }), // the address: { session: "S1" } (a TypeBox schema) variants: { /* the members, concept 2 */ }, }); ``` ```ts // plugins/notes/worker.ts: the factory, runs on every session worker export default defineFacet((host) => { // runs when the plugin loads; host = this worker (concept 7) host.serve(Notes, (input, principal, self) => { // "this host builds Notes objects like this" // input = { session: "S1" }; principal: concept 2; self: concept 3 return { /* members */ }; }); }); ``` ```ts // plugins/notes/web.tsx: a caller, in the browser const notes = await host.get(Notes, { session: "S1" }); // browser → server → worker session:S1 ``` - **Where an object lives** is decided by the type alone. p2 defines two placements (`src/runtime/chord.ts`): - `sessionHost = hostFamily("session:", "session")`: the host name is the prefix `"session:"` plus the input's `session` field. For `{ session: "S1" }` that is `session:S1`. The input must have that field; - `serverHost = hostNamed("server")`: always the host `server`. The caller only names the type and the input; Chord computes the host and sends the request there through the server. - **Who builds it:** the host where it lives runs the factory. That is why the factory is in `worker.ts`: `server.ts` could not serve `Notes`, and Chord refuses that at startup. - **One object per address:** the first time anyone asks for `{ session: "S1" }`, Chord runs the factory. Everyone who asks later for the same address gets the same object (concept 2 refines this). `{ session: "S2" }` is another object, on another worker. - **`local` and `remote`:** a `local(...)` type can only be used on its own host; a `remote(...)` type can also be used from other hosts across links (here: the browser using an object on the worker). **Check questions** 1. Two tabs ask for the notes of S1. How many objects exist, and how often did the factory run? 2. What is the difference between `Notes` and "Notes for S1"? 3. Where does the object for S2 live, and who decides that? *Model answers:* (1) one object, the factory ran once (same address; concept 2 refines this when the callers are different people). (2) `Notes` is the description (type); "Notes for S1" is one instance (object). (3) In the worker of session S2; the type's `host` declaration decides, derived from the input. --- ## Concept 2: Who is calling (principals and variants) **The problem.** The owner may do everything; a read-only guest may only read; a guest of another session may do nothing. The object must know who calls, and the caller must not be able to lie. **The idea.** Every call carries a **principal**, the caller's identity. It is never taken from the message: the link stamps it, from what the server learned when the client connected (its cookie, see the setting). When the server relays a call to a worker, it passes the principal along; a worker accepts a passed-along principal only from its parent link, the server. So nothing a client sends, at any hop, changes who it is. p2's principals, each with a `kind`: | `kind` | Who | | --- | --- | | `owner` | the person running p2 | | `guest` | someone invited into one session: `{ kind: "guest", session: "S1", mode: "view" \| "full", … }` | | `worker` | a session worker calling the server | | `server` | the server calling a worker | | `host` | a host's own code calling an object on the same host | The factory gets the principal and decides what the caller gets. An object type can have **variants**: named sets of members. The factory returns one of them, named by `variant`. ```ts // plugins/notes/contract.ts: the variants of Notes variants: { reader: { state: { notes: NotesSchema } }, // can read the notes editor: { state: { notes: NotesSchema }, methods: { add: method(AddArgs) } }, // can also add one } ``` ```ts // plugins/notes/worker.ts: the factory picks the variant per caller host.serve(Notes, ({ session }, principal, self) => { if (!canView(principal, session)) throw forbidden(); // refuse: the caller gets an error, no object const notes = loadNotes(session); // simplified; see the problem below if (!canSend(principal, session)) return { variant: "reader", notes }; return { variant: "editor", notes, add: (args, ctx) => addNote(session, args, ctx) }; }); ``` `canView` and `canSend` are p2 helpers (`src/runtime/principals.ts`): - `canView`: the owner, or a guest of this session; - `canSend`: the owner, or a guest of this session with `mode: "full"`. - **A reader has no `add` at all.** Chord refuses calls to members that are not in the caller's variant, so `add` needs no check of its own. - **The caller sees which variant it got.** A reference is typed as one of the variants, told apart by `variant`. TypeScript lets you use `add` only after checking `notes.variant === "editor"`; at runtime, using a member of another variant throws `not_in_variant`: ```ts // plugins/notes/web.tsx: a caller const notes = await host.get(Notes, { session: "S1" }); notes.add({ text: "hi" }); // type error: `add` exists only on the editor variant if (notes.variant === "editor") { await notes.add({ text: "hi" }); // fine: narrowed to editor } ``` - **One object per (type, input, caller).** The factory builds something different per caller, so each caller gets its own object. Concept 1's "one object per address" holds per caller: two tabs of the owner share one `Notes`; two guests get one each. **The problem this creates.** If each caller has its own `Notes`, and the notes were loaded inside it, there would be one copy per caller again: exactly what concept 1 wanted to avoid. **Store plus view.** The data goes into a second object type that everyone shares, and the per-caller object only checks and forwards: ```ts // plugins/notes/worker.ts: a local type, defined where it is used const NoteStore = local("notes.store", { input: Type.Object({ session: Type.String() }), members: { notes: …, add: method(AddArgs) }, }); host.serve(NoteStore, ({ session }, principal, self) => ({ /* loads and holds the notes of one session */ })); host.serve(Notes, ({ session }, principal, self) => { if (!canView(principal, session)) throw forbidden(); const store = self.get(NoteStore, { session }); // get another object from inside a factory (concept 3) if (!canSend(principal, session)) return { variant: "reader", notes: store.notes }; return { variant: "editor", notes: store.notes, add: (args, ctx) => store.add(args, ctx) }; }); ``` - `NoteStore` is `local`: only code on this worker can get it, so no browser ever reaches it directly. - Every `Notes` factory asks for it with the same principal, `host` (this worker's own code), so there is one `NoteStore` per session, shared by all `Notes` objects of that session. - This "store plus view" pattern is everywhere in p2. **Check questions** 1. A read-only guest calls `add`. What happens, and where is it decided? 2. Could a malicious browser claim to be the owner? Why not? 3. The owner has a tab open, two guests are in S1. How many `Notes` and how many `NoteStore` objects exist? *Model answers:* (1) The guest got the `reader` variant (the factory decided), which has no `add`; Chord refuses the call. (2) No: the link stamps the principal from the server's authentication of the connection, never from a message; when relayed, the worker accepts it only from the server. (3) Three `Notes` (owner, guest A, guest B; a second owner tab would share the owner's), one `NoteStore` (every `Notes` gets it as `host`). --- ## Concept 3: References **The problem.** When can an object be cleaned up? Not while someone uses it, but not never either. **The idea.** Asking for an object gives you a **reference**, not the object. While at least one reference exists, the object lives. When the last one is released, Chord cleans it up (and the object releases the references it holds itself). There are three ways to get a reference; they differ in when it is released: | Where you are | How | Released when | | --- | --- | --- | | in a factory | `self.get(Type, input)` | the object holding it ends | | in a method | `await ctx.get(Type, input)` | that call ends | | outside Chord (startup code, UI) | `a - apps/p2/docs/codemode-tools/brief.md (13651 bytes)
# Code mode with many tools: brief For Armin. Mario and Claude, 2026-11-12. Durable (`packages/durable/src/codemode/`) and the coding agent (`packages/coding-agent/src/extensions/codemode/`) share the design below; nothing proposed here is built. ## Now The `codemode` prompt section lists callable tools, rendered by `codemodeSection` (`describe.ts`): - every listed tool as its description plus its full TypeScript declaration, a comment per field, schema constraints as JSDoc; - budget `inlineBudget` = 3,000 estimated tokens (chars / 4) for all declarations; - `selectCatalog`: ungrouped tools first, then namespaces by name; each round every group places its cheapest remaining tool; a group whose next tool does not fit drops out; - namespace heading + description always shown (outside the budget); deferred tools (MCP) never listed; - what does not fit: one generic line for ungrouped tools, **nothing** inside a namespace. One tool, as listed (about 130 tokens; `bash` about 200, `view_list` over 300): ```ts ### `artifact_list` The session's artifacts (id, kind, title, newest version, updated), the newest first. codemode tool declaration: declare const tools: { artifact_list(args: { kind?: "html" | "md"; }): Promise<{ artifacts: Array<{ // @pattern ^[a-z0-9][a-z0-9_-]{0,63}$ id: string; kind: "html" | "md"; // @minLength 1 @maxLength 200 title: string; updated: number; // @integer @minimum 1 version: number; }>; }>; }; ``` Discovery globals: ```ts ALL_TOOLS // every callable tool searchTools(query, { limit?, namespace? }): { name, description }[] // BM25; description includes the declaration describeTool(name): string // one tool describeNamespace(name): { name, description?, instructions?, tools: string[] } // names only ``` MCP: in p2 each server is a namespace (`mcp__radius`) with deferred tools; in the coding agent a separate `mcp_servers` section lists servers with their reach (`codemode` | `tool_search`). Both are built from configuration, so the prompt is byte-identical whether a server is connected or not. ## Problem A p2 session after last night's artifacts plugin: about 60 callable tools, about 20 fit. The section listed 9 of the 12 `artifact` tools in full; `artifact_write`, `artifact_edit` and `artifact_html_logs` lost the round-robin (long declarations sort late) and the namespace showed no hint. The agent found `artifact_write` only by calling `ALL_TOOLS`. | Measure | Now | | --- | --- | | tokens per listed tool | 130–300 | | tools that fit in 3,000 | ~20 | | callable tools in a p2 session (no MCP) | ~60 | | hint when a namespace is cut | none | | round trips to use a tool from a cut namespace | 2 (`describeNamespace` → names, `describeTool` → declaration) before the real script | | where namespace `instructions` reach the model | only `describeNamespace` | All seven models below put "a cut list reads as complete" first: the model concludes the tool does not exist and works around it (here: writes HTML with `bash`). ## Proposed ### The section Same session, same tools: ```text Scripts call `await tools.<name>(args)`; names are exactly as listed. Signatures are TypeScript, `?` = optional. A list is complete unless it ends in "+N more". A wrong tool, field or value throws with the right signature. Discovery: listTools(ns?) · searchTools(query | queries, { namespace?, limit? }) · describeTool(name | names). type Image = { type: "image"; data: string; mimeType: string } agent(args: { prompt: string; schema: object; model?: string; thinkingLevel?: string; tools?: string[]; label?: string }): unknown // One-shot subagent; answer validated against schema. read(args: { path: string; offset?: number /* line, 1-based */; limit?: number }): string | Image // Read a file (2,000 lines / 50 KB). write(args: { path: string; content: string }): string // Create or overwrite a file. edit(args: { path: string; edits: { oldText: string; newText: string }[] }): string // Exact replacements in a file. bash(args: { command: string; timeout?: number /* s */ }): { exitCode: number; output: string; truncated: boolean; fullOutputPath?: string } // Run a shell command. history_search(args: { query: string; before?: number; limit?: number }): string // Search this conversation's history. +6 more: history_read, reload_plugins, location_envs, location_dirs, location_move, send_file (describeTool(name)) ## artifact: Artifacts (html, md) of this session: write, edit, show inline or beside the chat. Write first, then show (transcript) or open (beside the chat). After each change to an open html page, read artifact_html_errors. type Artifact = { id: string; kind: "html" | "md"; title: string; version: number; updated: number } artifact_write(args: { id?: string; kind: "html" | "md"; title: string; content: string }): { id: string; version: number } // New artifact, or a new version of id. artifact_edit(args: { id: string; edits: { old: string; new: string }[] }): { version: number } // Exact replacements in the newest version. artifact_read(args: { id: string; version?: number }): Artifact & { content: string } artifact_list(args: { kind?: "html" | "md" }): { artifacts: Artifact[] } // Newest first. artifact_show(args: { id: string }): { id: string; kind: string } // In the transcript, for everyone. artifact_open(args: { id: string }): { view: string } // Beside the chat. +6 more: artifact_delete, artifact_html_errors, artifact_html_logs, artifact_html_eval, artifact_html_state, artifact_html_set_state (listTools("artifact")) ## view: Panes beside the transcript: list, open, close, arrange, resolve annotations. type Selection = { view: string; summary: string; data?: unknown; kind?: string; item?: string } view_open(args: { kind: string; item?: string; title?: string; id?: string }): { id: string } // markdown (item: a file), changes, or a plugin's kind. view_list(args: {}): { views: { id: string; kind: string; item?: string; title: string }[]; selections: Selection[]; annotations: Annotation[] } +3 more: view_close, view_arrange, view_resolve (listTools("view")) ## mcp__radius: MCP server "radius": organization artifacts and knowledge. Needs the user's sign-in (MCP panel). Tools: listTools("mcp__radius"). ``` The `+N more` lines are illustrative: at ~25 tokens a line all 60 tools fit in 3,000. Rules: | Rule | Detail | | --- | --- | | one line per tool | `name(args: {…}): Result // summary`; parameter names, `?`, literal unions, nested objects, result type | | results never elided | a large result shape becomes a hoisted `type` line, never `{…}` | | arguments elided only past one nested level | rendered as `{…}`; full form in `describeTool` | | short field docs inline | descriptions under ~40 chars as `/* … */` (units, bases: `/* s */`, `/* 1-based */`); longer ones and patterns in `describeTool` | | shared types hoisted | equal schemas within a group become one `type` line | | namespace instructions on the card | up to ~2 lines under the heading; the rest via `listTools` | | order | ungrouped first, then namespaces by name; inside a namespace `primary` first | | budget | ungrouped, then every namespace's `primary`, then the rest in proportion to namespace size | | overflow | 1. signature lines, 2. `+N more: a, b, c (listTools(ns))` with all names, 3. `+N more (listTools(ns))` only when names do not fit | | stability | rendered only from declared tools and configuration; same bytes until a tool is added or removed | Declaration additions: ```ts // ToolNamespace (harness/types.ts) readonly primary?: readonly string[]; // core tools: listed first, always with a signature while the namespace is shown // ToolRegistration readonly summary?: string; // one clause that tells the tool apart from its siblings; default: first sentence of description ``` ### Discovery ```ts listTools(ns?: string | string[]): string // heading, description, instructions, types and every signature line of ns (all namespaces without ns), // in the prompt's format; MCP state on its first line (below); replaces describeNamespace searchTools(query: string | string[], options?: { namespace?: string; limit?: number }): { results: { name: string; namespace: string | null; line: string }[]; instructions: Record<string, string>; // per namespace in results disconnected: { namespace: string; description: string; signIn?: string }[]; // configured MCP servers matching the query } describeTool(name: string | string[]): string // full declarations with field docs and constraints, plus the namespace's instructions ``` `ALL_TOOLS` stays but is no longer advertised. Errors carry the fix (nested-call path and schema validation): ```text Unknown tool artifact_create. Did you mean: artifact_write(args: { id?: string; kind: "html" | "md"; title: string; content: string }): { id: string; version: number } artifact_write: id "My Page" does not match ^[a-z0-9][a-z0-9_-]{0,63}$; expected artifact_write(args: { id?: string; kind: "html" | "md"; title: string; content: string }): { id: string; version: number } ``` Typical cold use: two scripts (discover, work). Listed tools: one. ### MCP Before (p2): ```text ## mcp__radius Tools of the MCP server "radius" (find them with searchTools()). It needs the user's sign-in: if it has no tools, ask the user to sign in from the MCP panel in p2's sidebar or at /mcp/sign-in/radius on p2. ``` Before (coding agent): ```text MCP servers whose tools are not declared to you. Call the tools of `codemode` servers from codemode scripts. - mcp__github (codemode): GitHub issues, pull requests and repositories - mcp__linear (tool_search): Linear issues and projects ``` After (both): ```text ## mcp__radius: MCP server "radius": organization artifacts and knowledge. Needs the user's sign-in (MCP panel). Tools: listTools("mcp__radius"). ## mcp__github: MCP server "github": issues, pull requests, repositories. Tools: listTools("mcp__github"). ``` - card from configuration only (name, configured description, static sign-in hint); tools stay deferred, never named in the prompt; - tool keys are `tools.mcp__<server>__<tool>`, exactly as `listTools` prints them; the preamble says names are exactly as listed; - `listTools("mcp__radius")` first line: `not connected: needs sign-in (MCP panel)` | `connected, no tools` | `connected, 47 tools`, then the lines; never an empty result; - `searchTools` returns matching disconnected servers in `disconnected`; - `describeTool` on a disconnected server's tool returns the not-connected line, not "unknown tool"; - coding agent: `mcp_servers` folds into these cards for code-mode servers; `tool_search` for sessions without code mode stays. ## What seven models said Same brief (current design + this proposal or its first draft) to each, answering as the model that uses it. Raw answers: `/tmp/tools-design-*.md` (first four), `/tmp/tools-review-radius-*.md` (Radius models), on slayer. | | GPT-6 Astra | GPT-6.1 Sol | Gemini Pro | Opus 5.5 | Kimi K3 | GLM 5.3 | DeepSeek V4.1 Flash | | --- | --- | --- | --- | --- | --- | --- | --- | | silent truncation is the worst failure | yes | yes | yes | yes | yes | yes | yes | | one line with full types for every listed tool | no: pinned contracts + name/purpose | no: pinned contracts + name/purpose | yes | yes | yes | yes | yes | | discovery returns contracts, not names | yes | yes | yes | yes | yes | yes | yes | | author-declared priority instead of round-robin | yes | yes | yes | yes | yes | yes | yes | | never elide result types | yes | yes | yes | yes | yes | yes | yes | | namespace instructions in the prompt / with every contract | with contracts | with contracts | short, inline | with contracts | on the card | with search | on the card | | errors carry the right signature | — | nearest matches | yes | yes | yes, incl. constraints | yes, incl. constraints | yes | | freeze section per conversation, changes as a note | — | — | — | yes | yes | yes | yes | | disconnected MCP servers visible to search | yes | yes | yes | yes | yes | yes | yes | | TypeScript over JSON Schema | yes | yes | yes (3–4× fewer tokens) | yes (3–5×) | yes (3–5×) | yes | yes | Points taken into the proposal from the Radius round: results never elided (all three), instructions on the card (Kimi, DeepSeek), short field docs inline for units and bases (Kimi, GLM), constraint violations carry the fix (Kimi, GLM), plain-object `searchTools` result with `disconnected` (all three), `listTools` replaces `describeNamespace` (GLM), explicit MCP tool keys (Kimi), three MCP states (DeepSeek). Sol and Astra's objection to one-liners: a shortened line looks like a complete contract while hiding constraints. The other five answer it with an asymmetry: a hidden constraint fails loudly and one retry fixes it (if the error carries the constraint); a missing signature fails silently or costs a round trip. ## Open - **Freeze the section per conversation** and announce reloads as `[tools changed] added: …; removed: …` (five of seven for; GLM: at the start of the next user message). Today a plugin reload changes the section. - **Budget**: keep 3,000 fixed (GLM) or pick it per model at conversation start (DeepSeek, Kimi: 3,000 below 100k context, 6,000 above). - **`summary`**: optional with first-sentence default (GLM), or required for namespaces over ~4 tools and linted to differ from siblings (DeepSeek). - **Coding agent**: same change, `mcp_servers` folded into cards. - apps/p2/docs/discord.md (12164 bytes)
# Pi on Discord `plugins/discord` lets people in a Discord channel wake a p2 session by mentioning a bot. The session reads what was said since its last turn, works in its sandbox, and its answer goes back to the channel as a reply. ## Bots (Settings › Discord) Bots are global: the owner sets them up once, then links sessions to channels from the sessions themselves. 1. At https://discord.com/developers/applications create an application, add a **Bot**, turn on **Message Content Intent** (Bot → Privileged Gateway Intents), and copy the bot token. 2. Settings › Discord lists the bots with their status. **Add bot**: paste the token, **Connect**. Discord's refusal (a wrong token, the intent off) shows there and nothing is kept; once Discord accepts it, the bot's sheet opens. 3. A bot's sheet: who it is and whether it is connected, the servers it is in (live: inviting it somewhere adds the server), the **invite link** (copy or open; it asks for view channels, send messages and in threads, embed links, attach files, read history), its linked sessions (read-only; tap one to open it), **Replace token** (a new token of the same application, kept once Discord accepts it) and **Remove bot** (its sessions are unlinked and unlocked). Several bots may run. Tokens live in `<data>/discord.json` (`{ "bots": { "<id>": { "token", "addedAt" } } }`, 0600) and are redacted from logs. The first release's `{ "token": … }` becomes bot `bot1` at the next start, and its links name it. A bot that cannot reach Discord tries again every minute; a refused token or intent waits for a new token. ## Linking a session (Session sheet → Discord) The Session sheet's **Discord** row shows the session's channel, or "Not linked". Its sheet: - Not linked: pick the **bot** (connected ones), the **server** (the bot's), the **channel** (that server's text channels; one linked to another session shows as taken), **who may wake Pi** (anyone in the channel, or members with picked roles, plus optional user IDs), the **sandbox** (existing ones), and **what the session may use** (MCP servers of `mcp.json`, the project's MCP servers, the global instructions, forwarding ports, messaging other sessions). **Link** stays disabled and says what is still to pick. Linking moves the session into the sandbox (as the owner's Relocate does, into the sandbox's first folder, else `/root`; a busy session refuses, nothing changes), stores its lock and the link, and starts listening after the channel's newest message. - Linked: the bot, server › channel and sandbox (fixed), then who may wake Pi and what the session may use, editable: **Save** (enabled once something changed and someone may still wake Pi) stores them in place (`Discord.updateLink`, contract version 3; version 2 pages still bind) and they apply at once: who may wake Pi is read at each message, the lock is mirrored into the running session. **Unlink** stops listening and lifts the lock; the session stays in its sandbox. - **The agent knows:** a linked session's main agent and its threads' subagents get a two-sentence `discord` prompt section: their input comes from the channel (or thread), their final answer of each turn is posted there as the reply with the files offered with `send_file` during the turn attached, and nothing else is posted (intermediate text, thinking, tool output), so the final answer must stand alone. Its text changes only with the link. The worker keeps the link in the session document `p2.discord-link` (written at each delivery, cleared at unlink through `DiscordIngress.unlinked`); other subagents do not get the section. A session has at most one link and a channel at most one session (its threads follow it). ## How a channel talks to its session - **Waking:** a message that mentions the link's bot, by someone allowed, in a linked channel; other messages only accumulate. Other bots count as people (in batches, and waking Pi by the same rules); the link's own bot never counts. Loop guard: at most 3 turns in a row started by bots per channel or thread; the next bot mention gets one short notice ("Not answering more bots here until a person writes"), later ones nothing (logged), until a person writes there (any message). The count is in memory. - **Batch:** every message after the link's cursor up to the mention (paged through Discord's history 100 at a time; a thread's first batch starts at the thread's start), oldest first, one `Name (<@id>): text` paragraph each (bots `Name (bot, <@id>)`, so the agent can mention them as `<@id>`; replies ping users, never roles or @everyone), the mention written `@Pi`, under a header naming the channel and the count. At most the newest 200 go into the input; the header says how many earlier ones were left out. It is one input of the session's main conversation by a guest author `Name · Discord` (entry kind `discord.batch`), steering the turn under way when the session is busy (the batch joins it). A turn that took several inputs has one answer (Durable settles them all with it): it is posted once, to the newest mention, with the files of the whole turn; the other pending replies go without a post. The cursor moves to the mention. Deliveries run one at a time per channel and per thread (a queue each): a second mention right after the first waits for it and starts at the cursor it left, so no message is in two batches and a new thread forks one subagent. - **Catch-up:** when a bot connects (p2 starting, a new token, a new gateway session after an outage), it reads each of its links and their known threads after the cursor. The newest mention there by someone allowed to wake Pi is the trigger: one batch per channel or thread with every message from the cursor up to it (earlier missed mentions are just messages of the batch), and the reply goes to that newest mention. Its events and live messages are handled one at a time, and a mention at or before the cursor never delivers twice. Threads created while p2 was down are not searched. - **Bots:** only the link's own bot listens; another bot mentioned in the same channel is ignored. A reply waits while its bot is not connected. - **Attachments:** each one up to 25 MiB (20 per batch) is downloaded by the worker into the session's uploads (source `discord`, key `discord:<message>:<n>`, Files sheet) and attached to the input as the composer's uploads are (paths listed, images inline). Larger ones are listed as links. - **Reply:** when the input's turn ends (checked on every status change of the session and every 16 s), the answer's text is posted as a reply to the waking message, split at 2000 characters. A turn without an answer posts why. While a reply is pending (queued or not) the channel shows "Pi is typing": sent the moment a mention is recognized, before the wait for an earlier delivery and the history's pages, refreshed every 8 s (Discord shows it about 10 s) through the delivery and while the reply is pending, and over once the reply is posted, the turn stopped or failed, or the delivery failed. Commands never type. A turn not done after 6 hours is given up. - **Files:** what the agent offers with `send_file` during a Discord-started turn (the main agent's, or a thread subagent's; also from scripts) is attached to the reply: `send_file` downloads carry their conversation as `ref`, and the worker's `turn` lists those made between the input and the answer. Read through the session's Files (`DiscordIngress.read`) and uploaded with the reply (multipart): at most 10 a message, together within the server's per-message limit (10 MiB; 50 MiB and 100 MiB on boosted servers). Larger files are named in the reply only. A turn with files and no text posts the files alone. - **Commands:** a mention of the bot by someone allowed to wake Pi whose remaining text is exactly `stop`, `status` or `compact` (any case, an optional leading slash: `@Pi /stop`) is a command, run at once (not behind a delivery under way) on the main conversation, or in a thread on its subagent. `stop` is the web's Stop: queued inputs are withdrawn, the run is aborted; it also aborts running compactions (tasks of their own, which Stop leaves running: the web's Escape path), read from Durable since a compaction just started may not be in the view yet. It answers "Stopped." (with how many queued messages were withdrawn), "Stopped compaction.", both, or "Nothing is running.", and the stopped turns post nothing. `status` answers the model, whether it works (and compacts), the queued inputs, the context's use, and the cost so far (the cost sheet's data: the target conversation's responses) with the last turn's (the responses since its newest input); `compact` starts a compaction and says so. Commands never go into a batch and are not caught up on after downtime (a missed `stop` is stale). They are left out of batches rather than moving the cursor past them, which would drop the undelivered messages before them. A mention of the bot's own role counts as a mention of the bot. - **Threads:** a thread of a linked channel gets its own subagent at its first mention: `discord-<thread name>`, forked from the main conversation at its last finished answer before the thread was created (a fresh conversation with the main agent's configuration when there is none). It lists in the sidebar like other subagents, follows the session's location and has no reporter: its batches go to it, its answers back to the thread. Its first batch holds what it never saw: the channel's messages after the channel's cursor up to the thread's creation (under "[#channel, before the thread]"), then the thread's messages up to the mention ("[The thread]"); the channel's cursor stays, so the main agent still gets those channel messages. After that the thread has its own cursor. - **Refusals:** a session that is gone, archived or no longer in a sandbox gets nothing; the channel gets a notice. ## The session lock Linking locks the session (`SessionLock`, kept by the sessions plugin as `p2.session-locks`, mirrored into the session's Durable state as `p2.session-lock`). Everyone who may wake Pi can use everything the lock allows, so it lists exactly what the owner picked: - **MCP servers:** only the picked servers of `mcp.json`; the project's `.pi/mcp.json` servers only when picked. - **Instructions:** the global AGENTS.md, skills and prompt templates only when picked (the project's own always). - **Capabilities:** `ports` (forwarding) and `conversations` (messaging other sessions) only when picked. Other tools pass: what they reach is already confined (location tools, bash and `reload_plugins` by the sandbox). The lock holds for everyone, the owner too (p2 has no per-turn authority). Unlinking lifts it. ## State - `<data>/discord.json`: the bots' tokens (above). - The bots' avatars reach the sheets through `GET /discord/avatar?bot=<id>` (owner only, fetched from Discord's CDN by the server and kept per avatar): the page's CSP loads only its own images. - Server state `p2.discord-bindings` (persisted; version 2 adds the bot): links by channel (bot, session, who may wake Pi, cursor, threads with their subagent conversation and cursor) and pending replies by waking message (bot, session, reply channel, submission). - The bots (discord.js, a server host package: plugins cannot bundle from the app's hoisted `node_modules`) run in the server's host-lifetime `DiscordStore` (`fleet.ts`, one `Bot` each, `bot.ts`); the relay logic (`relay.ts`) is plain code over bot and session ports, tested with fakes (`test/p2/discord.test.ts`). The sheets: `BotsSheet.tsx`, `LinkSheet.tsx` (`test/p2/ui-discord.test.ts`). - The worker side is `DiscordIngress` (server only): `deliver` (downloads, the input), `turn` (the submission's state and answer), `thread` (the fork). ## Not done - The transcript shows a batch as one guest message; there is no folded view of the channel's messages. - No `@Pi stop`, no per-link secrets (secrets are per sandbox), no DMs. - apps/p2/docs/durable-request.md (3753 bytes)
# Request to Durable: share the document transaction core **Superseded:** p2 builds its own document store in Chord with a storage interface and adapts Durable's `Storage` in p2 (`target-design.md` §9). Only moving `OutputBuffer`/`Progress` to Chord is still under consideration. From the Chord/p2 redesign (`target-design.md`). To be implemented on pi `main` (`packages/durable`, plus `packages/chord` where noted), then merged into `private`. ## Why p2 moves object state into transactional documents: drafts that record Chord ops, version plus `migrate` on read, `checkpointWhen`, create and retire, one atomic storage batch per commit, adopt and publish afterwards. Durable's `Transaction` already does exactly this and is battle-tested. p2 must reuse it instead of writing a second implementation. Chord itself must stay free of a Durable dependency; p2 already depends on Durable and builds its object store on top of it. ## What we need 1. **A document-only transaction core**, exported from Durable (for example `@earendil-works/pi-durable/documents`): - acquire a draft for a document address (`kind`, `scope`, optional `key`), creating it on first write (`initial()`), loading and migrating it through the host otherwise; - retire a document; - prepare every draft at settlement (revoking the drafts), plan the storage writes (`document.create` / `document.change` / `document.retire`, base or delta per `checkpointWhen`), discard on failure or abort; - adopt after the storage commit and return the publications (record, version, ops) so the caller can notify; - the existing rules: sealing (`Transaction has settled`), read-after-write, version mismatch handling (a token whose version differs from the cached value reloads; a stored version newer than the token is refused). Durable's `Transaction` then builds on this core and adds tasks, conversations, submissions and forks, so the core is covered by Durable's existing tests. The seam already exists: `TransactionHost` (`session/transaction.ts:99`); `conversationCreated` is its only Durable-specific member. 2. **Definitions usable outside a conversation:** `defineDoc` with `scope: "session"` already exists; the core needs to accept a session-scoped address with an arbitrary `kind` and `key` chosen by p2 (p2 maps an object's identity hash, the state name and the key into them). Confirm whether `kind` must be globally registered or may be dynamic. 3. **Move `OutputBuffer` and `Progress`** (`harness/output.ts`) to Chord (for example `@earendil-works/chord/progress`); Durable imports them from there and **re-exports** them under their current names, so nothing breaks. 4. **Review of p2's use of `Storage`:** p2's store calls the core and `Storage` directly (memory and SQLite backends), without a `Session`. It must uphold what `Session` guarantees to `Storage`: serialized commits (p2 serializes per host), create before change, no change after retire, ids from `mintId`, deltas against the stored version. We would like the Durable owners to confirm the list is complete. ## What p2 does with it (for context) - One store per host. Documents are owned by one object identity; the mutation line is per object; storage commits are serialized per host. In-memory documents use the memory backend semantics without persistence; `persist: true` documents go to the host's SQLite storage. - Commits notify local subscribers once and send one atomic `state` batch per hold on the wire. - No eviction: a document's memory copy lives as long as its object; persisted documents stay on disk until retired. ## Out of scope Tasks, conversations, submissions, forks, the harness and the scheduler stay as they are. - apps/p2/docs/glyph-editor-port.md (5009 bytes)
# Bringing Max's glyph editor back Max's glyph editor (gist `dgtlntv/0162ce4124d9286926b03a72d2cff052`) was a custom view: `glyph-editor.html` in a sandboxed frame, its data in the view's state, and `glyph-editor-bridge.js` as the agent's code-mode helpers on top of the `view` tool (`read`, `update` with `state`, `merge` and `html`). p2 removed custom views, view state, the view library and rooms on 2026-11-12 (docs/views.md, docs/artifacts.md, docs/agent-plugins.md). There is no migration: an open "glyph" view is gone after the update. This page says what to do before updating, and how to run the editor again. ## Before updating p2 1. Save the unsaved glyphs: in code mode, `const B = eval(await tools.read({ path: '/root/work/glyph-editor/bridge.js' }))`, then `await B.save()`. The view's `edits` (glyphs changed in the editor and not yet written to the UFO) live only in the view's state and are lost with it. 2. Keep the page: `/root/work/glyph-editor/editor.html` is the source; if it only exists in the view, copy it out first (`view read` does not return HTML: take the newest version from the gist or the work directory). ## Now: the editor as an html artifact (same page, new helpers) The page keeps working unchanged: html artifacts keep the page API `window.p2view` (`onState(fn)` with `fn(state, from)`, `from` being `init` or `update`; `state`; `readOnly`; `setState(next | fn)`, saved after a short pause; `select(summary, data)`; `clearSelection()`; `onDescribeRegion(fn)`), with the state stored with the artifact (every version of the page reads it). Only the agent-side helpers change, from the `view` tool to the `artifact_*` code-mode tools (namespace `artifact`; `describeNamespace("artifact")` gives the whole page API): | bridge.js today | with artifacts | | --- | --- | | `tools.view({ action: "open", kind: "custom", id: "glyph", html })` | `const { id } = await tools.artifact_write({ kind: "html", title: "Glyphs", content: html })` (or give `id: "glyph"`), then `tools.artifact_open({ id })` (a pane beside the chat) or `tools.artifact_show({ id })` (in the transcript) | | `tools.view({ action: "read", id })`, parsing `State:` out of text, cut at 40,000 characters (hence `edits` before `glyphs`) | `const { state } = await tools.artifact_html_state({ id })`: the whole state as an object, no text parsing, no cut | | `tools.view({ action: "update", id, state })` | `await tools.artifact_html_set_state({ id, state })` (the open page gets it at once, `from` = `update`) | | `tools.view({ action: "update", id, merge })` | read the state, change the keys, `artifact_html_set_state` (a read-modify-write: a glyph edited in the page in between is overwritten, as with `merge` before) | | `tools.view({ action: "update", id, html })` (`reloadPage`) | `await tools.artifact_write({ id, kind: "html", title, content })`, or `artifact_edit({ id, edits: [{ old, new }] })` for small changes: a new version, the open page reloads | | "Script errors its page logged" in `view read` | `await tools.artifact_html_errors({ id })` (and `artifact_html_logs` for console lines); new errors also arrive with the person's next message | | the selection in `view read` | the selection arrives with the next message (`<views>` block), as before; `view_list` shows it too | `readBig` stays only for reading `export.json` from disk (the `read` tool's 50 KB limit); the state no longer needs it. `artifact_html_eval({ id, code })` runs code in the open page when a helper needs the page itself. What stays as it was: `ge.py` (export, save, diff), the UFO sources, the `edits` overlay over the base `glyphs`, and the state's size limit (256 KiB of JSON: one master must fit, as `ge.py`'s docstring says). The artifact belongs to the session and travels with its export (and its clones and forks); it is listed in the session's artifacts sheet, so it can be reopened after closing its pane (nothing is lost on close). People with read-only links open it there too, on their own device and without saving changes. ## Later: the editor as a plugin The proper version is a `glyphs` plugin (skill `writing-p2-plugins`): glyph documents in the server store keyed by the session and the master, typed methods (`setGlyph`, `note`, `open`, `save`) the agent calls as code-mode tools instead of read-modify-write on one JSON blob, the base master read from the UFO files instead of being copied into state (no size limit), a conditional save so a glyph changed during a save stays unsaved, and several people editing at once. Its glyph list can show in the artifacts sheet through `ArtifactKinds`. `apps/p2/docs/rooms-v3/book.md`'s glyph chapter lists what it needs; most of it is now ordinary plugin code. ## The gist's other pages - `space-invaders.html`: works as an html artifact as it is (the high score in the artifact's state). - `pong.html`: used `p2view.live`, which is gone; real-time multiplayer is a plugin now (a server object running the ball, paddles as events). - apps/p2/docs/handoff.md (15153 bytes)
# p2 handoff For whoever continues p2, human or agent, starting cold. It assumes you know pi and this repository, but not the history of p2. Everything you need is in the repository; nothing depends on earlier conversations. ## What p2 is p2 (`apps/p2`) is Pi Mobile rebuilt on **Chord**, an object runtime (Chord v2: `@earendil-works/chord`, `packages/chord`; p2 runs on Chord v2 only, see `docs/p2-notes.md` **Chord v2**; Chord v1 is deleted). Every feature is a plugin (`apps/p2/plugins/*`) with facets per process: `server.ts`, `worker.ts` (one worker per session, running Durable), `web.tsx` and `tui.ts`. There is no manifest and no dependency list: p2 discovers plugins by listing their facet files and loads them alphabetically. Chord resolves dependencies by contract name, so their order does not matter; a missing provider fails loudly, naming the consumer. Compatible backend reloads preserve unchanged state and resources; a successful reload publishes rebuilt web and TUI bundles. p2 replaced Pi Mobile (formerly `apps/mobile`, removed; see git history). Its web UI matches Pi Mobile's (same DOM, classes, styles); the data layer is Chord objects instead of Pi Mobile's hand-written protocol. On top of Pi Mobile, p2 has hot reload with explicit reload/restart controls, logs and owner notices, chat mode with replies and `@pi` mentions, owner sign-in links, an isolated TUI with pi's look and keys, and security hardening (CSP, restricted guest web assets, WebSocket limits). p2 lives in `earendil-works/pi` at `apps/p2` and keeps the name p2. ## State at handoff | Where | What | Port | Data | | --- | --- | --- | --- | | Phone (Termux, Node 26) | p2, the daily driver; Pi Mobile's sessions were imported when it replaced Pi Mobile | 8787 | `~/.pi/p2` | | slayer (`slayer.marioslab.io`) | development and tests (`~/pi`, branch `main`); no p2 server runs there | | | Done: everything Pi Mobile did (checked by a four-part audit; what differs on purpose is under Decisions in `docs/p2-notes.md`), the switch of the phone, the TUI port phases P0–P7 and P9–P11, and Chord in `packages/chord` replacing the old facet/service runtime and the `pi-client`, `pi-protocol` and `pi-server` packages (pi `6db4070f1`). Open items are under [Open work](#open-work). ## The phone - **Checkout:** `~/pi`, branch `main` (Node 26.3.1). Old data: `~/.pi/mobile` (keep it: imported history refers to attachments there) and `~/.pi/p2.trial-oct10` (a leftover, deletable). The TUI's Node check (26.5.1 or newer) may be lowered to 26.3.1 locally in `src/hosts/tui-sandbox.ts`, uncommitted; leave it alone when updating. - **p2** runs as the runit service `$PREFIX/var/service/p2` (termux-services; `sv status|restart $PREFIX/var/service/p2`; logs in `$PREFIX/var/log/p2/current`). Its `run` script takes the wake lock, sets `PORT=8787`, `P2_DATA_DIR=~/.pi/p2`, `P2_ALLOWED_HOSTS=pim.ngrok.pro`, `P2_PUBLIC_URL=https://pim.ngrok.pro`, and runs the dev supervisor from `~/pi/apps/p2` (`node --import ./src/source.ts scripts/dev.mjs`). The service `p2-orphan-reaper` runs beside it. - **Deploy** (only pushed commits): in `~/pi`, `git fetch origin && git merge --ff-only origin/main`, `npm install --ignore-scripts` at the root if a `package*.json` changed, `cd apps/p2 && npm run check`, then `npm run reload -- --restart` (plugin-only changes: **Reload**). The restart replaces the server process; the supervisor stays. - **Per-host stores:** a p2 with per-host stores refuses a data directory in the older layout. Once: stop p2, `npm run migrate-data -- --dry-run`, then `npm run migrate-data` (old files go to `~/.pi/p2/pre-files-backup/`; `scripts/data-report.ts` before and `--compare` after), then start it (`docs/p2-notes.md`, Migration). - **Tunnel:** ngrok as the runit service `$PREFIX/var/service/ngrok`, inside the proot Ubuntu (`proot-distro login ubuntu`), forwarding `https://pim.ngrok.pro` to `127.0.0.1:8787` with the traffic policy `~/policy.yaml` (`sv status|restart $PREFIX/var/service/ngrok`; logs in `$PREFIX/var/log/ngrok/current`). - **Radius:** not configured (no `radius.json`); `npm run radius:login` enables it. - **No boot scripts.** The p2 service takes `termux-wake-lock` so Android keeps the processes alive. ## Running p2 ```sh cd apps/p2 npm run dev # dev mode: apply plugin changes, including web facets, with Reload or `npm run reload` npm start # without the dev supervisor P2_FAKE=1 npm start # fake model, no provider needed npm run owner-link [url] # one-time owner sign-in link for another device (run on the machine itself) npm run tui [url] # TUI (Node >= 26.5.1); on loopback it signs in with the data directory's owner key npm run reload [plugins] [--restart] npm run check # Biome + tsc npm test # full suite; TUI tests need Node 26, browser tests need CHROMIUM_BIN ``` Environment variables: `docs/p2-notes.md` (Running). Configuration in the data dir: `models.json`, `settings.json` (incl. `codemode`, default on in p2), `envs.json`, `projects.json` (imported once into the server store), `mcp.json`, `auth.json`, `radius.json`, `forwards.json`, `identity.json` (the server's name, only while the owner set one). **Sign-in.** Loopback alone is nobody (shared machines). A browser on the host signs in once per profile with the token the server prints at startup (pasted into the sign-in page); the TUI and scripts sign in with the data directory's owner key (README, Signing in on this machine). Other devices get owner rights with a link from `npm run owner-link` (single use, 10 minutes, long-lived signed cookie). Guests join through share links. Requests other websites make never carry owner rights. Sign out per device or all devices in the Devices sheet. **Dev mode.** A file watcher only marks plugins dirty; reloads are explicit (button, `npm run reload`, the `reload_plugins` tool) because a watcher can't tell when an agent has finished a multi-file edit. Runtime code (`src/hosts`, `src/runtime`, workspace packages such as `packages/chord`) needs a restart; the "Server code changed" pill appears only when a module the server actually loaded changed. **slayer.** Development happens in `~/pi` (branch `main`); scratch worktrees for verifying an agent's files or running browser tests go under `/tmp` (`git worktree add /tmp/<name>`), never into `~/pi`. No p2 server runs there (the old public instance at p2.mariozechner.at is gone). Node 24: `~/node-v24.4.1-linux-x64/bin`; Node 26 (TUI tests): `~/.local/bin/node26`. Browser tests: `CHROMIUM_BIN=/home/badlogic/.local/bin/chromium-docker`. ## Architecture in five minutes - **Chord v2** (`packages/chord`, `docs/design.md` there): objects per (type, params, principal) placed on host kinds (`Server`, `Session({ id })`), JSON Schema contracts, held references, documents (`defineState`) and derives, natives (`self.open`), keyed registries, binary links over WebSocket and IPC with routes (browser → server → worker), the workers component (`createWorkers`) and the plugins component (`createPlugins`: build, load, rollout). How p2 uses it: `docs/p2-notes.md` **Chord v2**. - **Hosts** (`src/hosts`): `server.ts` (HTTP, auth, WebSocket admission), `web.ts` (web serving), `bundles.ts` (content-named web and TUI esbuild bundles), `worker.ts` (session worker), `tui*.ts` (launcher, sandbox, filter, grants), `dev*.ts` (dev supervisor). - **Runtime** (`src/runtime`): shared contracts and helpers (principals, worker and Harness, Durable registry, extension switches, input entries, settings, logs, fetch). - **Plugins**: chat, chat-mode, sessions, familiar (the session above the others), subagents, agent (one-shot agents), codemode, cache-warming, coding-tools, fit-images, prompt, models, login, search, status, drafts, files, send-file, artifacts, workspace, file-index, terminals, envs, mcp, ports, radius, sharing, devices, settings, proxy-marker, logs, dev-tools, powershell, web-shell, projects, views, rollups (sidebar sections made from code). Before writing one, read Chord's tutorial (`packages/chord/docs/tutorial/`) and plugin guide (`packages/chord/docs/plugin-authoring.md`), then an existing p2 plugin of the same shape. - **Locations:** every conversation works at `{ env, cwd }`: the main one at the session's, subagents spawned with a cwd at their own (on the caller's environment), the others following the session. A session's location changes in place: relocating an idle running worker fences it, updates the catalog and moves the main conversation and its followers without a restart; subagents with their own location stay. Plugins resolve a conversation's location and environment through `Harness.environments`, the session's through `WorkerProcess.location` (`docs/p2-notes.md`, Agent locations and Relocation). - **Durable** (`packages/durable`) runs the agent: conversations, inputs as custom entries (`p2.input`, `chat.mention`), display-only writes placed immediately, code mode (QuickJS sandbox; scripts call tools as `tools.<name>`, one-shot agents via `tools.agent`, tools grouped by the namespaces their extensions declare), compaction with visible failures, deferred tools and `tool_search` (MCP tools are for code mode scripts, as in pi; `tool_search` loads them only while code mode is off). ## The TUI `npm run tui [url]` (Node ≥ 26.5.1). The launcher owns the terminal, the connection and sign-in; all TUI code (pi-tui and plugin facets) comes from the server as one bundle and runs in a worker that drops every Node permission first. Output passes a strict escape-sequence allowlist; privileged actions (copy, open URL, download, external editor, attach, saving preferences, sign-out) need launcher grants, mostly with a launcher-drawn consent screen. Preferences (keybindings, theme, display settings) are the visitor's own (`~/.pi/agent`, `~/.pi/p2-tui/prefs.json`). Threat model: `DEV.md` and `README.md` (TUI isolation). Design and as-built notes: `docs/p2-tui-port.md`. ## Rules From `AGENTS.md` and this project's history: - **One tree.** p2, the packages, root config and scripts change together in `earendil-works/pi`. After p2 changes run `npm run check` both at the root and in `apps/p2`. - **Commit only verified states.** When several agents edit one tree, copy an agent's files into a separate worktree on top of `HEAD`, run `npm run check` and the full `npm test` there, and commit exactly that content (`git hash-object -w` and `git update-index`). A browser test that times out under load and passes on its own is acceptable; anything else is not. Lockfile commits need `PI_ALLOW_LOCKFILE_CHANGE=1`; npm 11 strips `libc` fields, revert that noise. Never `--no-verify`. - **Deploy only pushed commits** (The phone, Deploy). - **Port from the diff.** Changes copied from elsewhere start from the commit (`git show <hash>`), never from a summary; briefs to agents name the commit and the files. - **No hand tests on real sessions.** Use throwaway sessions under `/tmp`, or a separate server on another port with a temporary data dir. Never print cookies, tokens or share links. - **Check `pwd`** before working in a subagent, and spawn subagents with an explicit `cwd`. - **`pkill -f` matches its own command line.** Kill by PID or by `/proc/<pid>/cwd` instead (it once killed the live p2 server). - TypeScript: no `any`, erasable syntax only, top-level imports only (except the plugin loader's dynamic import). Use Opus-class models for documentation and tutorials. ## Open work 1. **The phone's daemon binary:** p2's file index uses the pi-env daemon's `listFiles` when the packaged daemon has it, else the old git/walk index. slayer has a fresh build in `packages/env/bin/pi-env-linux-x64` (static musl; built with `RUSTFLAGS="-C linker=rust-lld -C link-self-contained=yes" cargo build --release --locked --target x86_64-unknown-linux-musl` in `packages/env/daemon`). On the phone: `cargo build --release --locked` there and copy it to `packages/env/bin/pi-env-android-arm64/pi-env`. The next pi-env release ships all targets through CI. 2. **TUI:** launcher images, a filtered terminal relay, a "trust like SSH" mode, web tool cards (`docs/p2-tui-port.md` §10). Fork from subagent conversations; fork/clone, shell and templates have no web UI yet beyond rendering. 3. **Cleanup leftovers:** two HTTP helpers were not merged because their semantics differ (byte ranges: files serves `bytes=-` whole, workspace answers 416; body readers: sharing answers 400 past the limit, devices 413); decide one behavior each. 4. **MCP:** `plugins/mcp` (WP6-8 of `docs/mcp-durable.md`: owner `mcp.json`, host stdio and HTTP, code mode tools (`tool_search` without code mode), servers on environments, project `.pi/mcp.json`, OAuth sign-in); code mode namespaces are done (one composing `mcp` extension whose wraps show each conversation the servers of its location's lease, each namespace `mcp__<server>` with pi's one-line summary; `mcp_servers` only without code mode; design and the Durable alternative: Radius artifact `01m7czh2mtetn9n716pzqnd7mk`); owner and account qualifiers for same-named servers of several people are p3's; whether Radius sends its cookie on a sign-in's redirect back is unverified. 5. **npm:** deprecate `@earendil-works/pi-client`, `pi-protocol` and `pi-server` (1.1.0), which no longer exist. 6. `docs/p2-notes.md` TODO: the TypeScript 5.9.3 pin (Termux), an isolating plugin loader, sandboxing third-party facets. 7. **Slow test suite:** about 11 minutes serially; files could run in parallel once their isolation is fixed. The chat-mode long-press reply browser test times out under load. 8. **After the Chord v2 port:** the TUI on Chord's sandbox (C2); no per-link call limit any more (owner decision; Chord v2 has none). Chord v1 and the old-page shim are deleted (C1). ## Documents | File | Content | | --- | --- | | `README.md`, `DEV.md` | running, deployment behind a proxy, TUI sign-in and isolation | | `docs/p2-notes.md` | decisions, configuration, plugin-by-plugin notes | | `docs/p2-tui-port.md` | TUI design, as-built notes, open phases | | `docs/optimistic-ui.md` | optimistic UI: the pattern for actions, pending states and loads (web and terminal) | | `docs/tool-ui.md` | how a plugin shows its tool (web and TUI registrations), with a complete example plugin | | `docs/workflows.md` | code mode workflows with one-shot agents | | `docs/tree.md` | `/tree`: branches of the main conversation as Durable forks | | `docs/views.md` | views beside the transcript (Markdown, changes, artifacts, plugin kinds), layouts, annotations | | `docs/artifacts.md` | html and Markdown artifacts: tools, pages and `window.p2view`, inline cards, the side panel | | `docs/glyph-editor-port.md` | moving Max's glyph editor from a custom view to an html artifact | | - apps/p2/docs/history-tool.md (8728 bytes)
# History tools for p2: spec Status: **approved by the owner** (built: plugins/history, plugins/search history methods; test/p2/history-tools.test.ts). Lets an agent find and re-read its own conversation's earlier entries after a compaction or a handoff, and hand off deliberately. Based on the evaluation of https://github.com/earendil-works/pi-history (commit `c26efb1`): its interface is borrowed, not its code (it is a coding-agent extension over `sessionManager.getBranch()`, with a `slice(-0)` bug and synchronous unbounded regex). ## 1. Why it works in p2 Durable keeps every entry. A compaction only appends `pi.compaction` (a summary plus `head`, the first retained entry); a reset or handoff appends `pi.reset` (the handoff text as a user message, nothing retained). The model then sees the system prompt, the summary or handoff, the retained entries (none after a handoff) and new entries. Older entries leave the model's context, not the store. p2 already indexes root and subagent user and assistant text in the server's FTS5 index (`plugins/search`, `search.sqlite`), across compactions and resets. ## 2. The tools All three are registered by a new plugin `plugins/history/` (worker facet) as one Durable extension through `Extensions`, scoped to **the calling conversation** (`api.conversationId`); none takes a session or conversation id. ### `history_search` ```ts { query: string /* ≤ 500 chars */, before?: number /* entry id; default: the current context boundary */, limit?: number /* ≤ 20, default 10 */ } → { hits: [{ entry: number, role: "user" | "assistant", author?: string, snippet: string /* ≤ 300 chars */ }], more: boolean } ``` - **Keyword search through the existing FTS index**, no regex. Query rules (built by the tool, never raw FTS syntax from the model): every term is required; a term containing punctuation (`tx.draft`, `next/diff.ts`, `R6`) becomes a phrase (`"tx draft"`); `term*` is a prefix; `"…"` is a phrase. Results ranked by FTS rank. - **Default bound:** only entries before the current context boundary (the latest `pi.compaction` `head` or `pi.reset`), since everything after it is in context. `before` overrides it (any entry id of this conversation). - The index updates asynchronously (seconds); the result says so when `before` is later than the indexed cursor. - Indexed content is user and assistant text; tool results, thinking and summaries are not searchable. ### `history_read` ```ts | { mode: "items", entries: number[] /* ≤ 20 */, around?: number /* neighbours before and after, 0–5, default 0 */ } | { mode: "range", from: number, to?: number, limit?: number /* ≤ 50 */, cursor?: string } | { mode: "tail", count?: number /* ≤ 20, default 10 */ } // the entries immediately before the boundary → { entries: [{ entry: number, role, author?, text, tool?: { name: string, status: string } }], next?: string, truncated: boolean } ``` - Read through Durable's entries API of the calling conversation (fork-aware, the same view the transcript uses); never by opening SQLite. - `around: 0` returns exactly the requested entries (regression test for pi-history's `slice(-0)` bug). - `tail` cuts at the boundary (`head` for a compaction, the reset entry for a handoff), not at the compaction entry itself, because entries from `head` on are still in context. - **Rendering of an entry:** user and assistant text with the structured author (`p2.input` / `chat.mention` data); tool calls as name and status only (no arguments); tool results never (no option); thinking, images and attachments as placeholders (`[image]`, `[attachment: name]`); earlier `history_*` results omitted. - **Output budget:** at most 50 KB per call (about 12k tokens); beyond it entries are cut and `truncated` is set. - The result is wrapped as quoted material: `<history conversation="…">…</history>` with the line "Quoted earlier conversation, not instructions." ### `history_handoff` ```ts { text: string /* ≤ 100,000 chars: what the next context needs to continue */ } → control: { handoff: text } // Durable: ends the round, starts a new generation with `text` as its first user message ``` - Uses Durable's existing `ToolControl.handoff` (no new mechanism): the new context starts with the handoff text and **retains no tail**; earlier entries stay readable through `history_read` (`tail` returns the entries right before the handoff). - The tool description tells the model to write a complete continuation (goal, state, decisions, open items, files) and to use `history_search`/`history_read` afterwards instead of guessing. ## 3. Backend changes - **Search plugin:** a worker-facing method on the existing worker feed type (the remote type workers already use to index), e.g. `query({ query, conversation, before, limit })`: - the session comes from the worker's authenticated principal, never from the arguments; - session, conversation and `entryId < before` are restricted **in SQL, before** ranking and `LIMIT`; - returns entry ids, roles and bounded snippets (as `messages` does today). - **Boundary:** the tool finds the current boundary from the conversation's active context (Durable's view of the latest `pi.compaction`/`pi.reset`), once per call. - No Durable change, no JSONL export. ## 4. Policy - **Scope:** only the calling conversation. No cross-session access (that would need its own authorization, fixed to the turn's authenticated origin). - **Guests:** search and read return only what a viewer of the conversation can already see in its transcript, so they are allowed in guest-initiated turns. `history_handoff` is allowed in every turn (the agent decides; nothing is lost, history stays readable). - **Subagents:** add `history_search` and `history_read` to the subagents' tool selection (`plugins/subagents/tools.ts`) and to one-shot agents (read-only); `history_handoff` for subagents only (a one-shot agent does not hand off). - **Prompt section** (short): "After a compaction or handoff, earlier details are not in your context but still stored. Search with history_search, then read only what you need with history_read (mode tail for what came right before the last boundary). Do not guess from the summary." ## 5. UI On the web each call is a step of its turn's activity (`src/web/activity/`: its label in `model.ts`, its detail in `Detail.tsx`); TUI cards are in the plugin's `tui.ts` (as `send-file`/`artifacts`). | Tool | Web step | TUI card | | --- | --- | --- | | `history_search` | the query; a list of hits (role, author, snippet, entry id); a hit is clickable and reveals the entry in the transcript (the existing `revealEntry`, loading older history pages if needed) | query, hit count; expanded: hit lines with entry ids | | `history_read` | mode and range ("10 entries before the handoff"); collapsed list of the entries read, each clickable to reveal it | mode, count, `truncated` marker; expanded: the entry lines | | `history_handoff` | the handoff text (collapsible, markdown) and the line "New context started; earlier history stays searchable" | the same, collapsed by default | - The `pi.reset` marker that follows a handoff already renders in the transcript; it gets the note "handoff" when its model message came from `history_handoff`. - Cards are read-only and visible to every viewer of the conversation. ## 6. Tests - Query building: punctuation becomes a phrase, prefixes, required terms, escaping (no raw FTS syntax). - SQL bounds: hits from other sessions, other conversations and after `before` never appear, also when they would rank higher (bounds before `LIMIT`). - `history_read`: `items` with `around: 0` (exactly the requested entries), `range` paging, `tail` after a compaction (cut at `head`) and after a handoff (cut at the reset), the output budget, placeholders, omitted history results. - `history_handoff`: the new generation starts with the text and no retained entry; `tail` afterwards returns the entries before it. - An end-to-end worker test: compaction, then search and read find an entry from before it. ## 7. Files and effort | Part | Files | Lines (with tests) | | --- | --- | --- | | `plugins/history/` (worker facet, three tools, prompt section) | 3 | 350–500 | | search plugin: worker query method and SQL bounds | 2 | 100–150 | | web cards (`src/web/Chat.tsx`) | 1 | 100–150 | | TUI cards (`plugins/history/tui.ts`) | 1 | 80–120 | | subagent tool selections | 1 | ~10 | | total | ~8 | ~650–950 | ## 8. Decisions (owner) 1. `history_handoff` is allowed in every turn. 2. `history_read` never returns tool results. 3. The search index stays as it is (no tool results indexed). - apps/p2/docs/mcp-durable.md (46345 bytes)
# MCP for Durable and p2: work plan This file is the brief for agents implementing MCP (Model Context Protocol) support for p2, the Durable-based app in `apps/p2`. Every work package (WP) below is self-contained: it says where the work happens, what to build, how to test it, and when it is done. Read sections 1 to 4 before any work package, then only the package you are given. The human-readable plan with background lives at https://radius.earendil.com/artifact/01m6em2r8gercbx2rn2y8mwwpq (revision 4). Where it and this file differ, this file wins. --- ## 1. Repository and rules Everything is in `earendil-works/pi` (local checkout `~/Development/pi-mono`, branch `main`): the packages under `packages/` (`env`, `mcp`, `durable`, `coding-agent`, `chord`, ...) and p2 at `apps/p2`. Until p2 moved there (commit `05dc8d239`), it lived in the private repository `earendil-works/pi-private`, which is no longer used. Rules (the repository's `AGENTS.md` overrides anything here): - **Packages and p2 change together.** A fix to `packages/env` or `packages/mcp` found during p2 work is part of the same tree; it still gets its own commit and changelog entry. - Several agents may work in the same checkout. Stage explicit paths only; never `git add -A`, `git stash`, `git reset --hard` or `git checkout .`. - After code changes: `npm run check` from the repository root (formatting, lint, types), and after p2 changes also `cd apps/p2 && npm run check` (p2 is outside the root check). It must be clean except for errors you can show are unrelated and pre-existing. If it fails on missing modules or stale generated model data, run `npm install --ignore-scripts` and `npm run hydrate:model-data` first (this does not change tracked files). - Tests: run the package's own tests, never the whole suite. From the package directory: `node "$(git rev-parse --show-toplevel)/node_modules/vitest/dist/cli.js" --run test/<file>.test.ts`. p2's tests use `node --test` (see `apps/p2/package.json`). - The pi-env daemon is Rust: `cd packages/env/daemon && cargo fmt --check && cargo clippy --locked --all-targets -- -D warnings && cargo test --locked`. Also lint for Windows: `cargo clippy --locked --all-targets --target x86_64-pc-windows-msvc -- -D warnings`. Rebuild the debug daemon (`npm run build:daemon` in `packages/env`) before running pi-env's TypeScript tests. - Public packages get a `CHANGELOG.md` entry under `## [Unreleased]` (`### Added`, `### Changed`, `### Fixed`). - Commit only when asked, one commit per work package, message `feat(<scope>): ...` / `fix(<scope>): ...`. - No `any`, no inline imports, erasable TypeScript only, no runtime checks of values TypeScript already guarantees; validate only at trust boundaries (wire input, configuration files, data from another process). --- ## 2. What we are building p2 gets MCP servers. The design, in short: - **One manager on the p2 server owns every MCP connection and process.** Session workers never spawn an MCP server or hold a credential. They talk to the manager through one Chord object per worker principal. - **Placement.** A stdio server runs on the p2 host or on an execution environment from p2's `envs.json` (an SSH machine with the pi-env daemon). On an environment, the manager starts it through the daemon's `process` operation (WP1, done). - **Instances.** Stdio servers run once per session and location by default; `"shared": true` in the owner's configuration makes one process for everybody. Streamable HTTP servers are shared by default. - **Configuration.** The owner's `<P2_DATA_DIR>/mcp.json` (trusted) plus a project's `<cwd>/.pi/mcp.json`, which the worker reads at its location and sends to the manager (untrusted, restricted, see WP7). - **Tools.** Workers turn the servers' tools into Durable tools named `mcp__<server>__<tool>`, deferred (found through `tool_search`) or callable only from code mode. - **OAuth.** Only HTTP servers use MCP OAuth. Sign-in links point at p2 (`/mcp/sign-in/<id>`); when one is opened, p2 picks the redirect from the request's principal: loopback (`http://127.0.0.1:<port>/callback`) for a browser on the p2 host, p2's `/mcp/callback` on the Radius or public origin otherwise, and pasting the final redirect URL always works. ```text worker host (session s1) server host ┌──────────────────────────────────┐ ┌──────────────────────────────────────────────────────┐ │ MCP worker object │ hold as │ p2.mcp: remote view, one per principal │ │ · reads .pi/mcp.json per ├─────────►│ worker variant ──┐ │ │ location via its ExecutionEnv │ worker │ ▼ │ │ · contributes Durable tools │ {s1} │ p2.mcp.manager: local, host lifetime │ └──────────────────────────────────┘ │ instances, leases, OAuth, pi-env connections │ │ owner variant ◄── web panel, TUI │ └───────┬──────────────────┬──────────────────┬────────┘ stdio on the p2 host stdio on env "gpu" Streamable HTTP ``` --- ## 3. Status and order | WP | Where | What | State | Depends on | | --- | --- | --- | --- | --- | | 1 | `packages/env` | daemon `process` operation, `openProcess()` | **done**: `9c9ce8547`, fix `032656b6c` | none | | 2 | `packages/mcp` | transport over a byte pipe | **done**: `c158fa2ee` | none | | 3 | `packages/mcp`, `packages/coding-agent` | OAuth sign-in split into start/complete, moved out of coding-agent | **done**: `bb266bf35`, fix `797e122cb` | none | | 4 | `packages/mcp`, `packages/coding-agent` | shared naming and config helpers (optional) | **done**: `76c040a2a` | 3 | | 5 | p2 | **gate:** p2's Chord v2 port has landed; WP1-3 present | **done**: p2 imported at `05dc8d239`, deps in "feat(p2): MCP servers on the host, on environments and from projects" | 1-3, the port | | 6 | `apps/p2/plugins/mcp` | manager, view, worker object; owner config; host stdio and HTTP | **done**: "feat(p2): MCP servers on the host, on environments and from projects" | 2, 5 | | 7 | `apps/p2/plugins/mcp` | environment placement; project `mcp.json` | **done**: "feat(p2): MCP servers on the host, on environments and from projects" | 1, 6 | | 8 | `apps/p2/plugins/mcp` | OAuth: flows, routes, loopback, pasting; web and TUI | **done**: see section 4.7 | 3, 6 | | 9 | `apps/p2/plugins/mcp`, `plugins/codemode` | code mode namespaces, progress, first-turn admission | open | 6 | Section 4.5 lists where the built APIs of WP2-4 differ from sections 5-7, section 4.6 where WP6-7 differ from sections 9-10, section 4.7 where WP8 differs from section 11. --- ## 4. Facts every package relies on ### 4.1 WP1 (done): the daemon's `process` operation In `packages/env` (commit `9c9ce8547`): - Daemon: `packages/env/daemon/src/process.rs`; protocol in `packages/env/docs/protocol.md` (section on `process`). - Client: `packages/env/src/process.ts`, exported from `@earendil-works/pi-env`: ```ts export function openProcess(connection: Connection, options: ProcessOptions): Promise<RemoteProcess>; export interface ProcessOptions { cwd: string; argv: readonly string[]; // run directly, no shell env?: Record<string, string>; // added to the daemon's environment inheritEnv?: boolean; // false: env is the whole environment onStdout: (bytes: Uint8Array) => void; onStderr?: (bytes: Uint8Array) => void; } export class RemoteProcess { readonly pid: number; readonly exited: Promise<number>; // exit code; rejects (code "aborted") after close(), or on a lost connection write(data: Uint8Array | string): void; // stdin, in order; dropped once stdin is closed or the program ended closeStdin(): void; // after the input written before it close(): void; // kills the process group } ``` - A daemon released before WP1 fails `openProcess` with `RemoteError` code `not_supported` (`hello.features` lacks `"process"`). - A lost connection kills the program (the daemon kills everything it started). - When the program exits on its own, the daemon kills what it left running in its process group (Unix) or its job object (Windows) before reporting the exit (`7367bc823`). Known gap on Windows: a child started before the daemon assigns the program to its job can escape (documented in `packages/env/docs/protocol.md`). - **Windows:** like `exec` argv commands, batch files (`npx.cmd`, `uvx.cmd`) are refused; callers must start them through `cmd.exe` (WP7). ### 4.2 Durable pieces (public, `packages/durable`) - `ToolRegistration` (`packages/durable/src/harness/types.ts`): `name`, `description`, `parameters` (JSON Schema object), `deferred?: true`, `callers?: ("model" | "tools")[]`, `replay?: "safe" | "unsafe"` (default unsafe), `structuredOutputSchema?`, `outputLimits?`, `execute(args, api, context)` returning `ToolExecutionResult` (`output`, `structuredOutput`, `isError`, `details`). `api.conversationId` and `api.env` identify the calling conversation and its execution environment. - `createToolSearchExtension()` from `@earendil-works/pi-durable/tool-search` (`packages/durable/src/tool-search/tool-search.ts`): the `tool_search` tool that loads deferred tools. - `createCodemodeExtension({ sandbox, namespaces })` from `@earendil-works/pi-durable/codemode`; `namespaces` are `{ name, description?, instructions?, prefix }` (`packages/durable/src/codemode/describe.ts`). - A tool call whose tool is not registered right now waits until it is (`packages/durable/src/harness/agent.ts`, `addTools`), so a reconnecting MCP server does not fail a replayed call. ### 4.3 pi-mcp today (public, `packages/mcp`) - `McpClient` (`src/client.ts`), transports in `src/transports/` (`stdio.ts`, `streamable-http.ts`, `in-memory.ts`, the `McpTransport` interface and `TransportEvents` in `transport.ts`), `toLlmContent()` (`src/protocol/content.ts`). - OAuth in `src/oauth/`: `McpOAuthProvider` and `McpOAuthStateStore` (`provider.ts`; the state persists `codeVerifier` and `oauthState`, so a sign-in can complete in another request with the same store), `authorizeMcp` (`flow.ts`), `OAuthCallbackServer` (`callback.ts`), discovery and errors. - No dependency on other pi packages, and only `cross-spawn` at runtime. Keep it that way. ### 4.4 p2 today and after the port - p2 runs on Chord v2; Chord v1 is deleted. - Chord v2 design: `packages/chord/docs/design.md`. The parts MCP uses: `local` objects with empty params (host lifetime, §5, §6), shared store with per-principal `remote` views (§6 "Pattern"), natives `self.open` (§13), routines `self.on`/`self.onLive` (§14), changing registry contributions with `self.writable` (§17; the design's own example is MCP tools), flows kept by their owner (§20), `ctx.incarnation` (§4), Chord calls from Durable code (§24). - p2 code style for Chord v2 code: `AGENTS.md`, section "Chord v2 Code Style". Use the names the port introduces (harness extension registry, host addresses, principals, HTTP routing, login flow); do not invent parallel ones. - Existing p2 precedents to copy: `apps/p2/plugins/ports` (a local store plus a remote view with owner and worker variants), `apps/p2/plugins/locations/server.ts` (worker-only methods fenced by the worker's generation), `apps/p2/plugins/terminals/main.ts` (the server opening its own pi-env connections per environment), `apps/p2/plugins/envs` (environment configuration, SSH options, host keys), `apps/p2/plugins/login` (provider OAuth with `remote: principal.access !== "local"`), `apps/p2/plugins/radius/relay-access.ts` (requests relayed through Radius), `apps/p2/plugins/powershell/worker.ts` (a deferred tool offered per conversation), `apps/p2/src/runtime/principals.ts` (`OwnerPrincipal.access` is `"local" | "relay" | "device"`). After the port these files move or change; find their v2 equivalents. ### 4.5 As built (WP2-4): differences from sections 5-7 - **WP2:** `PipeTransport`, `McpPipe`, `McpPipeHandlers`, `PipeTransportOptions` from `@earendil-works/pi-mcp`; pi-env's `RemoteProcess` is an `McpPipe` without an adapter. Shared framing in `src/transports/lines.ts`. - **WP3:** `@earendil-works/pi-mcp/oauth` exports `startMcpSignIn`, `completeMcpSignIn`, `listenForLoopbackSignIn`, `waitForLoopbackSignIn`, `responseFromRedirectUrl`, `createMcpAuthProvider`, `McpSignInCancelledError`; files `sign-in.ts`, `loopback.ts`, `auth-provider.ts`, `client-settings.ts`. Differences: - `store` is an `McpOAuthServerStore` (with `withRefreshLock`); start and the auth provider take the lock, never call them while holding it. - `startMcpSignIn` returns `{ kind: "authorized" }` or a plain-data `McpSignInRedirect` (`authorizationUrl`, `redirectUrl`, `state`, `scope?`, all strings): store it with the flow. - `completeMcpSignIn({ ...sameOptionsAsStart, authorization, response })`: pass start's `redirectUrl` (not `authorization.redirectUrl`), the kept start result, and a response with `state` (checked). A sign-in replaced by a newer one is refused. - `clientMetadataDocuments: { baseUrl, listsRedirectUrl(url) }`; without it, dynamic registration only (p2 remote). - Start refreshes with the stored client first, whatever the redirect URI; `authorizeMcp` has `refreshOnly`, so a failed refresh never disturbs a pending sign-in (`3ffb210d9`: the refresh uses the app's `redirectUrl` when the documents list it; pass `redirectUrl` to `createMcpAuthProvider` too). - The store's `withRefreshLock` throws `McpRefreshLockBusyError` when the lock stays held (start then goes to the browser step); other lock errors fail the sign-in. Aborting ends the wait for the lock. - **WP4:** from `@earendil-works/pi-mcp`: `createMcpToolName`, `mcpNamespace`, `normalizeMcpInputSchema`, `limitMcpContent(content, saveOutput?)`, `MCP_OUTPUT_MAX_BYTES`, `validateMcpServerConfig(name, raw, options?)` returning `{ kind: "stdio" | "http", config }` (unknown fields kept as `unknown`) or `{ kind: "invalid", error }`, `isMcpServerOverride`. Apps validate their own fields (p2: `shared`, `runOn`, project rules) on the result. `"cimd"` is refused unless `checkMetadataDocumentCallback` is passed (p2 should not pass it). ### 4.6 As built (WP6-7): differences from sections 9-10 - **Leases, not load/unload:** the worker variant has `setLocations({ locations, projects? })` (every location the session's conversations work at; leases elsewhere end) and `call({ location, server, tool, args })`; the manager has `setLocations` and `releaseLeases` with the caller's session, generation and incarnation. Worker state: `servers` and `errors` (the session's leases' project refusals). - **Project files as text:** the worker sends `< - apps/p2/docs/optimistic-ui.md (9343 bytes)
# Optimistic UI in p2 Every action whose effect the user waits to see shows that effect at once: the switch flips, the row is renamed, the message is in the transcript. This page is the pattern p2's plugins follow, on the web and in the terminal, and the reference for plugin authors. ## Why it works without flicker Two rules make an optimistic value safe to drop the moment its call settles: 1. **A method that should feel instant awaits its own effects** and keeps them on the host that answers (Chord's guide; decision "App authoring rules confirmed by the p2 audit"). `Sessions.rename` commits the catalog before it returns; `ChatMode.set` returns once the followed Durable document changed. 2. **A call's answer follows its state** (Chord decision of 2026-11-05): before a host sends a `result`, it sends the page the `state` it committed. When the call resolves on the page, the new value is already there. So the page shows a guess while the call is in flight and simply forgets it when the call settles: on success the real value has replaced it in the same frame; on failure the real value is the old one, so the guess rolls back by itself, and the error is shown. Nothing on the page reconciles a guess with the server's value, compares ids or waits for state after an answer. A method that returns before its effect (a background job, a native that reports later) breaks rule 1: its guess would vanish before the real value arrives. Such work shows a **pending state** instead (below), never a guess of its outcome. ## Web: `useOptimisticCall` `useOptimisticCall(value, apply, onError)` (`src/runtime/web.tsx`, from `@p2/app/v1/web`) is React's `useOptimistic` plus a transition: it returns `[shown, run, pending]`. ```tsx // plugins/chat-mode/web.tsx: the chat mode chip const { fail } = useComposerState(); // the app's toast const [{ enabled, name }, guess] = useOptimisticCall<Shown, Partial<Shown>>( { enabled: useReadable(mode?.enabled) ?? false, name: useReadable(mode?.name) ?? DEFAULT_AGENT_NAME }, (shown, change) => ({ ...shown, ...change }), fail, ); // ... <button onClick={() => guess({ enabled: !enabled }, () => mode.set({ enabled: !enabled }, BACKGROUND_CONTEXT))}> ``` - `value` is what the reference's state says now (`useReadable`); `apply(value, guess)` is a pure reducer. Guesses in flight are applied over the current value in order, so a state change arriving meanwhile is not hidden. - `run(guess, call)` shows the guess at once and keeps it until `call()` settles. Its errors go to `onError` (usually `useComposerState().fail`, the app's toast; a sheet with its own error line passes its setter). - `pending` is true while a call of this component is in flight: the label of a button doing long work. - The guess lives in the component that shows the value. When the value shows in several places, put the actions where the value is rendered: the Session sheet is rendered by the sidebar (`plugins/sessions/web.tsx`), which owns the rows, so a rename, pin, archive or restore changes the list at once. Other views (the top bar's title) show the state that arrives with the answer. Reducers in p2 to copy from: a list edit by id (`applyRow` in sessions), add/remove/flip (`applyPort` in `plugins/ports/PortsPanel.tsx`), save/delete (`applyEnv` in `plugins/envs/EnvsPanel.tsx`), a replaced value (the model choice in `plugins/chat/web.tsx`, the provider login state in `plugins/login/web.tsx`). ## Terminal: `scope.optimistic(read, reduce?)` The terminal has no React; Chord's `TuiScope.optimistic` is its `useOptimistic`, and p2's `TuiScope` offers it (redrawing the runtime): `scope.optimistic(read, reduce?)` returns `[read, add, pending]`. While calls are in flight the reader shows `reduce(current, action)` applied for each action in order (without a reducer the last action is the value), until every call added settles. `add(action, call)` returns the call: as with a React transition, its failure is the caller's, and p2 shows it with `runtime.perform` (`ctx.perform` in commands and overlays). ```ts // plugins/chat-mode/tui.ts [this.shownEnabled, this.switching] = this.scope.optimistic(() => this.enabledState); get enabled(): boolean { return this.shownEnabled(); } toggle(): void { const enabled = !this.enabled; this.runtime.perform( this.switching(enabled, owner.set({ enabled }, BACKGROUND_CONTEXT)).then(() => this.runtime.notify(enabled ? "Chat mode on" : "Chat mode off"), ), ); } // plugins/sessions/tui.ts: a reducer applies row edits by id [this.shownRows, this.editing] = this.scope.optimistic(() => this.received, applyRow); ``` - A surface keeps the server's value as received and reads it through the optimistic reader. - `pending()` shows long work (the `/compact` indicator); a reader whose value only changes on `add` and settle is a cheap input for a surface that rebuilds on change (the chat transcript's messages in flight). - Overlays and commands get a scope from their context: `ctx.scope()`, disposed when the overlay closes (`/agents`, `/ports`, `/mcp`). ## What has no guess: server-made values and long work - **Client-chosen ids** make creation instant. Send on the new-session screen shows the message at once (`plugins/chat/session-start.ts`); the page chooses the session's ID (`Sessions.create({ id })`), so a creation retried after a lost answer answers the same session, and holds the new session's main conversation while `create` runs, so its worker starts beside the creation, not after it. The messages then move to that conversation's outbox and the session opens; a refused creation puts them back into the composer. A sidebar folder's "+" opens the same screen in that folder. `/new`, `/clone` and `/fork` in the terminal choose the ID (`id`, `copy`) and open the session at once (it shows loading while a copy's files are copied). Each message has its own `requestId`. - **Server-made ids** are not guessed (`Ports.add`: the forward id is part of its address; `ProviderLogin.start`: the attempt): the page shows a pending row or screen, the new forward with its port and name but no address and no actions, the login sheet's spinner until the provider's first prompt arrives. - **Long work** is a job on the server; the page shows that it runs and how it ended, never a guessed result. Session lifecycle operations (`Sessions.operations`: a move, a clone, a fork, an archive) mark their row in the sidebar ("Moving…", "Copying files…") and in the terminal's selector; a failed one stays on its row with its error and Dismiss (`dismissOperation`; `/dismiss-failed` in the terminal), and the top bar says "Moving…" from the tap until the move is done (a move the busy worker refused shows in the toast too). Retry shows "Retrying…" until the worker runs, or its error; "Rebuild search index" (`Search.rebuilds`, `/reindex`) shows it running until done or failed; "Save and test" shows "Working…"; `/compact` shows the compaction indicator at once. - **Sending** is optimistic on both sides: the web's outbox (`plugins/chat/outbox.ts`) and the terminal's send show the message at once (in the queue while a run is busy, in the pending area while the conversation connects) and clear the editor; a refused one comes back to the editor with its error. A retry after a lost answer reuses the request ID, so it is admitted once. ## Load: no waterfalls The same instinct applies to loading: hold what a view needs in parallel, from values the page already has. - **Cached owner flag:** `useOwner()` remembers whether this browser is the owner across starts; `useCanWrite(session)` adds a guest's grant. Views that are the owner's (unread marks, search, models, the workspace button, voice) hold at once instead of after `Access` is live. The terminal starts with the role the launcher was admitted as. - **Holds with the view:** `SubagentControl`, `Drafts` and the file index hold when the view opens (who may write is known), not after `Chat` is live or the subagent list names the child; `/tree` holds one `SessionTree` per session for all its actions. - **Short links:** `/s/<id>` opens the session's main conversation as this device last saw it (`knownMain`), so the page holds the right conversation first instead of conversation 1 and a redirect. - **No waiting for state after an answer:** a share link is read from `links` when `Sharing.link` resolves. ## Tests - `test/p2/ui-optimistic.test.ts` (fixture `test/fixtures/optimistic.ts`): in Chromium, the guess and the pending state while a call is held open, the server's value after it passes, the rollback and the error after a refusal. - `test/p2/tui-optimistic.test.ts`: in the terminal, a session rename and pin shown while their calls are held open and a refused rename rolled back with its error; a sent message shown at once and returned to the editor when refused; a running operation marking its row, a failure announced and dismissed. - `ui-drawer.test.ts` (a new session shows its first message and attachment at once), `ui-sidebar.test.ts` (a folder's and a project's "+"), `session-start.test.ts` (one ID across retries), `tui-location.test.ts` and `tui-p4.test.ts` (`/new` opens the client-chosen session), `tui-fork.test.ts` (`/fork`, `/clone` open the copy). - apps/p2/docs/p2-notes.md (218318 bytes)
# p2 notes Current reference for p2: running, configuration, decisions and plugin notes. Runtime: Chord v2 (p2 runs on Chord v2 only), its [design](../../../packages/chord/docs/design.md) and [tutorial](../../../packages/chord/docs/tutorial/); how p2 uses it: **Chord v2** below. ## TODO - **TypeScript 5.9.3 pin:** p2 pins `typescript` 5.9.3 (nested in `apps/p2/node_modules`, a devDependency used only by `npm run check`; nothing imports it at runtime) only because the repo's TypeScript 7 has no Android ARM64 build, so `npm run check` could not type-check on Termux. Move p2 to the repo's TypeScript (drop the pin and the nested copy) once a portable TS 7 exists or checks run off-device; same for apps/mobile. - **TUI plugins:** the TUI still loads its facets with p2's own bundle and a `host.load` in the sandbox, not through Chord's plugins component (C2: Chord's child-process sandbox for the launcher). - **Guest TUI isolation:** the owner superseded the earlier QuickJS/secure-eval-worker plan with a plain locked-down Node worker, Node >=26.5.1 (verified on 26.11.1). Permission drops run in the trusted bootstrap before any data-URL bundle import. The launcher retains terminal, transport and credentials. Denial of service is explicitly accepted; V8 limits and an RSS/ArrayBuffer watchdog mitigate it without an OS sandbox. See **TUI** and README threat model. ## Running ```sh cd apps/p2 P2_FAKE=1 npm start # http://127.0.0.1:8788, data in ~/.pi/p2 ``` | Variable | Default | Meaning | | --- | --- | --- | | `P2_DATA_DIR` | `~/.pi/p2` | data directory: `server/` and `sessions/<id>/` (see **Host stores and files**), `settings.json`, `models.json`, `envs.json`, `projects.json` (imported once, see Projects), `mcp.json`, `mcp-auth.json`, `auth.json`, `radius.json`, `forwards.json` | | `PORT` | `8788` | listen port (loopback only), so p2 never collides with pi-mobile on 8787 | | `P2_FAKE` | unset | `1`: the faux provider (`faux/faux-1`, echoes the prompt; `slow …` streams long; `rename-session <title>` calls the tool; `subagent`, `subagent-bad`, `subagent-answer`, `subagent-configure`, `subagent-send`, `subagent-archive`, `subagent-status` exercise subagents, `subagent <json>` calls the tool with those arguments, as do `team <json>`, `history_search|read|handoff <json>`, `location_envs|dirs|move <json>`, `tool_search <json>`, `terminal <json>` and `mcp__<server>__<tool> <json>`; `codemode` runs a script calling bash and read, `codemode-restart` one whose slow middle call a restart interrupts, `codemode <script>` runs that script; `artifact` runs a script writing the html artifact `demo` and showing it; `view_<name> <json>` and `artifact_<name> <json>` call those tools directly (refused: they are code mode only); `steps` makes a turn of thinking, calls and words, `inline-steps` one with an inline artifact between two calls; `json <text>` answers exactly `<text>`; a one-shot agent whose task is not `json …`/`slow…` answers a small value matching its schema) | | `P2_IDLE_MS` | `300000` | idle time before a session worker stops | | `P2_PROVIDER`, `P2_MODEL` | unset | default model for new sessions after settings.json's `defaultModel` (both or neither; Pi Mobile's `PI_MOBILE_PROVIDER`/`PI_MOBILE_MODEL` work too) | | `P2_RADIUS` | unset | `0`: disable the outbound Radius brain connection even when signed in | | `PI_RADIUS_GATEWAY` | production Radius | gateway selected by `npm run radius:login` | | `P2_TERMINAL_DAEMON` | unset | optional local pi-env daemon binary; otherwise use packages/env release/development builds | - **Web UI** (Chord's web surface, slice 11; `src/hosts/server.ts`, `src/hosts/web.ts`): `webSurface` with `routes: false` builds the pages' base bundle from `src/hosts/web/entry.ts` (`main.tsx`: Chord's `startPage` with p2's API modules, core facet, React root, connection state and error reports; `themes: false`, p2 styles itself) right after the listen, resolving workspace packages through p2's tsconfig `paths` as the server's source resolver does (`src/hosts/web/base.ts`, Chord's `tsconfig`): their sources, never a `dist/` (pi-ai's `./utils/*` exports only `dist/`, which the phone never builds; `web-sources.test.ts` fails on any `dist/` input); the plugins' web facets are Chord plugin builds of kind `web` (host packages `src/hosts/web/packages.ts`: the API's kind-neutral and web modules, React, react-dom, pi-schema), served as scripts that register themselves, the current set as one combined file. `web.ts` serves them with p2's access rules: the page (p2's template with `pageScripts`, `data-web-version` the base hash), `/_chord/web/…` files (named by content, `immutable`; guests too: it is the app's code), and `/_chord/web/base` (public, `no-store`). A failed base build is an owner notice and pages answer 503. A plugin reload rolls web facets out with the server's and workers' (the surface's step last): an open page fetches the changed scripts and applies them in place (`host.reload` in the page; each plugin's stylesheet swapped with its code), so the transcript's scroll and the other panes stay; only base code (the entry, `src/web`, `src/runtime`, the API, Chord, React) needs a restart, and pages reload onto the new base: when its link drops a page asks `/_chord/web/base` alongside the reconnect (never before it: one check at a time, a drop while one runs asks nothing more), and another base stops the reconnecting and reloads the page. No polling. A page's link names its base (`/chord?web=<base>`); every other client names itself (`?client=tui`, `cloud`, `script` for scripts and tests; `src/runtime/stale-page.ts`). The server refuses a link naming another base, or neither a base nor a known client (a page from before 2026-11-12, which would re-hold types removed since), at once (close code 4001; the page reloads) and the server logs (source `stale-pages`, a log line, not a notice: nothing to act on) who it was: the owner's access or device, or the guest, the browser and its address. A TUI launcher from before the marker is refused too (it retries until updated). The last four plugin sets' files stay served. Web facets must not share module state with another plugin's files (each plugin's script holds its own copy of a file it imports from another plugin): `plugin-practices.md`. The TUI launcher still polls `/__p2_tui` (p2's own TUI bundle). There is no Vite, source-module server or `index.html`. - **Radius Cloud Brain** (`scripts/cloud-bundle.ts`, `.github/workflows/p2-cloud-bundle.yml` after CI passes on main; the R2 bucket `pi-ui` under the commit's full SHA, served by Radius under `/pi/ui/`): a static page built with Chord's builders as a p2 page is (`buildBase`, the plugins' web scripts registered with `registeredSource`, `pageBootstrap`), written as one script (the bootstrap, the plugins, the base bundle) and one stylesheet, assets under `/pi/ui/<sha>/assets/`. `main.tsx` sees Radius's `data-pi-bootstrap` on `#root`: it connects to its Chord URL (adding `client=cloud`; the brain forwards the relayed path with its query) and starts static (`startPage({ follow: false })`: no updates, never outdated). Test: `ui-cloud-bundle.test.ts`. - `npm run dev` also watches plugins and supervises runtime restarts; see **Dev mode** below. Under `npm start`, code changes (web included) take effect only after a restart. - **Data report** (`scripts/data-report.ts`, read-only, beside a live server): per session its title, location, conversations with their entry counts and newest entry IDs, subagents, share modes, artifacts, uploads and downloads, plus environment and forward names; never secrets. `--compare before.json` lists differences (exit 1). `scripts/verify-sessions.ts [url]` checks a running p2 against it as the local owner (signed in with the data directory's owner key: `--data`, P2_DATA_DIR or ~/.pi/p2): the session list, and the newest entry of every conversation through `Chat` for the 10 newest sessions and all with subagents or shares (stopping workers it started). `test/fixtures/v1-data` is a data directory p2 wrote on Chord v1 (generated at aee97d3a9; the generator went with v1); `v1-data.test.ts` opens a copy and checks both tools and the contracts. - One server per data directory (process lock). To try p2 next to a real setup without touching it, set `P2_DATA_DIR` to a temporary directory. - Loopback alone is nobody: the local owner holds a local session cookie, from a sign-in link opened on the host or a local sign-in with the data directory's owner key (TUI, owner-link, scripts; **Loopback owner** under Devices). Through an HTTPS tunnel host listed in `P2_ALLOWED_HOSTS` (comma-separated), the Devices plugin's owner cookie admits an associated remote owner; a share cookie admits only that session's guest. Open `/owner` to sign in with a one-time token (see README). Other Host/Origin headers and forwarded localhost requests get 403. Radius independently authenticates its owner; proxies never do. Associated cookies are remote owners (`access: "device"` with a device ID). See README for owner links, logout and cookie migration. - Tests: `node --import ./src/source.ts --test test/p2/<file>.test.ts` (fake provider, temporary data dirs, port 0). Wait for state changes with `until(readable, predicate)` (it subscribes). Never poll `ref.member.value`: an unsubscribed reference member keeps its last value. - Headless Chromium on Termux: `chromium-browser --headless=new --no-sandbox --no-zygote --single-process --remote-debugging-port=<port> --user-data-dir=<tmp>`; without `--no-zygote --single-process` every navigation fails with `net::ERR_ABORTED`. Drive it over CDP (`Page.navigate`, `Runtime.evaluate`). - Browser tests (`test/p2/ui-parity.test.ts`) find Chromium through `CHROMIUM_BIN`. On a Linux server without a system Chromium or root, a wrapper can run the Playwright image's Chromium with host networking, as the calling user, with `/tmp` shared for profiles (as on slayer: `CHROMIUM_BIN=~/.local/bin/chromium-docker`): `docker run --rm -i --network host --user "$(id -u):$(id -g)" -e HOME=/tmp -v /tmp:/tmp mcr.microsoft.com/playwright:v1.55.0-noble /ms-playwright/chromium-1187/chrome-linux/chrome "$@"`. Browser tests use `--disable-dev-shm-usage` because Docker's default shared-memory limit can otherwise make module reloads fail with `net::ERR_INSUFFICIENT_RESOURCES`; readiness waits for the new document, not stale DOM. - `proxy-marker` replaces `proxy-auth`: `X-P2-Proxy-Secret` is a constant downgrade marker, not a credential. Every proxied request must preserve its public Host and overwrite that header, including WebSocket upgrades. Marked/forwarded localhost Hosts are rejected; allowed public Hosts are remote and require owner/guest cookies. A forged marker cannot grant owner/device access. Neither HTTP basic auth nor `P2_PROXY_SECRET` is used. ## Dev mode ```sh cd /home/badlogic/pi/apps/p2 export PATH=/home/badlogic/node-v24.4.1-linux-x64/bin:$PATH # The proxy sets X-P2-Proxy-Secret: 1 on every request (never an owner grant). P2_ALLOWED_HOSTS=p2.mariozechner.at P2_PUBLIC_URL=https://p2.mariozechner.at \ P2_DATA_DIR="$HOME/.pi/p2" PORT=8788 npm run dev ``` Do not launch this beside another server using the same data directory or port. Moving a standalone server into supervised mode requires stopping that server and waiting for its exit first. To preserve its currently running sessions, seed `P2_DEV_RESUME` with their JSON ID array. A fresh supervisor otherwise starts no session workers. Use a temporary `P2_DATA_DIR` and a different `PORT` for experiments. - **Code changes:** saves never apply automatically. Adding or removing a plugin or facet, or changing a loaded plugin input, marks that plugin dirty; Reload atomically commits its backend changes and rebuilt bundles, then visible pages switch when their next five-second poll detects the new manifest. Loaded non-plugin runtime or bundle inputs mark **Server code changed**; Restart applies them, although a later successful plugin Reload also rebuilds bundle-only inputs. Plugin discovery does not parse source; facet load hooks and the bundles' captured input lists track the files in use. Configuration files retain their existing live-reload behavior. - **Owner controls:** **N plugins changed: Reload** lists the dirty names; **Server code changed: Restart** requests a graceful supervised restart. The owner-only, same-origin button API is `POST /__p2_dev`; state is `GET /__p2_dev`. All triggers publish a shared owner notice, such as **Reloaded chat, sessions** or **Reload refused: …**. **Show logs** opens the Logs plugin when loaded; otherwise the message stands alone. - **Plugin reloads:** an explicit reload builds exactly the named (or dirty) plugins (Chord's `plugins.build`), then rolls them out to the server, every running worker and the pages (`plugins.rollout([server step, workers step, web surface step])`) and rebuilds the TUI bundle. A plugin whose build fails keeps its old code (an added one stays out) and the reload is refused; a plugin that is not loaded cannot be reloaded. Compatible changes keep objects, streams and references (Chord swaps them in place); added plugins' objects activate; a removed plugin's objects end (their holders see them end). Dirty marks come from Chord's plugin watcher (`plugins.state`); saves during a reload stay dirty. State and contract changes still need Chord versions. - **Explicit restarts:** the supervisor sends SIGTERM and waits for exit, with SIGKILL only after ten seconds as fallback. IPC tracks/fixes the pre-shutdown running set. The replacement restores non-archived workers, including ones with no viewer, without resubmitting inputs; Durable resumes committed work and browsers reconnect. Detached terminals remain running. A crash or startup error is not retried automatically: fix the source, inspect Logs, then request another restart. The CLI waits for replacement readiness and reports startup failure. - **Supervisor controls:** `kill -USR2 "$P2_DEV_SUPERVISOR_PID"` is still an explicit graceful restart, with an owner notice. Stop the supervisor with SIGTERM/SIGINT. The running set is in memory and is lost if the supervisor dies. Supervisor/package/dependency/environment changes require relaunching `npm run dev`; a supervisor already running the older automatic-watch script must be relaunched once to adopt these explicit-only triggers. `npm start` has no dev controls unless explicitly given `P2_DEV=1`; standalone dev mode cannot restart itself. ### Agents: apply edits explicitly Run from `apps/p2`, with `P2_DATA_DIR` matching the running server (default `~/.pi/p2`): ```sh npm run reload # all dirty plugins, including additions/removals npm run reload -- chat sessions # exactly these plugins npm run - apps/p2/docs/p2-tui-port.md (145085 bytes)
# Porting pi's interactive TUI to p2 Goal: when an existing pi user opens p2's TUI (`npm run tui`), it should look and behave like pi's coding agent (`packages/coding-agent`, interactive mode), wherever that makes sense for a server-backed, multi-user, sandboxed app. This is not a feature-parity checklist. The guide lists every user-facing feature of pi's TUI, says how p2 gets each one, and gives the designs and the work plan for the port. Sections 1–9 are the porting design; where the build differs, the "As built" sections of section 10 describe the code. Notation: - **(a) copy**: copy the source into p2 (usually `apps/p2/src/tui/` or a plugin's `tui.ts`) and strip what section 11 lists. p2 must not import `packages/coding-agent` at runtime. `@earendil-works/pi-tui` as a library is fine. - **(b) Chord**: re-implement on Chord objects. The table names the object or contract that supplies the data, marked **new** when it does not exist yet. - **(c) skip**: not applicable, with the reason. **(d) deferred**: applicable, scheduled late. - **L:** a launcher action under a launcher grant (section 3). **G:** a guest-local preference (section 4). Path aliases. Every pointer starts with one of these or a repo-root path: | Alias | Path | | --- | --- | | `CA` | `packages/coding-agent/src` | | `IM` | `packages/coding-agent/src/modes/interactive/interactive-mode.ts` | | `CMP` | `packages/coding-agent/src/modes/interactive/components` | | `THM` | `packages/coding-agent/src/modes/interactive/theme` | | `REN` | `packages/coding-agent/src/core/tools/renderers` | | `TUI` | `packages/tui/src` | | `ENV` | `packages/env` | | `DUR` | `packages/durable/src` | | `P2` | `apps/p2` | `IM`, `CMP`, `THM`, `REN` and `TUI` line numbers refer to coding-agent and pi-tui as of the port; p2 code is named by file and symbol. Section 2 maps where the port lives in p2. --- ## 1. Feature inventory and how p2 gets each ### 1.1 Layout, header, footer, renderer | Feature | coding-agent pointer | p2 route | | --- | --- | --- | | Two renderers: regular (main screen, scrollback) and fullscreen (alt screen with own scroll, search, selection), switchable at runtime | `IM:873` mountInteractiveTui, `IM:890` switchTuiMode, `CA/modes/interactive/tui-renderer.ts:22` | Fullscreen only, as a **product decision**. Regular mode is technically possible: the filter passes text, cursor and erase sequences (`P2/src/hosts/tui-filter.ts:5-13`, `:49`). But in regular mode a worker swap (reload) re-renders the transcript below the old scrollback. That duplicates content unless the new worker restores `TuiMainScreen.captureRenderState` from the snapshot. If regular mode comes later, the reload state must carry that render state. | | Fullscreen layout: transcript `ScrollView` above a fixed dock (pending, status, widgets, editor, widgets, footer) | `CA/modes/interactive/chat-viewport.ts:22-45`, `IM:968-991` | (a) copy `CA/modes/interactive/chat-viewport.ts` (as built: `ChatSurface.layout`, `P2/plugins/chat/tui.ts`). | | Transcript scroll keys (page, half page, line, top/bottom, prompt jumps), "Jump to latest" indicator | `TUI/keybindings.ts:160-210`, `CA/modes/interactive/tui-renderer.ts:30-34` | (a) Use pi-tui's built-in keys. | | Transcript search | `TUI/tui-alt-screen.ts:168` options, `CA/modes/interactive/tui-renderer.ts:25-29` | (a) Free with `TuiAltScreen`; pass themed styles. Key precedence: section 6.2. | | Mouse: wheel, selection, copy-on-select, right-click paste, clickable logo | `CA/modes/interactive/tui-renderer.ts:35-46`, `IM:256` | Wheel and selection: (b), behind a **G** mouse policy plus launcher mode ownership (section 3.6). Copy is **L:copy**; right-click paste is **L:paste**. Easter eggs are (c). | | Startup header: logo, version, key hints (Ctrl+O toggles compact/expanded), onboarding line | `IM:1009-1075`, `CMP/pi-logo.ts:19` | (a) Hints come from the command registry (section 6). Show the server origin and the role (owner/guest) in place of the version. | | Loaded-resources listing | `IM:1772` | (d) Owner only: **new** resources view (AGENTS.md and skills from `P2/plugins/prompt/worker.ts`, enabled plugins). | | Changelog / update notices | `IM:843`, `IM:4647`, `IM:4679` | (c) Server releases are not pi releases. Reuse the slot for owner notices. | | Footer line 1: cwd, git branch, session name | `CMP/footer.ts:165-177` (session name `:174-177`), rendered at `:286` | (b) `Sessions.sessions` gives title, env and cwd. The branch comes from `Workspace.query({kind:"status"})`, participant only. Guests in view mode get no branch. | | Footer line 2: tokens, cache rate, cost, context %, model and thinking | `CMP/footer.ts:179-212` stats, `:220-284` model side | (b) Data provenance in section 1.16. | | Footer line 3: extension statuses | `CMP/footer.ts:290-299` | (b) `TuiFooter` registry (section 7). | | Terminal title | `IM:1132` | **L:title** (section 3). | | Progress / program status | `CA/modes/interactive/program-status-reporter.ts:17`, `TUI/terminal.ts:571` setTitle, `:576-584` setProgramStatus, `:587-600` setProgress | **L:progress**, gated by **G** `terminal.showTerminalProgress`. | ### 1.2 Editor and input | Feature | Pointer | p2 route | | --- | --- | --- | | Multi-line editor: kill ring, undo, word motion, jump-to-char, newline | `TUI/components/editor.ts:297`, `TUI/keybindings.ts:71-158` | (a) Already used. | | Mouse editing / cursor placement in the editor | `TUI/components/editor.ts:655-693` | (b) Requires mouse mode (section 3.6). | | `CustomEditor`: `onAction`, Escape/Ctrl+D handlers, paste-image hook, extension shortcuts, embedded working status | `CMP/custom-editor.ts:13`, `:84`, `:90-140` | (a) Copy, wired to the command dispatcher (section 6). The paste-image hook is removed: that key is launcher-owned. | | Border color by thinking level; bash-mode color for `!` | `IM:4528`, `IM:3125` | (a) | | Prompt history | `TUI/components/editor.ts:345-490` | (a) Seed it from the transcript's own user items, like `populateHistory` (`IM:3975`). Secret prompts never enter history (1.8). | | Paste markers `[paste #1 +N lines]` | `TUI/components/editor.ts:1309`, `:1092` | (a) Free; bracketed paste is allowed. | | Slash autocomplete with argument completion | `IM:725`, `TUI/autocomplete.ts:314` | (b) Fed by `TuiCommands` (section 6). Never instantiate `CombinedAutocompleteProvider`: it calls `readdirSync`/`statSync`/`spawn` (`TUI/autocomplete.ts:1-3`). | | `@file` fuzzy completion | `TUI/autocomplete.ts:149-240`, `:772`, `:503` | (b) **new** `FileIndex` (section 8). | | Tab path completion (`~`, absolute, parent) | `TUI/autocomplete.ts:520`, `:549-588` (homedir/statSync), `:600-680`, `:850-859` | (b) Cwd-relative only in p2 (section 8.2). | | Extension autocomplete providers | `IM:825` | (b) `TuiCompletions`, keyed by provider id (section 7). | | Ctrl+V clipboard paste (files, image, text) | `IM:3152`, `CA/utils/clipboard-image.ts:200` | **L:pasteImage**, launcher-matched (section 3). | | Drag & drop files | `IM:1047` (the terminal pastes paths) | **L:attach** with trusted consent (section 3.3). | | External editor | `IM:4608`, `CA/modes/interactive/external-editor.ts:14-46` (`editInExternalEditor`), `CA/core/settings-manager.ts:946` | **L:externalEditor**, launcher-matched; the command is **G** and never sent to the worker. | | Ctrl+C clear / double Ctrl+C exit / Ctrl+D exit on empty editor | `IM:4281`, `IM:4291`, `CMP/custom-editor.ts:118` | (a) As pi (10.1). | | Ctrl+Z suspend | `IM:4450` | **L:suspend**, launcher-matched. | | `!cmd` / `!!cmd` user bash | `IM:3346-3360`, `IM:6999`, `CMP/bash-execution.ts:21` | (d) **new** `Chat.shell` (participant, streamed through a live slot or an entry). | ### 1.3 Message rendering | Feature | Pointer | p2 route | | --- | --- | --- | | User message (Markdown on `userMessageBg`; OSC 133 zone markers) | `CMP/user-message.ts:13`, `:49-50` | (a) Plus p2 data from `buildTranscript` items (`P2/src/web/transcript.ts`): author label with the guest "unverified name" note, peer attribution, reply quote, plugin note, files and image count. | | Assistant message: ordered text/thinking, hidden-thinking label, error/abort/length notices | `CMP/assistant-message.ts:14`, `:58`, `:91`, `:142-168` (thinking blocks), `:185-198` | (a) `AssistantMessageComponent` (`P2/src/tui/components/messages.ts`), streamed with `updateContent(message, true)` by `AssistantView` (`P2/src/tui/components/transcript.ts`). | | Per-thinking-block and per-card expansion (click) | `CMP/assistant-message.ts:142-168`, `CMP/tool-execution.ts:162-167` | (b) With mouse mode; otherwise global Ctrl+T/Ctrl+O. | | Markdown + syntax highlighting; code-block indent | `THM/theme.ts:1022` getMarkdownTheme, `THM/theme.ts:930` highlightCode | (a) Port highlighting to `lowlight` (already a p2 dep, `P2/src/web/Code.tsx`). `markdown.codeBlockIndent` is **G**. Watch the 8 MiB bundle cap (`P2/src/hosts/tui-sandbox.ts:3`). | | Mermaid | `CMP/mermaid.ts:1-5`, `:60-88` | (d) Pure rendering (grok-mermaid). Mode `markdown.mermaid` is **G**. Lands as an ordered Markdown transformer (section 7). | | Skill invocation block | `CMP/skill-invocation-message.ts:11`, `IM:3945-3953` | (d) With `/skill:` (1.10). | | Custom messages / entries | `CMP/custom-message.ts:12`, `CMP/custom-entry.ts:11`, `IM:3858` | (b) `TuiEntryRenderers` by entry kind (section 7). | | Compaction summary | `CMP/compaction-summary-message.ts:10` | (a) Data: the `pi.compaction` entry. | | Branch summary | `CMP/branch-summary-message.ts:10` | (c) Until p2 has branch navigation. | | Reset marker | — | (a) `pi.reset` plus its handoff text (`MarkerView`, `P2/src/tui/components/transcript.ts`). | | Cache-miss / warming / thinking-dropped / compaction-cost notices | `IM:4122-4210` | (b) Cost notices from `pi.usage`. Cache warming is server state (1.14). | | Status lines (consecutive lines collapse), errors, warnings | `IM:3836`, `IM:4635`, `IM:4641` | (a) `StatusLines` in the dock (`P2/src/tui/components/chrome.ts`; 10.1). | | Nested codemode calls | `IM:3607` | (b) `docs["p2.nested-calls"]` (`P2/plugins/chat/contract.ts:182`). | ### 1.4 Tool call/result rendering The generic card is `CMP/tool-execution.ts:31`: - Backgrounds for pending/success/error, set in `updateDisplay` (`CMP/tool-execution.ts:270-276`). - Fallbacks (`CMP/tool-execution.ts:142-159`). - Expansion (`CMP/tool-execution.ts:202`) and the render context (`CMP/tool-execution.ts:120`). - Lifecycle methods `updateArgs`, `markExecutionStarted`, `setArgsComplete` and `updateResult` (`CMP/tool-execution.ts:170-187`). - Images (`CMP/tool-execution.ts:354`). Renderers are combined at `REN/index.ts:32`. | Tool | Pointer | p2 route | | --- | --- | --- | | Generic fallback | `CMP/tool-execution.ts:23`, `:142-159` | (a) Section 11 checklist. | | `bash`/`powershell` | `REN/bash.ts`, `CMP/visual-truncate.ts:69` | (a) Map Durable details (`DUR/tools/bash.ts:130` `fullOutputPath`). | | `read` | `REN/read.ts` | (a) Strip `getReadmePath` and local path resolution (section 11). | | `write` | `REN/write.ts` | (a) | | `edit` (word diff; pre-execution preview) | `REN/edit.ts:173-252`, `CMP/diff.ts:79` | (a) `renderDiff` needs the `diff` package, pinned. Drop the local preview (`computeEditsDiff` reads the local fs, `REN/edit.ts:189`). Use `details.diff` (`DUR/tools/edit.ts:127-137`). | | `grep`/`find`/`ls` | `REN/grep.ts`, `REN/find.ts`, `REN/ls.ts` | (a) For subagent/MCP tools; Durable's coding tools are read/write/edit/bash. | | p2 tools: artifact, send_file, subagent, ports, reload_plugins | web `P2/src/web/Chat.tsx:164-302` | (b) Plugin renderers (section 7). | | Images | `CMP/tool-execution.ts:354`, `TUI/components/image.ts:103` | (d) Text placeholders now (`P2/src/hosts/tui-worker-runtime.ts:103`). A later launcher-rendered image is described in section 3. | | Expand all (Ctrl+O) | `IM:4569-4590` | (a) | ### 1.5 Working indicators | Feature | Pointer | p2 route | | --- | --- | --- | | Spinner in the editor border; `setWorkingMessage/Visible/Indicator` | `CMP/status-indicator.ts:9`, `:38`, `IM:2346-2380` | (a) Text derived from `pi.live` (`ChatSurface.syncIndicator`, `P2/plugins/chat/tui.ts`; indicators in `P2/src/tui/components/status.ts`). | | Retry countdown (Esc aborts) | `CMP/status-indicator.ts:51`, `CMP/countdown-timer.ts:7`, `IM:3744` | (a)+(b) `live.generation.retry`; Esc runs `Chat.stop`. | | Compaction indicator (Esc aborts) | `CMP/status-indicator.ts:85`, `IM:3672` | (a)+(b) Needs **new** `Chat.abortCompaction` (P3 contract before the P4 UI). | | Hidden-thinking label, idle status | `IM:2382`, `CMP/status-indicator.ts:114` | (a) | ### 1.6 Themes | Feature | Pointer | p2 route | | --- | --- | --- | | `Theme` class | `THM/theme.ts:185-373` | (a) Instance design (section 5). Do not copy `THM/theme.ts:375-391` (built-ins read from disk). | | Tokens, schema, built-ins, system theme | `THM/theme-tokens.ts:24`, `THM/theme-schema.ts:52`, `:97`, `THM/dark.json`, `THM/light.json`, `THM/system-theme.ts:465` | (a) Built-ins are imported as JSON into the bundle. | | Terminal color query and scheme notifications | `THM/theme-controller.ts:31`, `TUI/tui.ts:1494`, `:961` | **L:terminalColors** (the launcher queries; the worker gets data). | | Custom theme discovery and loading | `THM/theme.ts:436` getCustomThemeInfos, `THM/theme.ts:487` loadThemeJson, `THM/theme.ts:734` startThemeWatcher; `THM/theme.ts:672` registers already-loaded themes into `registeredThemes` (declared `:670`); `CA/config.ts:614` | **G** The launcher reads and watches the files and posts validated JSON. | | Theme picker with preview; light/dark pair | `CMP/theme-selector.ts:13`, `THM/theme.ts:578` | (a) Saving goes through **L:savePrefs** with trusted confirmation. | ### 1.7 Models and thinking | Feature | Pointer | p2 route | | --- | --- | --- | | Model selector: search, all/scoped toggle (Tab), current and default badges, Ctrl+S make default, background catalog refresh with progress and timeout | `CMP/model-selector.ts:40`, `:94`, `:137-158`, `:184-218`, `:262-296`, `:362-403`; `IM:5354`, `IM:5206` | (a) Section 11 checklist: inject data and actions instead of `ModelRuntime`. Data: `Models.models` and `Models.defaultModel`. "Refresh" uses `ProviderLogin.refresh` per provider. Owner only: `Models` refuses non-owners (`P2/plugins/models/server.ts:62-64`) and `Chat.configure` is owner-only (`P2/plugins/chat/worker.ts:269-272`). Guests see the model read-only. | | Exact `/model provider/id` | `IM:5229` | (a) | | Scoped models, Ctrl+P cycling | `CMP/scoped-models-selector.ts:97`, `IM:5390`, `IM:4550` | (b) **new** `enabledModels` in `P2/src/runtime/settings.ts`, exposed by `Models` (contract in P3, UI in P4). | | Thinking selector, Shift+Tab, save as default | `CMP/thinking-selector.ts:36`, `IM:4539`, `IM:5185` | (a) `Chat.configure` / `Models.setDefault`; owner only. | | Per-model default thinking (`modelThinkingLevels`) | `packages/coding-agent/docs/settings.md:16` | (b) Server setting; - apps/p2/docs/p3-functions/conversations.md (16133 bytes)
# Functionality map: conversations, agent loop, chat mode, drafts, cross-session messages, sent files, artifacts Read-only, based on origin/private `5abe3f6d6`. Abbreviations: - **who:** O owner, GR guest who can read, GW guest who can write (full grant), W session worker, S system. - **where:** srv server, wkr session worker, web, tui, env (remote execution environment). - **durability:** eph ephemeral, rst until restart, pers persisted, tx transcript-bound. ## Viewing a conversation | domain | function | data | durability | who | where | depends on | |---|---|---|---|---|---|---| | conv | View a conversation live: newest entries (≤100 or 512 KiB, at least 20) plus all conversation docs | entries ~KB each; window up to 512 KiB | tx | O, GR | wkr→web, tui | watch conversation | | conv | Window start stays put while within 1.5× the limits, so new entries arrive as small appends | previous window start | eph | S | wkr | view | | conv | Page older history: 100 entries / 1 MB / 2000 scanned per page; compaction and reset markers always included | pages ≤1 MB | tx | O, GR | wkr | view | | conv | Keep entries that a compaction or reset dropped from the view; client keeps pages of 4 conversations | client cache | eph | O, GR | web, tui | history paging | | conv | Jump to an entry (search hit, quoted message), loading older pages until it is shown, then highlight it | entry id | eph | O, GR | web | history paging, search | | conv | Context usage (estimated tokens vs. model window), latest response's cache usage | numbers | tx (derived) | O, GR | wkr→web, tui | model metadata | | conv | Cost sheet: each response's cost, cache hit, provider, model, stop reason | ~100 B per response | tx | O, GR | wkr→web, tui | view | | conv | Model metadata for viewers: name, window, thinking levels; no credentials | small | eph | O, GR | wkr | model catalog | | conv | Working indicator: retry countdown, deferred response, compaction (with reason and attempt), running tool, working | live doc | tx | O, GR | web, tui | view | | conv | Render the transcript: thinking (toggle), per-tool cards (bash, codemode with its nested calls, read, write, edit diff, subagent, artifact, ports, send_file), inline images, failures, markers, subagent reports | entries | tx | O, GR | web, tui | renderer extensions | | conv | Show a tool cut short by a restart as "interrupted" (not failed) | error text pattern | tx | O, GR | web, tui | — | | conv | "Stopped" marker after Stop until an aborted response shows | client flag | eph | O, GW | web | stop | | conv | Tool paths relative to the conversation's own cwd; open a file from a card in the workspace | cwd | eph | O, GW | web | workspace | | conv | Expand tools / show thinking toggles; reading position kept per conversation across switches and reloads | client state | rst (TUI snapshot) | O, GR | web, tui | — | | conv | Follow output, scroll-to-bottom button, keep the reader's position while sending | client state | eph | O, GR | web, tui | — | | conv | Read-only bar for view-only guests ("Joined as X (unverified)"); share-ended screen | grant name | eph | GR | web, tui | access | | conv | "Session unavailable" with Retry and Show logs (owner), "ask the owner" (guest) | worker error | rst | O, GR | web | worker status, logs | | conv | Leave the view with a notice if the session is archived or removed elsewhere | catalog row | eph | O, GR | web | session list | ## Sending and the queue | domain | function | data | durability | who | where | depends on | |---|---|---|---|---|---|---| | conv | Send a message: text ≤1 MB, at least text or one attachment | text | tx | O, GW | web, tui→wkr | admission rules | | conv | Retried send with the same request id is admitted once, per author (id ≤200) | request id | tx (Durable) | S | wkr | — | | conv | While busy: steer the current run (default) or queue a follow-up | mode | tx | O, GW | wkr | — | | conv | Wait before admitting input until prompt resources are loaded (first message of a fresh worker) | — | eph | S | wkr | prompt resources | | conv | Attribute input: speaker block in the model message, structured author in the entry; owner name read at write time | author | tx | S | wkr | settings | | conv | Legacy speaker-block parsing of old entries (drop? p3 starts empty) | — | tx | S | all | — | | conv | Expand `/template` and `/skill:` on the worker before sending | text | eph | O, GW | wkr | prompt resources | | conv | `@` and Tab file completion | file index | eph | O, GW | web, tui | file index | | conv | List queued inputs with their mode | inbox | tx | O, GW | web, tui | view | | conv | Queue item back to editor: caller's text returns; other authors' texts become their restored drafts | texts | tx + drafts | O, GW | wkr | restored drafts | | conv | Remove a queue item (discarded, whoever wrote it) | — | tx | O, GW | wkr | — | | conv | Stop: withdraw every queued input, abort the run; own texts to editor, others' to their restored drafts; allowed even when read-only | texts | tx + drafts | O, GW | wkr | restored drafts | | conv | Withdraw own queued inputs (Alt+Up), report how many were already placed | texts | tx | O, GW | wkr, tui | — | | drafts | Restored drafts per (conversation, author); claimed once, first tab wins | ~KB each | rst (should be pers) | O, GW (own) | wkr→web, tui | stop, dequeue | | drafts | Composer draft survives navigation: web one draft in sessionStorage; TUI per conversation plus across worker swaps | text | eph / rst | O, GW | web, tui | — | | drafts | TUI pending send ids reused, sent once connected ("Sending when connected…") | request id | rst | O, GW | tui | reconnect | | drafts | TUI drafts and snapshots partitioned per scope, pending sends never replayed into another scope | snapshot | rst | O, GW | tui | project plugins | ## Configuration and conversation actions | domain | function | data | durability | who | where | depends on | |---|---|---|---|---|---|---| | conv | Choose model and thinking level: owner only, known model, known level | ref | tx | O | wkr | model catalog | | conv | The root conversation's model also updates the session's default (sidebar, next worker start) | ref | pers | S | srv | session list | | conv | Compact now (instructions ≤10k); abort running compactions (Escape) | task id | tx | O, GW | wkr | — | | conv | Start a new context with a handoff (≤100k); owner only | text | tx | O | wkr | — | | conv | Read-only conversations (scripts' one-shot agents): refuse send, dequeue, configure, compact, shell; Stop allowed; set at creation | flag | tx | S | wkr | subagents | | conv | Fork at a person's message into a new session, the message waiting in its editor (root conversation only) | — | pers | O | tui, srv | session clone | | conv | Clone the whole session | — | pers | O | srv | session clone | | conv | Copy the last answer (TUI via launcher confirm, ≤1 MiB; web copy button) | text | eph | O, GR | web, tui | launcher | | conv | External editor (Ctrl+G); double Ctrl+C exits; double Escape opens fork | text | eph | O, GW | tui | launcher | | conv | Open subagent and one-shot conversations; subagent bar shows tools, cwd, archive/restore | child list | pers | O, GW (archive) | web, tui | subagents | | conv | New-session screen: folder, env, model, first message with attachments | — | pers | O | web | sessions, uploads | | conv | Inline images ≤5 MiB if the model accepts images, otherwise list the path | image bytes | tx | S | wkr | uploads, model metadata | ## Shell commands | domain | function | data | durability | who | where | depends on | |---|---|---|---|---|---|---| | conv | `!cmd` (output joins context) and `!!cmd` (display only): one per conversation, in the conversation's cwd on its environment | command ≤100k | tx (entry) | O, GW | wkr, env | execution env | | conv | Live output tail: 50 KB / 2000 lines, sampled every 200 ms, at most 100 KiB/s; full output spilled to a file | ≤50 KB live | eph | O, GR | wkr→web, tui | — | | conv | Shell result becomes an entry, placed at the run's next boundary while busy; never starts a run | entry | tx | S | wkr | — | | conv | Cancel a command: its author or the owner; timeout reported | — | eph | O, GW | wkr | — | | conv | Retried shell request runs once (100 keys kept) | ids | rst | S | wkr | — | | conv | Shell cancelled when the conversation has no viewer for 30 s, or on reload (drop? accidental) | — | — | S | wkr | — | ## Chat mode | domain | function | data | durability | who | where | depends on | |---|---|---|---|---|---|---| | chatmode | Switch per conversation: owner switches, everyone sees it; forks and new subagents start off | bool | tx-side doc | O / GR | wkr | — | | chatmode | Messages without `@pi` become display-only chat entries (author, text, attachments, reply) | entry | tx | O, GW | wkr | send | | chatmode | `@pi` mention: digest of the chat since the last mention, recording which entries it passed on | ≤100 messages, ≤20 files, ≤4 images; scan ≤2000 | tx | O, GW | wkr | send, uploads | | chatmode | Composer chip with the count of chat messages not yet passed on | count | eph | O, GR | web, tui | view | | chatmode | Reply to a person's or Pi's message: client names the entry, worker derives author and a 60-char quote | quote | tx | O, GW | web, tui, wkr | — | | chatmode | Quote line and "with N chat messages" note in the transcript; dashed bubble; `@pi` completion first; hint text | — | — | O, GR | web, tui | renderers | ## Cross-session messages | domain | function | data | durability | who | where | depends on | |---|---|---|---|---|---|---| | xconv | Agent lists other top-level sessions: id, title, model, working/idle/stopped; archived excluded; ≤200 | ~20 KB | eph | W (root only) | srv | session list, worker status | | xconv | Agent sends another session an attributed message (≤32 KiB, steer or follow-up); starts the recipient's worker | text | tx (recipient) | W | srv→wkr | start worker | | xconv | Send is idempotent by tool call id; retried once if the recipient's worker went away | ids | tx | S | srv, wkr | — | | xconv | Agent reads another session's latest completed answer without sending (≤64 KiB, scans ≤2000, busy flag) | text | eph | W | wkr | — | | xconv | Only owner-initiated turns or authenticated peer messages may use it (follows nested calls ≤32 deep, scans ≤2000); guests never | — | — | S | wkr | attribution | | xconv | Refuse a stale worker generation, sending to yourself, archived sender or recipient; recheck after start | — | — | S | srv | worker hosts | | xconv | System prompt section describing the tool | text | — | S | wkr | prompt | | xconv | Peer input shown with sender label; owner opens the peer session, restoring it if archived | — | tx | O (link), GR (label) | web | session list | | xconv | Sender attribution as a JSON text prefix (drop? use a structured author) | — | tx | S | — | — | ## Uploads (attachments) | domain | function | data | durability | who | where | depends on | |---|---|---|---|---|---|---| | files | Upload an attachment: ≤25 MiB, same origin, outside the project directory | blob ≤25 MiB | pers | O, GW | web→srv | — | | files | At most 20 per message; ids resolved only to this session's uploads | ids | pers | S | wkr | — | | files | Copy attachments to a remote environment before admission; no fallback on failure | blob | pers (env) | S | wkr, env | execution env | | files | TUI attach through the launcher (picker, path confirm, Ctrl+V); remove last or all | upload id | rst | O, GW | tui | launcher | ## Sent files | domain | function | data | durability | who | where | depends on | |---|---|---|---|---|---|---| | sent | Agent offers a file: private snapshot ≤25 MiB read through the environment (remote too), optional name | blob | pers | W | wkr, env→data dir | execution env | | sent | Download for viewers: cross-site refused, byte ranges for audio/video, safe filename | stream | pers | O, GR | srv | session list | | sent | Card in the transcript; viewer panel with preview (image, audio, video, text) | — | tx | O, GR | web | — | | sent | TUI list of files sent in the loaded transcript; text preview ≤256 KiB (binary detected); save or open in browser | ≤256 KiB | eph | O, GR | tui, wkr | launcher | | sent | Workspace also creates download snapshots | blob | pers | O | srv | workspace | ## Artifacts | domain | function | data | durability | who | where | depends on | |---|---|---|---|---|---|---| | artif | Agent creates or updates an artifact: html, svg, markdown or text; ≤2 MiB; slug name; title ≤200; new immutable numbered version each time | ≤2 MiB per version | pers | W | wkr→data dir | — | | artif | Per-conversation index, not rewound by compaction or reset; forks start empty | ~100 B per version | pers | S | wkr | — | | artif | List: count button, picker newest first | list | pers | O, GR | web, tui | index | | artif | View: HTML/SVG in a sandboxed frame with CSP (https CDNs allowed, no app identity); markdown rendered; text; download; open tab | ≤2 MiB | pers | O, GR | srv, web | — | | artif | Preview diagnostics bridge: console errors, CSP violations, WebGL loss, ≤50 kept | messages | eph | O, GR | web | — | | artif | TUI card (title, type, version, text content when expanded); inline view ≤512 KiB (HTML/SVG as source); open in browser; download | ≤512 KiB | eph | O, GR | tui, wkr | launcher | ## Session lifecycle and liveness | domain | function | data | durability | who | where | depends on | |---|---|---|---|---|---|---| | lifecycle | Clone copies uploads, downloads and artifacts | blobs | pers | O | srv | clone | | lifecycle | Delete or archive cleans up blobs (missing today) | — | — | S | srv | session delete | | presence | Worker stays up while a person views; running calls block idle stop; sends refused during a move | counters | eph | S | wkr | keep-alive | | presence | Viewer count in the footer | count | eph | O, GR | web | presence | ## (1) Cross-cutting needs | need | where it shows | |---|---| | Blob storage with caps, immutable ids, private modes, range reads | uploads, sent files, artifacts, shell spill files | | Per-session data that clone copies and delete removes | blobs, restored drafts, chat-mode switch, artifact index | | Transcript-bound vs. app state | chat-mode switch and artifact index have "fork starts empty" and "not rewound" semantics | | Readers that must not start a worker | artifact list and preview, sent-file preview, cost sheet, peer state list | | Long work outliving the call, with cancel, dedupe and display until placed | shell, peer send | | Cancellation | Stop, abort compaction, abort shell, caller abort, worker going away | | Keep-alive vs. presence | viewers vs. lingering tabs vs. active calls vs. tasks | | Idempotency per author request id | send, write, shell, peer send | | Authorization split | view / send / owner-only, applied in every route and method | | Launcher grants for TUI side effects | copy, attach, download, open in browser, e - apps/p2/docs/p3-functions/core.md (44336 bytes)
# Core runtime, hosts and UI shells | domain | function (plain words) | data it needs or produces (what, roughly how big) | durability | who may see or do it | where it must be available | depends on (other functions) | |---|---|---|---|---|---|---| | sessions | Browse sessions grouped by execution machine and directory | Session summaries; titles, locations, timestamps, pin/archive flags | persisted metadata; ephemeral grouping | owner; guests see their shared session | web; server supplies summaries | session catalog, access, worker status | | sessions | Put pinned sessions first within a directory | One boolean per session | persisted | owner | web; server | session catalog | | sessions | Rename a session through its options sheet | Nonempty title | persisted | owner | web; server | session catalog | | sessions | Archive a session through its options sheet | Session identity | persisted | owner | web; server | archive operation, worker stop, access revocation | | sessions | Restore an archived session when opening it | Session identity; optional child conversation | persisted | owner | web; server | restore operation, navigation | | sessions | Browse child conversations and archived children | Child IDs, names, models, activity, archive flags | persisted metadata; ephemeral expansion | owner, guest read, guest write within shared session | web | conversation listing, access | | sessions | Remember choices for the next new session | One location and model/thinking choice | until restart of page | owner | web | environment selection, model selection | | sessions | Show work, unread answers and worker failures in session lists | Busy conversation IDs, unread flag, error string | ephemeral display; read marks supplied by session storage | owner; scoped status for guests | web, TUI | activity reporting, read-mark tracking, worker lifecycle | | sessions | Pause a worker safely for cloning | Session ID, admission fence, prior running state | ephemeral coordination | system for owner operation | server, session worker | task inspection, worker stop, external database/blob copying | | sessions | Stop a worker before archive/delete cleanup | Session ID; lifecycle operation queue | ephemeral coordination | system for owner operation | server, session worker | worker shutdown; catalog/blob cleanup is outside core | | conversations and agent | Open a session’s durable conversation store | One SQLite database per session; size grows with history and work | transcript-bound | worker, system | session worker | exclusive process lock, model configuration, execution environment | | conversations and agent | Initialize the root conversation | Model, thinking level, cwd | transcript-bound | worker, system | session worker | session catalog, durable storage | | conversations and agent | Reject startup when the saved root model is unavailable | Model identity and configured model inventory | until restart; failure remains until retry | owner sees failure; worker validates | session worker, server | model configuration, worker error reporting | | conversations and agent | Resume interrupted durable work after worker restart | Persisted tasks and submissions | transcript-bound | worker, system | session worker | durable storage, installed tools and prompt resources | | conversations and agent | Open an archived session for indexing without resuming work | Startup flag; archived transcript | ephemeral mode; transcript-bound input | system | server, session worker | worker startup, search rebuilding | | conversations and agent | Wait for initial prompt resources before accepting input | Readiness/cancellation state | ephemeral | worker, system | session worker | prompt-resource loading | | conversations and agent | Keep tool definitions consistent throughout an executing phase | Immutable definition snapshot | until phase ends | worker, system | session worker | tool availability, reload coordination | | conversations and agent | Enforce conversation-specific human-input restrictions | Read-only reason; conversation ID | ephemeral policy | worker enforces; owner/guest write affected | session worker | conversation ownership; stop remains separately allowed | | conversations and agent | Attribute human input independently of typed text | Author kind, stable ID, display name, text, attachment IDs/names | transcript-bound | owner, guest read, guest write; worker assigns | session worker, web, TUI | authenticated principal, owner-name setting | | conversations and agent | Put authoritative speaker identity before model-visible input | One short structured speaker block | transcript-bound | worker produces; model consumes | session worker | author attribution | | conversations and agent | Read legacy speaker and attachment encoding **(drop? old-session compatibility only)** | Leading speaker block and trailing attachment note | transcript-bound | worker, owner, scoped guests | session worker, web, TUI | legacy transcript | | conversations and agent | Make unanswered model failures visible after crashes and reconnects | Display-only error entry and deduplication receipt per failed input; recovery scans 100 records/page | transcript-bound | worker writes; session viewers read | session worker, web, TUI | submission settlement, durable commits | | conversations and agent | Build a readable transcript from committed and live activity | Entries, streaming response, tool results, nested calls, pending inputs, reset/compaction markers | ephemeral projection of transcript-bound data | owner, guest read, guest write | web, TUI; worker supplies data | conversation snapshots, retained history | | conversations and agent | Preserve visible history when context resets or compacts | Loaded entries merged by entry ID | until restart of client; bounded number of retained conversations, not total history bytes | owner, scoped guests | web, TUI | history paging | | conversations and agent | Load older transcript pages without accepting stale replies | Paging cursor, snapshot epoch, loaded entries | ephemeral | owner, scoped guests | web, TUI, session worker | history retrieval | | conversations and agent | Keep recently visited conversations inexpensive to reopen | Four inactive history stores; conversation references linger two minutes, cap four | until restart; timed retention | owner, scoped guests | web, TUI | subscription lifetime, history cache | | conversations and agent | Reveal a searched or quoted message, loading history if necessary | Session, conversation and entry IDs | ephemeral | owner, scoped guests | web | navigation, history paging | | conversations and agent | Show thinking, streamed answers and tool progress | Potentially large text; expandable tool/thinking cards | ephemeral display; transcript-bound source | owner, scoped guests | web, TUI | transcript projection | | conversations and agent | Show tool-specific results and nested script calls | Commands, paths, diffs, files, artifacts, child-agent links, nested call statuses | ephemeral display; transcript-bound source | owner, scoped guests | web, TUI | transcript projection, file/artifact/conversation navigation | | conversations and agent | Estimate current context usage | Active entries with edits applied; token estimate and model limit | ephemeral derived values | owner, scoped guests | session worker, web, TUI | active transcript, model metadata | | conversations and agent | Show latest-response cache effectiveness | Input/cache-read/cache-write token counts | transcript-bound source; ephemeral calculation | owner, scoped guests | web, TUI | usage accounting | | conversations and agent | Inspect response costs separately from session totals | One cost/model/time/result row per response; charts | transcript-bound source; ephemeral display | admitted conversation viewers | web; session worker supplies rows | usage accounting, active transcript | | conversations and agent | Record a user-run shell command without starting an agent turn | Author, command, exit/cancel/error state, output tail ≤2,000 lines/50 KiB, optional spill path | transcript-bound; spill file separately stored in execution environment | owner, guest write; scoped viewers read result | session worker, remote env, web, TUI | shell execution, durable write | | conversations and agent | Choose whether shell output enters the next model context | Inclusion flag: `!` includes, `!!` display-only | transcript-bound | owner, guest write | session worker, web, TUI | shell-result recording | | composition | Keep an unsent web draft through navigation | App-scoped text, send mode, reply target, submission identity, errors | until restart; text also persisted in tab storage | owner, guest write | web | current conversation, send operation | | composition | Restore the matching web draft after page reload | One saved conversation key and draft string | persisted in tab storage | owner, guest write | web | browser storage | | composition | Retain pending attachments across web navigation | Native files, upload IDs, preview URLs; ≤25 MiB/file | until restart of page | owner, guest write | web | file selection, upload | | composition | Show an image preview before upload | Browser object URL and original image | until restart of page | owner, guest write | web | attachment selection | | composition | Keep a reply target scoped to its conversation | Entry ID and display name | ephemeral; cleared on conversation change | owner, guest write where replies enabled | web | transcript selection, reply support | | composition | Retain stop/restored-draft state without attaching it to another conversation | Stopped conversation identity; restored text supplied by send/stop implementation | ephemeral; restored text can enter saved draft | owner, guest write | web, TUI | stop/dequeue and draft restoration outside core | | composition | Keep TUI drafts and uncertain submissions per conversation | Text; request ID and steer/follow-up mode per session/conversation | until restart; carried across UI-worker replacement | owner, guest write | TUI | send acknowledgements, snapshot transfer | | composition | Expand prompt commands before sending | Draft text and expanded template/skill text | ephemeral; sent result becomes transcript-bound | owner, guest write | web, TUI, session worker | prompt resources | | composition | Complete cwd-relative file references | Query ≤200 characters; ≤100 requested results, truncation flag | ephemeral | owner, guest write; not guest read | web, TUI, session worker, remote env | confined file index, filesystem changes | | composition | Insert quoted path completions correctly | Cursor location, token prefix, file/directory suggestion | ephemeral | owner, guest write | web, TUI | file completion | | composition | Keep agent mentions and slash commands distinct from file completion | Short input token; `@pi` and slash-command context | ephemeral | owner, guest write | web, TUI | command completion | | environments and execution | Select local or remote execution for a session | Environment name, cwd, connection | persisted location; until restart connection | owner configures; worker uses | server, session worker, remote env | environment configuration | | environments and execution | Use a local execution daemon when available | Daemon path and lazy connection; otherwise direct Node execution | until restart | worker, system | session worker | local execution support | | environments and execution | Resolve each conversation’s own working directory | Per-conversation cwd and environment cache | transcript-bound cwd; until restart cache | worker, system | session worker, remote env | conversation configuration, session location | | environments and execution | Repair cwd-following conversations before work resumes | Root and follower IDs; changed cwd values committed together | transcript-bound | worker, system | session worker | conversation listing, durable commits | | environments and execution | Refuse relocation while work cannot safely move | Active calls, tasks, pending submissions, busy conversations, refusal reason | ephemeral | owner requests; worker/system enforce | server, session worker | liveness, durable inspection, relocation checks | | environments and execution | Move an idle running session without losing client views | New machine/cwd; location revision; switched environment connections | persisted catalog and conversation cwd; ephemeral connections | owner; system coordinates | server, session worker, remote env | admission fence, catalog update, environment opening | | environments and execution | Recover a failed or hung relocation | Committed target location; 60-second move timeout | persisted target; ephemeral recovery | system; owner sees failure | server, session worker | worker restart, current catalog | | environments and execution | Reopen location-dependent resources after a move | Location revision and notifications | ephemeral | worker, system | session worker, remote env | committed relocation, prompt/workspace watches | | background and long-running work | Start a session worker on demand | Session initialization, selected capabilities, generation, PID | until restart | system for authorized session access | server, session worker | session authorization, catalog, exclusive lock | | background and long-running work | Prevent duplicate workers opening the same store | Lock keyed by real data directory and session ID | ephemeral OS-held lock | system | server, session worker | process locking | | background and long-running work | Bound worker startup | Readiness state; 60-second timeout | ephemeral | system; owner sees error | server, session worker | worker bootstrap | | background and long-running work | Stop workers only after genuine inactivity | Viewer, active-call and durable-task counts; default five-minute idle interval | ephemeral | system | server, session worker | liveness reporting | | background and long-running work | Keep detached active work from appearing idle | Active-call lease for work outside ordinary request lifetime | ephemeral | worker, system | session worker | work cancellation/completion | | background and long-running work | Distinguish shutdown from capability removal | Stopping flag; interrupted-work state | ephemeral flag; transcript-bound work | worker, system | session worker | durable resumption, reload disposal | | background and long-running work | Report worker crashes without corrupting newer status | Generation, exit code/signal, redacted stderr tail ≤4,000 characters | until restart | owner; system produces | server | worker lifecycle, log redaction | | background and long-running work | Retry a failed worker explicitly | Session ID; clears sticky startup failure and related notices | until restart | owner | server, web, TUI | worker stop, fresh access | | background and long-running work | Shut down workers on server loss or termination | Sh
- apps/p2/docs/p3-functions/environments-workspace.md (18388 bytes)
# Functionality map: environments, execution, terminals, workspace, file index, ports, radius remote access Read-only map of what the app must do in environments, execution, terminals, workspace, file index, ports and remote access, drawn from the p2 code. **Legend.** Who: O = owner; GR = guest read; GW = guest with write access; W = session worker; S = system. Where: Srv = server; Wkr = session worker; Web; TUI; Rem = remote env; Host = detached terminal host. Durability: E = ephemeral; R = until restart; P = persisted; T = transcript-bound. ### Environments (local, SSH) | domain | function | data | durability | who | where | depends on | |---|---|---|---|---|---|---| | env | list configured remote envs with config, test result, session count | ≤ tens × ~300 B | P (config), R (test) | O | Srv, Web, TUI | session catalog | | env | add or edit env: name, host, user, port, identity file, shell path, login shell, shell env vars | ~300 B each | P | O | Srv, Web | validation | | env | validate: name format; no leading `-`, whitespace or control chars in host/user; no `"` in identity file | – | – | S | Srv | – | | env | invalid config file keeps last valid set and shows an error | error string | R | O | Srv, Web, TUI | config watch | | env | config file edited by hand is picked up (file watch, polling fallback) | – | R | S | Srv | file watch | | env | delete env, refused while any session (archived too) uses it; removes its managed key | – | P | O | Srv, Web | session catalog | | env | env revision = hash of config; part of env identity ("same id sees same files") | 12 hex chars | derived | S | Srv, Wkr | – | | env | generate an ed25519 key per env, kept in the private data dir, adopted as identity file; existing key kept | key files | P | O | Srv, Web | key tool | | env | upload a private key (≤ 16 KiB, passphrase keys refused), derive the public key | key files | P | O | Srv, Web | key tool | | env | show the public key to install on the host (lookup debounced 300 ms) | ~100 B | – | O | Web | – | | env | connection test: connect like a session (detect, deploy daemon, start), latency, platform, arch, home, version, watch mode, warnings | ~500 B | R (per revision) | O | Srv, Web, TUI | SSH, daemon deploy | | env | unknown or changed host key: show fingerprints, trust after confirm, or forget | key lines | P (own known_hosts) | O | Srv, Web, TUI | test | | env | host keys kept apart from the user's known_hosts, aliased per env name | file | P | S | Srv, Host | – | | env | warm SSH connections of all envs when the folder picker opens (drop?: opens every env, even unused ones) | – | E | O | Srv | browse connection | | env | browse an env's directories: `~` and relative paths, symlinks followed, ≤ 10k entries scanned, ≤ 1000 shown, "truncated" flag | ≤ ~100 KB | E | O | Srv, Web | browse connection | | env | browse this device's directories (same caps) | same | E | O | Srv, Web | – | | env | one shared browse connection per env, closed after 10 min idle or on config/key change | SSH connection | R | S | Srv | – | | env | recent directories per env offered in the picker | from session list | derived | O | Web | session catalog | | env | resolve a working directory for a new or moved session, optionally create it, check it is a directory | path | – | O | Srv | browse connection | | env | TUI `/envs`: list, mark current, Enter moves the session, Ctrl+T test, Ctrl+K review host key, Ctrl+F forget | – | E | O | TUI | relocation, test | | env | sessions on a changed env restart their worker once idle (no running tasks, same generation) | pending map ~ sessions × 40 B | R | S | Srv | worker status, catalog | | env | separate copy of test results (drop?: duplicates the persisted test state) | – | R | S | Srv | – | | env | worker opens an env-config watcher it never reads (drop?: unused) | watcher | R | S | Wkr | – | | env | full list as a method next to the live list (drop?: same data as the live state) | – | – | O | Srv | – | ### Execution | domain | function | data | durability | who | where | depends on | |---|---|---|---|---|---|---| | exec | local execution env at the session cwd | – | R | W | Wkr | – | | exec | remote execution env over SSH (pi-env daemon), connects at first operation; offline is that operation's error | SSH connection | R | W | Wkr, Rem | env config, daemon deploy | | exec | a remote session never falls back to local tools; unknown env refuses | – | – | S | Wkr | – | | exec | daemon binary per remote platform/arch, deployed over SSH; local dev build or override variable | binaries ~ MB | P (remote) | S | Wkr, Host, Rem | – | | exec | subagent with its own cwd on the same connection | – | R | W | Wkr | – | | exec | path rules of the remote OS (Windows/POSIX), remote home `~` | – | – | S | Wkr | connection | | exec | prompt names the SSH host the tools run on | string | – | S | Wkr | env config | | exec | move a session between this device and envs, or between directories: history stays, files not copied; refused while work runs (fence) | location | P | O | Srv, Wkr, Web, TUI | dir resolve, idle check | | exec | after a move, watchers reopen at the new location; old envs close after commit | – | R | S | Wkr | location change | | exec | private data dir denied to file views on this device (canonical check) | path | – | S | Wkr | – | | exec | active calls counted so a move or idle stop waits | counters | R | S | Wkr | – | ### Terminals | domain | function | data | durability | who | where | depends on | |---|---|---|---|---|---|---| | term | list a session's terminals (title, cwd, env, size, created, exit status) | ~200 B each | R (host lifetime) | O | Host, Srv, Web | – | | term | open a terminal at the session's cwd and env; refused for archived sessions | PTY | R (host) | O | Host, Srv, Web, Rem | session info, env config | | term | terminals survive server and worker restarts (detached host, one per data dir, process lock) | – | until host exits | S | Host | – | | term | host exits after 10 min with no terminals and no clients | – | – | S | Host | – | | term | attach to an existing terminal; only one of this session | – | E | O | Srv, Web | list | | term | attaching gets the current screen (headless emulator, 10k-line scrollback snapshot) | ≤ MB | R | O | Host | emulator wasm | | term | several viewers share one terminal; resize follows; last viewer's theme answers color queries; light/dark change reported | – | R | O | Host, Web | – | | term | output flow control: 256 KiB window, acks per 64 KiB, a viewer that falls behind gets a fresh snapshot; 8 MiB backlog cap disconnects | counters | E | S | Host, Srv, Web | – | | term | input typed before the program starts is buffered (≤ 8 MiB) | bytes | E | O | Host | – | | term | close: asks first if a foreground job runs (shows its name), force close kills | – | – | O | Host, Web | – | | term | title tracked from the program | string | R | S | Host | – | | term | one connection per env, reused, closed when no terminal runs on it; daemon checked for PTY support | connection | R | S | Host, Rem | env config | | term | byte channel opened with a single-use token (30 s), owner only; JSON on the control channel, bytes only on the byte channel | token | E | O | Srv, Web | – | | term | "host runs older code" notice; restart ends running terminals | version hash | E | O | Srv, Web | host version | | term | host startup and failure log, bounded and redacted | ≤ KB | P (bounded) | O | Host, logs | logs | | term | web: side or bottom placement, panel height, key-row toggle | localStorage | P (browser) | O | Web | – | | term | web: on-screen key row with sticky Ctrl/Alt, clipboard paste, long-press native menu, mobile IME edits retyped, touch scroll with fling, mouse reporting | – | E | O | Web | – | | term | web: parse output in time slices per frame so floods don't freeze the page | – | E | S | Web | – | | term | TUI terminals: none today | – | – | – | – | – | ### Workspace (files, git, review, uploads) | domain | function | data | durability | who | where | depends on | |---|---|---|---|---|---|---| | ws | list a folder: dirs first, gitignored entries marked, ≤ 2000 entries, `.git` hidden | ≤ ~200 KB | E | O, GW | Wkr, Web, TUI | exec env, git | | ws | read a file: ≤ 1 MiB text, binary detected, truncated flag | ≤ 1 MiB | E | O, GW | Wkr, Web, TUI | exec env | | ws | file as of a commit | ≤ 1 MiB | E | O, GW | Wkr, Web | git | | ws | git status: branch, head, changes incl. untracked and conflicts, ≤ 5000 | ≤ 1 MiB | E | O, GW | Wkr, Web, TUI | git | | ws | diff vs HEAD: context lines (3, 10, 25, full), untracked shown as added, unborn branch, ≤ 512 KiB | ≤ 512 KiB | E | O, GW | Wkr, Web, TUI | git | | ws | commit log: paged, unpushed commits marked, upstream name | ≤ 512 KiB | E | O, GW | Wkr, Web | git | | ws | one commit's message, changed files and per-file diff | ≤ 1 MiB | E | O, GW | Wkr, Web | git | | ws | literal text search: smart case, include/exclude globs, ≤ 500 matches, ≤ 20 per file, works outside a repo | ≤ 256 KiB | E | O, GW | Wkr, Web, TUI | git | | ws | all file paths for name filtering (git tracked+untracked, else walk ≤ 50k paths / 5000 dirs) | ≤ 4 MiB | E | O, GW | Wkr, Web | git | | ws | change notifications (watch tree, HEAD, index, config; debounced 100 ms) → viewers reload | counter | R | S | Wkr, Web, TUI | file watch | | ws | confinement: path rules, canonical root, no `.git`, private dir denied (also excluded from search) | – | – | S | Wkr | – | | ws | git hardening: no optional locks, no fsmonitor, filters disabled, no prompts or pager, no system config, 10 s timeout, output caps | – | – | S | Wkr | – | | ws | view of the viewed conversation's own cwd (subagent worktree) | – | – | S | Wkr | conversation cwd | | ws | download a workspace file (≤ 25 MiB; media by byte range; sandboxed attachment headers) | stream | E | O, GW | Srv, Wkr | read chunks | | ws | preview: images, audio, video, rendered Markdown | – | E | O, GW | Web | download | | ws | upload into a project folder: ≤ 25 MiB, 1 MiB chunks, temp file then rename, never overwrite, abort cleans up, writes serialized | ≤ 25 MiB | P (project files) | O, GW | Srv, Wkr, Web | exec env | | ws | line or range review comments on a file or diff, quoting the line, sent as the user's next message (steer or follow-up) | ≤ KB each | P (browser) until sent | O, GW | Web | chat send | | ws | review draft and open comment editor survive page reloads, per conversation | localStorage | P (browser) | O, GW | Web | – | | ws | outdated comment marked when its line changed | – | E | S | Web | – | | ws | UI: tabs (files, changes, history, search, terminal), wrap toggle, panel width, scroll memory, prefetch, split view | localStorage | P (browser) | O, GW | Web | – | | ws | TUI `/files`, `/changes`, `/grep` (read-only) | – | E | O, GW | TUI | queries above | | ws | TUI footer shows the git branch, refreshed on changes (500 ms debounce) | string | E | O, GW | TUI | status, changes | | ws | an open panel counts as a worker viewer (keep-alive) | counter | R | S | Wkr | – | ### Session files and blobs | domain | function | data | durability | who | where | depends on | |---|---|---|---|---|---|---| | blob | upload attachments to a session (≤ 25 MiB; image MIME types detected; stored outside the project) | ≤ 25 MiB each | P | O, GW | Srv, Web, TUI | – | | blob | resolve attachment ids to files of that session only; ≤ 20 per message | metadata | – | W | Wkr | uploads | | blob | copy attachments to the remote env before input is accepted; local copy kept for inline images | ≤ 25 MiB | P (remote) | S | Wkr, Rem | exec env | | blob | download snapshots: immutable private copy (≤ 25 MiB) made by the agent's send-file tool; link shown to viewers | ≤ 25 MiB | P | W makes; O, GR download | Wkr, Srv | – | | blob | serve a snapshot (cross-site requests refused; media by byte range) | stream | E | O, GR | Srv | – | | blob | clone/fork copies the session's uploads, downloads and artifacts | dirs | P | S | Srv | – | | blob | cleanup on archive or delete: none (drop? no: p3 needs it on session delete) | – | – | – | – | – | | blob | server-side snapshot save method that nothing calls (drop?: unused) | – | – | – | – | – | ### File index | domain | function | data | durability | who | where | depends on | |---|---|---|---|---|---|---| | idx | `@` completion: rank like pi (exact, prefix, substring, then fuzzy), dirs +10, limit 20 (max 100), kind filter | index ≤ 50k items | R (cache, kept 60 s) | O, GW | Wkr, Web, TUI | listing | | idx | Tab directory listing for path completion | ≤ 2000 entries | E | O, GW | Wkr, TUI, Web | folder list | | idx | completion refuses paths outside the cwd (`/`, `~`, `..`) | – | – | S | Wkr | – | | idx | build from the env's own listing, else from git/walk; at most one rebuild per second while files change; discarded after a move | – | R | S | Wkr | exec env, change notifications | ### Ports and forwards | domain | function | data | durability | who | where | depends on | |---|---|---|---|---|---|---| | port | list forwards with name, private/public URL, opener session title | ≤ 50 × 200 B | P | O | Srv, Web, TUI | radius origin | | port | add (≤ 50; p2's own port refused; random id; removed then re-added = new URL), remove, toggle public (confirmed) | – | P | O | Srv, Web, TUI | – | | port | agent tool `ports` list/open/close: top-level conversation only; owner-initiated or peer input only; idempotent | – | P | W (session from the worker's identity) | Wkr, Srv | input origin | | port | sessions on remote envs cannot forward | – | – | S | Srv | session info | | port | forwards a session opened close when it is archived, and at start for already-archived sessions | – | P | S | Srv | archive event | | port | tool cards web and TUI (only Radius-shaped URLs shown) | – | T | O, GR | Web, TUI | – | ### Remote access (radius) and proxy trust | domain | function | data | durability | who | where | depends on | |---|---|---|---|---|---|---| | radius | sign in to Radius by device code from a CLI; credential, server id, public origin in a private file | ~2 KB | P | O | CLI, Srv | – | | radius | outbound relay connection: owner check, token refresh, heartbeat (20 s ping, 60 s timeout), backoff 1–30 s, signed-out recheck every 30 s, disable switch | socket | R | S | Srv | credential | | radius | relayed browser requests become loopback requests marked with a per-process secret: owner, but treated as remote (no localhost-only powers); secret redacted from logs | secret | R | S | Srv | – | | radius | flow control: HTTP credit, congestion pause, one socket per WebSocket channel; old relay protocol versions kept (drop?: if the relay is current) | – | E | S | Srv | – | | radius | forward channels → `localhost:<port>`: Host/Origin/Location rewritten, 502 when nothing listens, unknown forward 404, anonymous only to public forwards, stream channels refused | – | E | S | Srv | forwards | | radius | rememb - apps/p2/docs/p3-functions/models-prompt-shells.md (17690 bytes)
# Functionality map: models, credentials, settings, prompt resources, code mode, coding tools, one-shot agents, logs, dev and reload, PowerShell, web and TUI shells **Legend.** - **Dur** (durability): E = ephemeral, R = until restart, P = persisted, T = transcript-bound. - **Who:** O = owner, GR = guest read, GW = guest write (send access), W = worker, S = system. - **Where:** Sv = server, Wk = session worker, Web, TUI, L = TUI launcher (visitor's machine), RE = remote environment. ## Models, credentials, settings | function | data | Dur | Who | Where | depends on | |---|---|---|---|---|---| | List the models of signed-in providers (the faux provider in demo mode) | ~hundreds of entries: id, name, thinking levels, context window; ~100 KB | R | O | Sv, Web, TUI | credentials, models.json, catalog cache | | Compose custom providers, models and overrides from models.json (baseUrl, api, headers, compat, cost, thinking map, input limits) | models.json, KB | P (owner-edited file) | O edits, S reads | Sv, Wk | config values | | Resolve config values: `!cmd` (10 s timeout), `$VAR`, `${VAR}`, `$$`; register the result as a secret | — | E | S | Sv, Wk | redaction | | Hot-reload settings.json, models.json, models-store.json and auth.json: watch, re-read once on a half-written file, keep the last valid state, report errors per file | fingerprints | R | S | Sv and every Wk separately | file watch | | Fall back to a 2 s poll when the watcher fails | — | R | S | Sv, Wk | — | | Default model for new sessions: setting, then env `P2_PROVIDER`/`P2_MODEL`, then the first signed-in model; thinking level clamped to the model | settings | P | O sets, all read | Sv, Web, TUI | catalog | | Set the default model and thinking level ("set as default" from a picker) | settings.json | P | O | Web, TUI → Sv | settings | | Model scope for cycling: `enabledModels` patterns, globs, `:level` suffix, model ids that contain colons; set and clear | ≤500 patterns | P | O | TUI only (none on web) | catalog | | Pick the conversation's model and thinking level | conversation agent config | T | O; guests see a read-only line | Web, TUI → Wk | conversations | | Cycle models (Ctrl+P / Shift+Ctrl+P over the scope) and thinking levels (Shift+Tab) | — | E | O | TUI | scope | | Refuse an unknown model at session create, at configure, and at worker start (visible error with Retry) | — | E | S | Sv, Wk | catalog | | Model access for runs: a request in flight keeps the configuration generation it started with | provider objects | R | W | Wk | credentials | | Run settings read at each decision: stream (transport, timeout, retries, cache retention), retry, compaction, tool execution, steering and follow-up modes | small | P | O edits file | Wk | settings | | Catalog cache (models-store.json): restore offline, refresh from the network on demand | per provider, KB–MB | P | S | Sv writes, Wk reads | — | | Providers with their login methods and signed-in type; never secrets | small | R | O | Sv, Web, TUI | catalog | | Sign in with an API key (secret prompt) or OAuth (URL, device code, pasted code, account select); 10 min deadline; one login at a time server-wide; it belongs to the starting device | prompts, notices | E (credential P) | O | Sv, Web, TUI | credentials, questions | | Remote owner: device-code login selected automatically | — | E | S | Sv | login | | Answer, cancel or dismiss a login prompt; select answers validated; secret answers redacted before use | — | E | O (starting device) | Web, TUI | redaction | | Abandon a login when its owner closes the sheet everywhere or disconnects | — | E | S | Sv | presence | | Device identity for providers that need one (device.json) | UUID | P | S | Sv | — | | Sign out of a provider (credential and cached catalog removed); refused while a login runs | — | P | O | Sv | credentials | | Refresh a provider's models; report the count | cache | P | O | Sv | catalog | | Credentials in auth.json: locked, validated, fail-closed at startup, last-good kept on a bad edit, keys from env or command, never sent to clients | KB | P | S; written by server and workers (OAuth refresh) | Sv, Wk | locked file | | Owner display name on new owner entries (≤40 chars, no `<>` or control characters) | — | P | O | Wk | settings | | Code mode on/off, applied live in every worker | bool | P | O (file edit) | Wk | settings | | Show a model error in the transcript as a display-only entry, deduped and recovered at start | small per error | T | O, GR | Wk | conversations | | Per-model cost rates and tiers for the cost panel | — | R | O, GR | Web | catalog | | Show settings, model and credential load errors | strings | R | O | Web, TUI | — | | (drop?) Fake provider with scripted prompts inside product code | — | E | S | Sv, Wk | move to test fixtures | ## Prompt resources | function | data | Dur | Who | Where | depends on | |---|---|---|---|---|---| | Global AGENTS.md (data directory, always local) plus AGENTS.md from the filesystem root down to cwd, read in the session's environment | ≤ tens of KB | R | S | Wk, RE | environment files | | Discover skills: `<data>/skills` or `~/.pi/p2/skills` on a remote, plus `<cwd>/.pi/skills`. Rules: a SKILL.md directory is one skill; root `.md` files count; frontmatter name and description; skip hidden and node_modules; first of a name wins | list of skills | R | S | Wk, RE | environment | | Skills block in the system prompt; `disable-model-invocation` skills hidden from the model | KB | R | S | Wk | discovery | | Prompt templates: global and `<cwd>/.pi/prompts` `.md` files, `description`, `argument-hint` | list | R | S | Wk, RE | environment | | Watch all of these (recursive, directory creation); poll fallback 30 s remote, 2 s local; 60 s load timeout; keep the last valid state plus an error | — | R | S | Wk, RE | file watch | | Reload from the new cwd and machine after a relocation | — | R | S | Wk | location | | A local session's first message waits for the first load | — | E | S | Wk | input gate | | App guide section (app source, docs, config, skills paths) | text | R | S | Wk | — | | List templates and skills for `/` completion | names, descriptions | R | O, GW (not viewers) | TUI; web has no completion UI | — | | Expand `/name args` (bash-style args, `$1`, `$@`, `${@:N:L}`, defaults) and `/skill:name args` (SKILL.md read at invocation) on the worker before send | text ≤1 MB | E | O, GW | Wk; Web composer, TUI | environment | ## Code mode, coding tools, one-shot agents, PowerShell | function | data | Dur | Who | Where | depends on | |---|---|---|---|---|---| | Run model-written JS (QuickJS VM, worker thread) that calls the conversation's tools as nested durable calls | script, journal | T | model in turns by O or GW | Wk | conversations | | Resume an interrupted script by replaying its journal; same VM fingerprint (fixed 256 MiB heap), else it ends with "sandbox changed" | journal | P | S | Wk | Durable | | Script deadline; output bounded to the token budget, overflow spilled to a file | — | T | S | Wk | — | | Offer or withdraw the tool live via the setting; a running script keeps its tool | — | P | O | Wk | settings | | (drop?) "Removed mid-script" report, then abort; only a reload artefact | — | E | S | Wk | — | | Code mode tool card: script, nested calls, agent rows | — | T | O, GR | Web, TUI | renderers | | read, write, edit, bash in the session's environment; send access permits tool actions (not filesystem isolation) | output caps | T | O, GW | Wk, RE | environments | | Preamble (speaker rules) and cwd section ("tools run on remote env X, SSH host Y") | text | R | S | Wk | location, environments | | Tool cards: bash, read, write, edit (diff), grep, find, ls | — | T | O, GR | Web, TUI | — | | One-shot agent (scripts only): fresh child conversation, prompt plus JSON Schema; parse plain or fenced JSON, validate, up to 3 retries with feedback; resolves to the value | child transcript | T (owned by the call) | W (script) | Wk | conversations, models, code mode | | Agent model, thinking level and tool subset (default: the caller's); validated before any commit | — | T | W | Wk | catalog | | Reattach after a restart (task document holds the child id; submissions `agent:N` deduped) | id | P | S | Wk | Durable | | Aborting the call aborts the child's work | — | E | S | Wk | ownership | | Child read-only for people (Stop allowed) and follows the session cwd | policy | T | S | Wk | conversations | | Agent rows on the script card (label, model); transcript opens read-only; not in the sidebar; cost counts toward the session | metadata | T | O, GR | Web, TUI | session cost | | PowerShell tool only on Windows: local from the platform, remote from the environment's OS report; ask again every 30 s if unreachable and after a move | bool | R | S | Wk, RE | environments | ## Logs and notices | function | data | Dur | Who | Where | depends on | |---|---|---|---|---|---| | Copy (tee) server stdout and stderr into a redacted ring without changing real output | 1000 lines / 256 KiB per source, 128 sources, lines ≤16 KiB | R | S | Sv | redaction | | Worker stdout, stderr and lifecycle under `session:<id>`; stderr tail (4000 chars) for the crash message | same bounds | R (server memory) | S | Sv, Wk | — | | Terminal host and dev supervisor publish redacted snapshot files; the server imports changes every 300 ms with dedupe; a final synchronous snapshot on exit | ≤2 MiB files | P | S | Sv, detached processes | file format | | (drop?) Importing those files by polling; they could be pushed over IPC | — | — | — | — | — | | Redact secrets: config files (auth, radius, models, settings, envs, owner key/token), secret env vars, resolved config values, login answers, codes and URLs, auth headers, Bearer/Basic, URL passwords, `sk-` keys; old secrets kept after rotation | secret set | R | S | Sv, Wk, detached | — | | View a source: filter, follow the tail, copy the visible lines, switch source; never starts a worker | ≤1000 lines | R | O | Web, TUI | — | | Owner notices: latest error per source (≤50) from host errors, worker failures, refused reloads and stderr `Error:` lines; dismiss | — | R | O | Sv, Web lane, TUI status lines and `/notices` | — | | Notice summary when a TUI opens; new notices shown as they arrive | — | E | O | TUI | — | | Dev notices merged with error cards; dismissal remembered per browser (identities only) | — | P (browser) | O | Web | — | | Worker start failure: error, Retry, Show logs; the failure is cached so background references do not respawn the worker | — | R | O | Sv, Web, TUI | sessions | ## Dev and reload | function | data | Dur | Who | Where | depends on | |---|---|---|---|---|---| | Watch plugin and runtime files; mark dirty only; track the files each bundle and host actually loaded | dirty set | R | S | Sv (dev) | — | | Reload the dirty or named plugins on the server and running workers; rebuild web and TUI bundles; refuse a removal while it is held; report the selection or the reason | — | R | O | Sv, Wk | bundles | | Supervised restart (graceful, SIGKILL after 10 s); restore non-archived workers; no duplicate inputs | running set (supervisor memory) | R | O | supervisor | — | | (drop?) Seeding running sessions through `P2_DEV_RESUME` | — | — | — | — | — | | Dev status and buttons (GET/POST `/__p2_dev`); "Server code changed" list | status | R | O | Web, TUI (consent) | notices | | CLI `npm run reload [-- names] [--restart]` over a 0600 Unix socket | — | E | account | host | — | | `reload_plugins` agent tool: dev server only, local session on the p2 checkout, root conversation, owner-initiated turn (attribution from the calling entry, nested calls walk up to it), not replayed after a crash; plus a prompt section about it | — | E | O-initiated | Wk | turn attribution | | Web: poll the manifest every 5 s while visible; reload on a new version or a newer link protocol | — | E | O, GR | Web | — | | TUI: poll the hash every 1 s; prepare a new worker; carry a ≤1 MiB snapshot (viewed identity, drafts, pending send ids, reading positions, toggles); swap; `/reload` forces | — | E | O, GR | TUI, L | — | | Serve bundles: content-named files, old files kept briefly, lazy first build, a failed build keeps the previous one; owners and admitted guests only | ≤8 MiB | R | O, guests | Sv | — | ## Web shell | function | data | Dur | Who | Where | depends on | |---|---|---|---|---|---| | Layout: sidebar (grouped by where tools run, pinned first, most recently active directory first, working and unread markers), main area, phone drawer, collapse state | — | browser | O; guests see their session | Web | sessions | | Navigation stack: `/s/<id>/c/<n>` URLs, modal layers with locators, workspace and terminal carry-over, scroll memory, sessionStorage, Android CloseWatcher, Escape | — | tab | all | Web | — | | Composer state kept across navigation: draft (also in sessionStorage), attachments, reply target, Stop notice | — | tab | O, GW | Web | — | | New-session choices kept while browsing | — | E | O | Web | — | | Theme per browser (system/light/dark), no flash on load, synced across tabs | — | P (browser) | all | Web | — | | Reconnect with backoff ≤5 s; detect an ended share (HEAD /) and stop reconnecting | — | E | all | Web | access | | Installable app (manifest, icons), CSP nonce | — | — | all | Web | — | | Listing cache for Back and revisits | — | E | all | Web | — | ## TUI shell | function | data | Dur | Who | Where | depends on | |---|---|---|---|---|---| | Sign in: loopback is the device owner; remote HTTPS owner link or share link (display name) redeemed into a per-origin cookie (0600); an ended share exits and is forgotten | config.json | P (visitor) | O, guest | L | access | | Fetch a bounded, SHA-256-checked bundle; run it in a permission-dropped worker (no fs, net or env); watchdog at 512 MiB RSS | ≤8 MiB | E | O, admitted guests | L, TUI | bundles | | Filter output (safe sequences, 64 KiB frames); track terminal modes and restore them on exit, crash and swap | — | E | S | L | — | | Actions under a grant (own key or consent screen): upload a file or clipboard image, paste, copy, open URL, download, external editor, suspend, save prefs (diff), dev reload, sign out (this or all owner devices) | — | E | O (guests some) | L | — | | Visitor preferences: pi keybindings, settings and themes (read only), p2 prefs.json, watched; server defaults (tui.json) below them; privileged bindings never from the server | KB | P (visitor, server) | visitor, O | L, Sv | — | | Query terminal colors (OSC 10/11/4, mode 2031) | — | E | S | L | theme | | Commands: pi ids, contexts, rebinding, conflict check, F1 palette, `/hotkeys`, `/settings`, `/keybindings`, `/quit`, `/reload`, thinking and tool-output toggles | — | E | all | TUI | — | | Fullscreen layout; reading position kept while output grows; drafts per (session, conversation) | — | E | all | TUI | — | | Reconnect with backoff; references rebind | — | E | all | L, TUI | — | - apps/p2/docs/p3-functions/README.md (8006 bytes)
# p3 functionality map What the app must do, independent of p2's plugin layout. Step 1 of deriving p3's objects (`../redesign-decisions.md`). The detailed tables per area are in this directory: `conversations.md`, `sessions-access.md`, `environments-workspace.md`, `models-prompt-shells.md`, `core.md`. This page condenses them. ## Domains | # | Domain | Main functions | |---|---|---| | 1 | Sessions | create (local or remote, directory check, defaults); live list (pinned, grouped by machine and directory) that never starts a worker; rename, automatic title, agent rename; pin; archive and restore (stop worker, end guest access, close forwards); relocate (idle only, fenced, followers move, cross-machine refusal); clone; fork at a user message; delete (new); unread marks | | 2 | Session workers | start on demand; one worker per store; idle stop after 5 min without viewers, calls or tasks; sticky start failure with retry; generation fence on reports; crash reports; shutdown | | 3 | Conversations and agent | live view window; history paging and retained pages; jump to an entry; context, cache and cost; working indicator; tool cards and nested calls; send, steer, queue, dedupe per author request id; input gate until prompt resources load; structured attribution; stop and dequeue with restored drafts; model and thinking (owner); compact and reset; read-only conversations; model error entries; resume interrupted work | | 4 | Shell commands | `!cmd` / `!!cmd` in the conversation's cwd and environment; one at a time; live tail; spill file; result entry placed at the next boundary; cancel; dedupe | | 5 | Chat mode | per-conversation switch; chat messages without a turn; `@pi` mention with a digest; reply quotes | | 6 | Agents | named subagents (spawn, steer, configure, archive, report answers once, own cwd); one-shot agents for scripts (read-only, JSON result); messages between sessions (agent tool, attributed, idempotent) | | 7 | Code mode | scripts calling tools as nested durable calls; replay after restart; switch on and off | | 8 | Session files | attachments (≤25 MiB, copied to remote environments); sent-file snapshots; versioned artifacts with sandboxed preview; shell spill files; previews; ranged downloads; copy on clone; delete with the session | | 9 | Environments and execution | environment config (validation, keys, host keys, test); directory browsing local and remote; connection pools; daemon deployment; execution environment per session and per conversation cwd; reopen after a move; OS probe (PowerShell) | | 10 | Workspace | list, read, search, git status, diff, log, commits; file index and `@` completion; change notifications; uploads into the project; downloads; review comments sent as a message; confinement and git hardening | | 11 | Terminals | detached host process owning PTYs; attach with a screen snapshot; several viewers; flow control; single-use channel tokens; host version notice | | 12 | Access | owner on loopback, device cookies, sign-in tokens, sign out and rotate; share links per mode, join, unique guest names, grant revision, remove, reset; origin checks; proxy trust; revocation of live access | | 13 | Models, credentials, settings | model catalog (custom providers, cache); credentials (locked, never to clients); API key and OAuth login with prompts and notices; default model and scope; run settings; code mode switch; owner name; hot reload with last valid state | | 14 | Prompt resources | AGENTS.md, skills, templates read in the session's environment; watched; expansion before send | | 15 | Status and presence | per-session status (busy, tasks, usage, error); viewer counts; connected guests; keep-alive inputs | | 16 | Search | message index per session with catch-up cursors; title and text search; open a hit | | 17 | Remote access and forwards | Radius relay link; relayed owner treated as remote; forwards (≤50, public toggle, agent tool, closed on archive) | | 18 | Logs and notices | capture and redact; bounded rings; detached process snapshots; owner notices; dismiss | | 19 | Dev and reload | dirty tracking; plugin reload; supervised restart; bundle publication; client update detection; `reload_plugins` tool | | 20 | Web shell | layout, navigation stack, composer state, theme, reconnect, installable app | | 21 | TUI shell | launcher sandbox; consent actions (copy, upload, download, open URL, editor); preferences; commands; snapshot swap; terminal safety | | 22 | Transport | frames and limits; reconnect and resubscribe; protocol refresh | ## Cross-cutting needs | Need | Where | |---|---| | One session file store (caps, immutable ids, ranges, previews; copied and deleted with the session) | attachments, sent files, artifacts, spill files, workspace downloads | | Per-session data lifecycle across hosts (archive, clone, delete) | store, files, links, guests, index rows, read marks, children, forwards | | Reading without starting a worker | session list, status, children, unread, artifact list, logs, cost sheet | | Worker-to-server reports with a generation fence | status, children, index feed | | Catch-up after commits (cursor plus wake) | search indexer, subagent reporter | | Work that outlives its caller, with cancel, dedupe and an outcome | shell, relocate, clone, fork, login, peer send, scripts | | Idempotency per author request id | send, write, shell, peer send | | Cancellation chains | script → nested calls → child conversation; stop; login abandon | | Keep-alive versus presence | viewers, lingering tabs, active calls, tasks | | Execution environment that follows the session; file watching in it with poll fallback and last valid state; connection pools | workspace, file index, prompt, PowerShell, terminals, browsing | | Single writer for configuration files; central redaction; credentials never to clients | settings, models, auth, radius, envs | | Attribution of input (structured author, peer, owner-initiated checks) | send, shell, peer messages, privileged tools | | One permission matrix (owner, guest read, guest write, worker; terminals owner only; workspace not for guest read) | every function | | Revocation of live access | guests, devices | | Chunked byte transfer with ranges | uploads, downloads, attachment copies | | One notice channel | logs, dev, model errors, grants, failures | | UI state that survives navigation, reload and reconnect | drafts, pending sends, reading positions, review drafts | | TUI consent for local side effects | copy, upload, download, open URL, editor, sign out | ## Duplicated today (one function, several implementations) Input gate loop (4); latest answer text (3); download snapshot writers (2); upload handling (2 routes, 2 caps, 2 layouts); size and id validation (3); environment config parsing and SSH options (3); SSH connections to an environment (4); confinement and private-directory denial (3); full file listing (2); path resolution (4); "archived?" checks (5); presence leases (3); generation-fenced report stores (2); commit catch-up loops (2); revocation fan-out (3); thinking clamp and model lookup (4); watch plus last valid state (5); secret registration for redaction (5); notice channels (5); staleness checks (3); reconnect backoff (3); drafts (3); following the session location (5); "offer the tool only when" switches (3); themes and transcript renderers (web and TUI). ## Candidates to drop Legacy speaker and attribution parsing; peer attribution as a text prefix; shell abort after 30 s without a viewer; catalog and cookie migrations; guest last-seen; the subagents copy of conversation config; warming every environment connection when the folder picker opens; duplicate environment test state; the worker's unused environment watcher; old relay protocol versions; the proxy-marker object; unused snapshot methods; the fake provider in product code; polling log snapshot files; "removed mid-script"; `P2_DEV_RESUME`; old TUI snapshot formats; legacy default-model environment aliases.
- apps/p2/docs/p3-functions/sessions-access.md (15027 bytes)
# Functionality map: sessions, sharing and guests, devices and owner access, status and presence, search, subagents, provider logins **Legend.** - Durability: E ephemeral · R until restart · P persisted · T transcript-bound (forks and rewinds with history). - Who: O owner · Gr guest read · Gw guest write · W worker · S system. - Where: Srv server · Wk session worker · Web · TUI · Env remote environment · CLI. ## Sessions | function | data | dur | who | where | depends on | |---|---|---|---|---|---| | create session in a directory, local or on an environment; optional create-dir; optional title; model and thinking default to the last used | row ~300 B | P | O | Srv, Web, TUI, Env | dir check, model list | | check directory: absolute, `~` expanded, is a directory; remote: resolved on the environment | path | E | O | Srv, Env | environments | | refuse a model the configuration doesn't know | model ref | E | O | Srv | model catalog | | list all sessions live: pinned first, then recent; never starts workers | N × 300 B | P | O; Gr: sessions shared with them, not archived | Srv→Web, TUI | — | | suggest default cwd and environment for a new session (most recent active) | derived | E | O | Srv | list | | rename: 1–120 chars, whitespace collapsed; an explicit title is never auto-replaced | title, edited flag | P | O | Srv, Web, TUI | — | | automatic title: first user message, line breaks to spaces, 100 chars | text | P | S | Srv | message indexing | | agent renames the session via a tool (root conversation only; tool explains when to rename) | title | P | W | Wk→Srv | rename | | updated-at from message activity, never moves back; a move counts as activity | timestamp | P | S | Srv | message feed | | pin / unpin; pinned filter in TUI | bool | P | O | Web, TUI | — | | model and thinking level follow the root conversation's configuration | model ref | P | W | Wk→Srv | conversations | | archive: stop the worker whatever it does; end guest access; close session-owned port forwards; keep storage | flag, time | P | O | Srv | worker stop, revocation, ports | | restore; guests regain access | flag | P | O | Srv | sharing | | browse and search archived sessions separately | — | E | O | Web, TUI | search | | relocate (cwd and/or environment) | location | P | O | Srv, Wk, Env | worker lifecycle, subagents | | relocate · guards: refuse while busy (name the running subagents); refuse a cross-machine move if a subagent has its own cwd | — | — | — | — | — | | relocate · steps: fence the worker; update the record; move the running worker in place (60 s, else restart); release the fence on failure | — | — | — | — | — | | clone: store plus artifacts/downloads/uploads under a new ID, worker paused, refused while work runs, title "(copy)"; dirs 0700, files 0600, no links, no overwrite; partial copy removed on failure | session files (MB) | P | O | Srv | worker pause, session files | | fork before a user message of the main conversation: active context as of the entry before it, edits applied, head marker; title "(fork)"; failure archives the copy; subagent conversations not supported | entry drafts KB–MB | P | O | Srv + 2 Wk | history read, empty-seed check | | delete session — p2 has none (needed: removes all per-session data everywhere) | all | — | O | all | every per-session store | | worker start parameters from the session record; refuse archived (except an index rebuild without resuming); refuse an unconfigured environment | init | E | S | Srv | record, environments | | start worker on demand; idle stop after 5 min without viewers, calls or tasks | — | R | S | Srv | keep-alive | | failed worker start sticks with its error until the owner retries | error text | R | O retry; Gr sees no error | Srv, Web, TUI | — | | worker generation fences stale reports | counter | R | S | Srv | — | | TUI selector: current folder vs all, fuzzy title/cwd/ID, sorts threaded/recent/relevance, show IDs, confirm archive, `+ New session` | — | E | O (Gr list) | TUI | list, children, search | | `/new [env:]path`, `/cd`, `/name`; completion of environment names and directories | — | E | O | TUI | directory listing | | web drawer rows: busy, error, unread, subagents; Session sheet: rename, pin, share, cwd picker, archive (navigates away) | — | E | O | Web | status, children, read marks | | catalog schema upgrades (drop? p3 starts empty) | — | — | S | Srv | — | ## Access: owner, devices, guests, sharing | function | data | dur | who | where | depends on | |---|---|---|---|---|---| | loopback without cookie = device owner (TUI, CLI, first page load) | — | — | S | Srv | — | | loopback owner browser cookie: wins over a guest cookie; tells the owner's browser from a fresh profile on the join page | 32 B secret file | P | S | Srv, Web | — | | browser requests from another site get no principal, except a top-level navigation | — | — | S | Srv | — | | one-time sign-in token: new one per server start in a private file; `owner-link` prints the link; 10 min, single use, ≤32 outstanding, memory only | 32 B | R | local O mints; devices cannot | Srv, CLI, Web, TUI | — | | sign-in page: token from the fragment or pasted; claim body ≤1 KB, same-origin | — | E | anyone with token | Srv | token | | device cookie: signed (HMAC), per port, 30 days, checked per request (expiry, revocation, signature) | key 32 B | P | S | Srv | key file | | sign out this device (revoked until its expiry) / all devices (rotate key, clear tokens); live connections end; TUI via launcher consent | revocation list | P | O | Srv, Web, TUI | revocation | | one-time migration of the old cookie scheme (drop?) | — | P | S | Srv | — | | share link per session and mode (read-only, read/write), ≤1 each, created on first copy; URL on the first tunnel host, else loopback | 43-char token | P | O | Srv, Web, TUI | session open | | confirm a read/write link on a remote environment ("guests can run commands there") | — | E | O | Web, TUI | session env | | reset link: new token, its guests lose access at once | — | P | O | Srv | revocation | | join page: server-rendered, CSP nonce, title and access text; owner or already-joined guest redirected | — | E | anyone with link | Srv | session title | | join: name unverified, cleaned, ≤40, unique per session ("Ann (2)"); ≤1000 guests per link; cookie 30 days HttpOnly Strict; body ≤2 KB, same-origin | guest ~200 B | P | link holder | Srv | link | | same browser keeps its identity; a removed guest cannot mint a new one | — | P | S | Srv | — | | guest grant per request: session, mode, revision; void while the session is archived or unknown | — | E | S | Srv | session state | | revision bump on remove, reset, archive or restore ends live objects and sockets on server and workers | int | P | S | Srv, Wk | — | | remove (block) a guest | flag | P | O | Web, TUI | revocation | | a guest holds one grant per session; a browser joining more links gains their sessions; it lands in the session shared or joined last | grant row ~100 B | P | S | Srv | links | | share with a person by email (while connected to Radius): they open the brain in Radius, in any organization, signed in with that email; no cookie, no link; Enter copies the session's Radius address | email on the guest row | P | O | Srv, TUI, Radius | Radius relay principal | | report the emails with grants on unarchived sessions to Radius after every change and connect; Radius admits and lists the brain for them, and learns nothing else | email list | P (in Radius) | S | Srv→Radius | grants, archive | | connected guests now (name, mode), per tab | — | E | O | Web, TUI | presence | | guest reaches only its session: worker, files, subagents | — | — | S | Srv | grant | | guest UI: name and access in the header; read-only replaces the editor; full mode can send, upload, stop | — | E | Gr / Gw | Web, TUI | grant | | guest last-seen written, never read (drop?) | ts | P | S | Srv | — | | copy a link from the TUI through the launcher's consent | — | E | O | TUI | — | ## Status and presence | function | data | dur | who | where | depends on | |---|---|---|---|---|---| | per session: running, error, live task count, busy conversations (foreground only), usage and cost | ~200 B per session | R | O all; Gr own, no error text | Srv→Web, TUI | worker reports | | worker reports its task graph and usage on each change; retried after reload conflicts | — | E | W | Wk→Srv | — | | worker keep-alive counts Durable tasks (today fed by the status reporter) | count | R | S | Wk | task graph | | viewer count per session (unique per page or TUI) | — | E | O, Gr own | Web, TUI | presence | | TUI footer: env:cwd (branch) • title › subagent; tokens, cache hit, context %/window, session total cost, model and thinking | derived | E | O, Gr | TUI | conversation view, catalog | | `/session` details | — | E | O, Gr | TUI | footer | | retry a failed worker; notify the failure once | — | E | O | Web, TUI | worker lifecycle | ## Search | function | data | dur | who | where | depends on | |---|---|---|---|---|---| | index user and assistant messages of root and subagent conversations; skip shell output, Durable internal inputs, sender headers | ~transcript text | P (derived, rebuilt on schema change) | W→S | Wk→Srv | message stream, conversation list | | catch up from a per-conversation cursor after each commit; 200 per page; cursor moves after the whole range; idempotent | cursors | P | S | Wk, Srv | — | | search: title/cwd substring plus literal message terms (all required, no regex or phrases); active or archived scope; ≤41 hits, snippet ≤300, query ≤500; TUI debounce 200 ms | — | E | O | Srv, Web, TUI | index, list | | open a hit at its session and conversation | — | E | O | Web, TUI | — | | rebuild one session's index: stop, drop, rescan; archived without resuming (drop? no UI seen in this area) | — | E | O | Srv, Wk | worker lifecycle | | unread: latest root answer vs owner read mark, shared by all owner devices; marked when the root conversation is visible; seeded as read at first creation | 2 ints per session | P (marks), derived (answers) | O | Srv, Web | index | ## Subagents | function | data | dur | who | where | depends on | |---|---|---|---|---|---| | spawn named subagent: model, thinking (clamped), tool subset, optional own existing cwd, first message | registry ~200 B each | T | W (agent tool) | Wk, Env | models, conversations | | send (steer / follow-up), configure, stop, archive/restore, status, models (≤200, 10 s) | — | T | W | Wk | — | | fetch latest answer (scan ≤2000 entries) | — | E | W | Wk | — | | report each answer to the main agent exactly once; withdraw a message stuck in the queue | tasks | T | S | Wk | background tasks | | subagents cannot spawn subagents | rule | T | S | Wk | — | | code-mode one-shot agents: read-only, listed on the script card | registry | T | W | Wk | code mode | | children list per session for the sidebar without workers (name, model, thinking, tools, cwd, last entry, archived) | ~200 B per child | P (server copy) | O, Gr own | Srv, Web, TUI | worker reports | | archive/restore from UI; refused for one-shots | flag | T | O, Gw | Web, TUI | — | | `/agents`, `/main`, back-to-main link, TUI tool card | — | E | O, Gr | Web, TUI | — | | follow session cwd on relocate unless own cwd | — | T | S | Wk | relocate | | mirror of conversation model/tools into the registry (drop? duplicates conversation config) | — | T | S | Wk | — | ## Models and logins | function | data | dur | who | where | depends on | |---|---|---|---|---|---| | providers with login methods and signed-in type | small | derived | any O | Srv, Web, TUI | model config | | login by API key or OAuth: one at a time on the server | — | — | — | Srv, Web, TUI | credential store | | login · prompts: text, secret, select, pasted code | — | — | — | — | — | | login · notices: auth URL, device code, progress, info | — | — | — | — | — | | login · limits: 10 min deadline; device-ID file | — | — | — | — | — | | login · behaviour: device code auto-selected on remote connections; model list refreshed after | — | — | — | — | — | | login · state: small, E (finished state kept until dismissed); visible only to the starting owner, others are refused | small | E | starting O | — | — | | URL open and copy through the launcher | — | — | — | — | — | | cancel when the starting owner leaves everywhere (sheet closed, disconnect); back navigation cancels | — | E | S | Srv | presence | | cancel, dismiss | — | E | O | Web, TUI | — | | redact codes, URLs and typed secrets from logs; credentials never sent to clients | — | — | S | Srv | logs | | logout (refused during a login), delete cached models | auth file | P | O | Srv | — | | refresh a provider's model list, return the count | model cache | P | O | Srv | — | | `/login [provider]`, `/logout`, with completion | — | E | O | TUI | — | ## (1) Cross-cutting needs | need | where it shows | |---|---| | readers that must not start workers | session list, status, children, unread, search all need server-side copies | | worker→server reports with a generation fence | status, children, index feed | | catch-up after commits (cursor plus wake signal) | search indexer, subagent reporter | | per-session data across hosts with copy, archive, delete | store, artifacts, downloads, uploads, links, guests, index rows, read marks, children, forwards | | presence and leaving | guests connected, viewer counts, login abandon, idle stop | | keep-alive | tasks, calls, viewers | | cancellation of multi-step operations that outlive the caller | relocate, clone, fork, index rebuild | | revocation of live principals | guest remove/reset/archive, device logout | | secrets | redaction, file mode 0600, no credentials to clients | | server-rendered pages | sign-in, join (CSP nonce) | | size caps and timeouts | title 120, name 40, 1000 guests, 2 KB/1 KB bodies, query 500, 41 hits, 10 min token/login, 60 s move, 5 min idle | | blob directories per session | artifacts, downloads, uploads | | launcher consent actions in the TUI | copy, open URL, sign out | ## (2) Duplicated today | function | places | |---|---| | "archived?" checks | worker start, share open, relocate, clone, fork | | title cleaning | session rename, agent tool | | `~` and path resolution | create, relocate, TUI location parser, web picker | | presence leases | guests, viewers, login abandon | | subagent registry read | subagents, code-mode agent, search | | generation-fenced report stores | status, children | | commit catch-up loops | search, subagents | | revocation fan-out | sharing archive, link reset, device logout | | thinking-level validation and clamping | sessions, subagents | | server-page rendering with nonce | join, sign-in | - apps/p2/docs/p3-handoff.md (29779 bytes)
# Handoff: Chord v2 and p3 (read this first after every compaction) Maintained by the redesign session (`8b6ab5cc-8f95-421b-989a-de109636ff5a`, runs in p2 on the phone; its tools run on slayer). Updated whenever the state changes, together with the decision log. Use `history_search`/`history_read` for anything not written here. ## 1. What we are doing Redesigning Chord (the object runtime) and p2 (Pi Mobile) from scratch as **Chord v2** (`packages/chord/src`) and **p3** (`apps/p3`, later). Goal (owner): a minimal, powerful runtime with the fewest footguns, no memory leaks, the best possible performance and easy-to-grok code (size is not a goal by itself: owner, 2026-10-27). ## 2. The process (owner's) Chord v2 is built in **vertical slices** (`packages/chord/docs/slices.md`), each the thinnest end-to-end path through the layers it needs. Per slice: a spec written by this session (`slice-N.md`), reviews of the spec (Astra, Fable), the owner's approval, a build by a fresh Opus builder (never commits unless told), reviews of the uncommitted code (this session, Astra, Fable), fixes, the owner's review, commit. Every slice adds runnable examples (`test/examples/NN-topic.ts`); hot-path slices add benchmarks (`benchmarks.md`). Code follows the root AGENTS.md section "Chord v2 Code Style". Decisions go into the log at once and into `design.md` in the same commit. ## 3. Where we are (update this section) **UPDATE 2026-11-03 midday (supersedes what follows):** p2 runs on Chord v2: private `16329187c` (switch), phone deployed 12:38 (backup `~/p2-backup-20261103-1235/`, data-report --compare only growth, verify-sessions clean). Phone lesson: a pull without restart is not inert (the dev server hot-reloads changed plugins). Biome fix for the phone's WASM fallback `6c425c2c5`. Upstream since: loud mistakes `5bd014b1c` (+ test race `0c7c74198`), workers type fix `51bb8f5c0`, tutorial `24d3993b2` (Radius 01m6g8fnaqfdsa8ad21y2dfpy8; p2 chapter example waiting in `/tmp/tutorial/p2-chapter-example/`), int table keys `d7704516e`. Footguns list `/home/badlogic/p2-port/chord-footguns.md` (13 + 4 tutorial); owner: no trial app, p3 finds and fixes as we go. Open: C1 delete Chord v1 (`src/old`), C2 p2 TUI on Chord's child-process sandbox, place the tutorial's p2 example, Chord gaps (React state across plugin reload vs P4, one-time read forces restart, contract imports in tests, `invalid` not exported), i18n build, slice 9. **UPDATE 2026-11-03 night (history):** Chord v2 is the package root: upstream `e9c0ff4d8` (CI green), private `ce43d89b8` (p2 imports v1 from `/old`); Chord docs now `packages/chord/docs/` (decisions.md, design.md, slices.md), examples `test/examples/`. 59-chat on a table upstream `3c7a9ca0e`. Decision "p2 moves to Chord v2, then v1 is deleted" (`d5c4b80eb`, owner yes to all 11 points; 7 = createHttp later). Running overnight (owner wants the port done by morning, no phone deploy): `ui-back` (Opus; brief `/tmp/ui-back-brief.md`, report `/tmp/ui-back/report.md`; worktree `pi-ui-back` + `/home/badlogic/bundled-ui-work`; reverts Cristina's `1cf20071f`, ports bundled-ui UI changes back, bundled-ui builds from pi-private; UNCOMMITTED until the owner checks in the morning) and `p2-port` (Opus; worktree `/home/badlogic/pi-p2-v2`, brief `/home/badlogic/p2-port/build-brief.md`, plan `/home/badlogic/p2-port/plan.md`; P1-P3, S0 now; after ui-back, bring its result into pi-p2-v2 and tell p2-port to do S1-S8 with the UI; I make local checkpoint commits there, never pushed until reviewed). Chord move scripts kept in `/home/badlogic/chord-root-move/`. bundled-ui is public on purpose (jsDelivr needs it); its git history holds the UI source. **UPDATE 2026-11-03 (history):** slice 11 committed upstream `5bc414516` (CI green), merged into private `46905bc1c`. Decided and recorded: i18n (`9c248717d`, built after slice 11: carry-over), Chord v2 becomes the package's root (`1d81a6972`: src/old, /json and /watch subpaths, short Node subpaths, docs -> docs, no version bump). schema-guard scripts deleted (owner). `chord-root` is doing the real run (brief `/tmp/chord-root-run-brief.md`, scripts `/home/badlogic/chord-root-move/`): worktrees `pi-chord-up` (upstream part) and `pi-chord-priv` (p2's part); then an Opus reviews package description/README wording; then commit upstream, merge into private, commit p2's part. Next after that: i18n build, telemetry (slice 9), small examples; p3 work packages after the meeting. **UPDATE 2026-11-02 late night (history):** - Slice 11 sandbox is a child process (decision "The terminal's sandbox is a child process", upstream `128912d24`, design `10cf3264f`, slice-11.md `65c85bda2`). Opus rounds: recheck 2 (`/tmp/s11-recheck2-opus.md`: E7 getter on process prototypes, E8 SIGUSR1, E9 attach before yes, E10-E13) fixed by audit-builder (round 3); recheck 3 running in `s8-spec-opus` (`/tmp/s11-recheck3-opus.md`). pi-s11 is a local commit `wip slice 11` (made with --no-verify: REDO the final commit with hooks, amend message), rebased on upstream/main (no conflicts), slices.md carry-over cleaned and "What exists" gained 11-13, package-lock regenerated with npm 11.21.1 (`/tmp/npmlatest`, the repo's lock needs a newer npm than node 24's 11.4.2) and left unstaged; `npm run check` clean; full Chord tests running (`/tmp/s11-final-tests.log`). Then: commit (amend, PI_ALLOW_LOCKFILE_CHANGE=1 with package-lock.json), push upstream, CI, merge into private. - `chord-root` subagent (Opus) in `/home/badlogic/pi-chord-root` (worktree of origin/private): trial of making Chord v2 the package root, old v1 to `src/old` or p2 (Durable and env import the root too), as a re-runnable script (`/tmp/chord-root/`), brief `/tmp/chord-root-brief.md`, report `/tmp/chord-root/report.md`. Owner wants pi-private merged (into pi) this week. **CURRENT STATE 2026-11-02 (night); the older paragraphs below are history.** - **Owner's working mode now:** "go ahead, just do all the things. tired of reviewing": this session writes specs, runs builds, reviews and fix rounds, commits and merges without asking per point; only genuinely new decisions go to him (explain with code; one at a time). All subagents on `anthropic/claude-opus-5-5` (never Fable: "costs too much"; `s8-spec-fable` is just a name, it runs Opus). Sol (`s8-spec-sol`) and fresh reviewers keep failing with model_error on security reviews: drop them, use Opus (`s8-spec-opus`) alone if needed. - **Committed upstream (CI green, merged into private `edf7b4bed`):** slices 10, 12 (`9823079ad`, plugin model), 13 (`072e413bf`, app kit), the schema switch (pi-schema, TypeBox gone except coding agent; `6536f8012`, `be039fb31`, `a6bde34`, `6291b9bcc`), example `60-session-sync` (`15f00a786`). p2 migrated to pi-schema in private; phone runs `66641ca8c` (voice fixes round 2) + sts-server `08f6cf8d6` (partial transcripts) deployed on slayer. - **Slice 11 (web and TUI) is building, uncommitted:** `audit-builder` in `/home/badlogic/pi-s11` (base upstream `d6af4d18e`; later docs commits not imported: rebase at commit). Built: surfaces, `startPage`, React hooks, themes, TUI launcher + sandbox, app-kit follow-ups (`startServer`, `exportData`/`importData` without stopping), example `59-chat` (worker per channel, uploads), fix round 1 done. **Now:** the TUI sandbox becomes a **child process** (owner decided after Opus escaped the hardened worker twice: `/tmp/s11-recheck-opus.md` E1 `Set.prototype.has`, E2 `EventEmitter.prototype.emit` → real `process`): pipes not the terminal, env {}, `--permission`, `--no-experimental-sqlite`, length-prefixed frames, allowlist kept as second layer, V8 eval-flag toggle dropped, E3-E6 launcher fixes, escape tests ported, benchmark vs the worker (startup, 160 KB frame loop, key latency). Then: Opus recheck/escape round, fix, commit (rebase on upstream, check, Chord tests, CI), merge into private. Known gap: the child can still signal user processes (`process.kill`); in the Node gist. - **Node gist for Matteo** (owner forwarded): https://gist.github.com/badlogic/a83cd80d0669dbcf77e029ac7b25ac96 (`/tmp/node-gist/`): sqlite ignores fs permissions (and writes despite ERR_ACCESS_DENIED), fd/tty writable, real `process` via EventEmitter.prototype; asks: sqlite checks, fd permission, per-Worker permissions, signals/env. - **Decided today, recorded in `decisions.md` (upstream), not all folded into design.md yet:** plugin model points 1-8 (folded §21), app kit points 1-5 (folded §23), slice 11 P3-P9, looks and themes, `chord.plugins` as manifest, App kit naming and export, sync applies batches with the host stopped, p3 meeting summary (Armin), p3 notes (subagent `context: "inherit"`, `/tree` as in-session forks). - **Open with the owner:** i18n for the app kit (explainer `docs/i18n/`, Radius 01m6ek54jff7atbe4rrkg1w3zz v3: `defineMessages` per plugin, shared `messages.ts`, `t.message` typed messages as data, `Translations` registry per host, locale per user, servers send keys) - asked "record as decided?", no answer yet. Building it would follow slice 11. - **Meeting doc for p3** (2026-11-03): `docs/p3-features/` (book.md with {{img}}/{{clip}} tokens; build with `/tmp/p3-doc/inline.py` then the explainer build; pictures in `/tmp/p3-prep/shots/`), Radius 01m6efr04nezms4zx32p7pp0w1 v2. Research files `/tmp/p3-prep/{features,extensions,envs,sync-example}.md`. Other explainers today: theming (`docs/theming/`, 01m6ecp0ndf8hb1m0ea6dqy8jn), plugin primer (01m69waqspfwwt21bjhvsp5w28), schema charts (01m69x3dz2fsg9k1r33vde7hv6 v6). - **Worktrees:** `/home/badlogic/pi-observers` (docs, detach to upstream/main before each edit), `pi-s11` (slice 11), `pi-sync-ex` (sync example, committed), `pi-s12`, `pi-s13` (committed slices), `pi-schema-switch`, `pi-schema-chord`, `pi-schema-pkg` (schema work, committed; guard scripts untracked in pi-schema-switch, `packages/coding-agent/scripts/schema-guard/`, owner never decided keep/drop), `p2-voice`, `p2-shots`, `p2-schema` (private, committed), `/home/badlogic/pi-private` (private), `p2-live`, `pi-mono` (owner's). Phone deploys only through the phone dev session `d602d653-61f1-4f99-aa54-99267e9dbf4e`, from owner-started turns. - **Next after slice 11:** i18n (if decided), telemetry (slice 9), the small-examples pass; p3 work packages after the 2026-11-03 meeting. - **Process lesson (again):** never run a file write and the dependent git command in one parallel tool batch (it happened 3 more times today). State as of 2026-10-28: **slices 6b and 8 are committed upstream** (pi `main`: 6b `14b4c56e5`, the design-audit fixes for slices 1-6a `391e83fb1`, slice 8 `6eb99ad65`; CI green; merged into `private` `c684ee727`). Since 2026-10-27 all of Chord v2 (`src`, its tests and `packages/chord/docs`, the decision log `decisions.md` included) lives in pi upstream: commit there first (worktree `/home/badlogic/pi-observers`, detach to `upstream/main`), then merge into `private`. p2's check is clean on `c684ee727`; the phone runs `ff7cdfeae` (p2 itself unchanged since). Next: slice 7 (reload and rollout): spec written by this session reading the code itself, decisions one at a time, spec reviews, build, reviews. Then slice 9 (telemetry; C11 `ctx.trace` to decide). **State 2026-10-29 (afternoon):** slice 7a (the dependency model) is committed upstream (`339847eb1`, CI green, merged into `private`). Slice order now (owner): 7b reload in the kernel, 7c loader and dev, 10 HTTP and workers, 11 web and TUI (the app kit, optional `chord/*` subpaths), 9 telemetry last. 7b's brief `docs/slice-7b/` (artifact `slice-7b`, Radius `01m63r5q1febvtwpwqxrz76zn5`) and its six points are decided ("Slice 7b's open points, as recommended"; plugin npm dependencies bundled, external by opt-in). **7b is building:** spec `slice-7b.md` (`0c3675e64`), `s7b-builder` (radius/claude-opus-5-5) in `/home/badlogic/pi-7b`, brief `/tmp/s7b-builder-brief.md`, notes `/tmp/s7b-build-notes.md`; then build reviews by `s8-spec-{astra,fable,opus,sol}` (Opus on radius), fixes, commit. All Opus subagents run on `radius/claude-opus-5-5` (owner). The old `docs/explainer/` is stale (rewrite or delete: owner). Dependency model docs: `docs/dependencies/`. **State 2026-10-30 (afternoon):** slice 7b is committed (`4b535d57e`), and so is the **simplification audit** (`50af8c160`, CI green, merged into `private`): decisions A1-A15 in `decisions.md` ("Simplification audit"); cut `installing` and the static registry cycle edges; retry by a hop budget (`retry.hops`); a fresh `bound` after a swap; one rule after a live failure; `parked_long` for any long park (`state`); one seal token; about 255 code lines fewer. Accepted noise: the text codec's small calls (+4-8%, code shape of its decoder, `benchmarks.md`). **Slice 7c (plugins) spec** is written (`docs/slice-7c.md`, uncommitted in `/home/badlogic/pi-observers` with its decision "Slice 7c's points" and design §21; patch backup `/tmp/7c-docs.patch`); spec reviews by Fable and Opus (`/tmp/s7c-spec-review-{fable,opus}.md`) are folded only in part: the owner decided Q1 (b) a `mapStack(error)` helper, Q2 a facet throwing while evaluating fails its host's whole step, Q4 copied files may not import their plugin's packages, Q5 no kept esbuild contexts and remote shipping in 10/11 but the build result is plain data exposed for them; **open: Q3, per plugin at startup and reload** (the owner wants it analysed: why all-or-nothing was chosen, history #66178, #67234-#67252; my analysis says v2's closed schemas, version binding with waiting, ownership and registry versions remove the old counter-traces; next step proposed: Opus and Fable punch holes, then the owner decides). The mechanical review fixes (complete facet sets per host, relative source maps, host-package namespaces, `import.meta` as build errors, watcher ignores, require order, discovery details) still need folding into the spec. Then: 7c build, 10, 11, 9. `audit-builder` (radius/claude-opus-5-5) is idle with the v2 code in context. **State 2026-10-30 (evening):** **slice 7c (plugins) is committed upstream** (`a14cdf90f`, docs `b7e824952`): `src/node/plugins/` (discovery, esbuild builds, vm evaluation, `pluginFile`, `mapStack`, `installDependencies`, the watcher, `rollout`), `host.load`/`host.reload` per plugin with the closure rule, `details.committed` on an abort after a reload's commit (owner "yes"), examples 35-39 as folders with real plugin directories (owner: full coverage), benchmarks in `benchmarks.md`. Fix round 1 had 19 items from four build reviews and rechecks (all clean). The previous paragraph's 7c state is superseded. Reading the sources (owner, 2026-10-30): understand the APIs (every `src/*/README.md` and `index.ts`, `node/plugins/README.md`, the `Self`/`StoreSelf`/`Ctx`/`Host` interfaces); read implementations in full only where this session or a subagent changes them (all of `src` is ~1.26 MB, more than one con - apps/p2/docs/p3-reviews/digest.md (31773 bytes)
# Digest — native-object code-change detection review ## Status and scope This records the findings, review points, questions and recommendations produced by the `digest` subagent in this conversation. No implementation changes were made. Experiments were written under `/tmp/digest-exp/`. The functionality map is not included. **Owner-supplied design assumptions (decided for this review):** - Natives (sockets, processes, databases, watchers, sandboxes and class instances wrapping them) may only be held by objects, through `self.open(async open, close)` in the object's factory. The value never changes during the object's life. - Facets are pure objects and may not open natives. - Factories are synchronous and declare only. - A plugin reload loads the facet bundle using loader D (one esbuild script per facet), stages facet setup and every live object's new factory with a recording `self`, rolls back if staging throws, and then commits. - Pure objects swap in place, with new methods and retained documents and references. - Native objects restart (close, then reopen) only if their code changed. **Owner proposal, not a confirmed decision:** the native object's digest covers the module containing its factory (described as the `host.serve` call site) plus its transitive imports from esbuild's metafile. **Existing documented decisions used by the review:** loader D externalizes host packages and `apps/p2/src/**` into the base bundle; base changes cause full host restarts. Plugin graph guardrails prohibit dynamic imports, `require`, top-level await, module-level side effects and raw timers in facets. These are documented target rules, not claims that D is already implemented. **Still open:** all recommendations below, including `defineNative`, dedicated native modules, lint rules, `codeFiles`, digest manifests and native-holder separation. No owner acceptance of these recommendations was received in this conversation. The repository inspected still uses `self.resource` and resource versions. Its old resource replacement, dependency propagation and routines are evidence about existing code, not automatically decisions for the new `self.open` model. Where the original response assumed old resource-dependency behavior, that remains an open integration question. ## Original report to the main agent The following preserves the original report, with its original assertions and estimates. Qualifications and additional review notes follow it. **Recommendation:** use a digest per native module, not per facet. Each native object type is declared in its own module, `plugins/<p>/native/<name>.ts`, with `export const X = defineNative(Type, factory, opts)`, and the facet only calls `host.serve(X)`. The digest is that module's import closure taken from the esbuild metafile. The proposal as written (the module with the `host.serve` call site plus its imports) is correct today, but in every real plugin it restarts on any facet change. ### 1. Mechanics (measured) I bundled real facets in `/tmp/digest-exp/build.mjs`. `src/**` and bare packages were external. I used Node 24 because Node 26 on slayer is missing `libatomic`. - **The data:** `metafile.inputs[path].imports[]` gives `{path, kind, external}`. Internal edges form the graph, and a module's closure is a depth-first walk from it. The digest is sha256 over the sorted `(path, contentHash)` pairs of that closure. - **Type-only imports are erased.** `radius/contract.ts` does `import type { RadiusBrain }` and has no edge in the metafile, so type-only imports never cause a restart. - **Build cost:** radius/server.ts 15.8 ms, terminals/server.ts 7.0 ms, codemode/worker.ts 5.5 ms. Loader D runs this build anyway. - **Closure and hashing cost:** 0.2–0.5 ms per facet. - **Closures today:** - `radius/server.ts`: 5 of 5 bundled files (brain, relay-access, contract, ports/contract). - `terminals/server.ts`: 6 of 6 (including `sessions/contract.ts`). - `codemode/worker.ts`: 4 of 4 (including `models/contract.ts`). The factory's module is the facet entry, so its closure is the whole facet bundle. Here the proposal is the same as the coarse rule. **Where the digest is stored:** - The server computes it at build time and ships it in the facet manifest: `{facetDigest, natives: {modulePath: digest}}`. The revision already lists every facet with its digest (target-design §15). - `defineNative` gets its module path injected by the bundler, the same way loader D injects `import.meta.url` (B4). - Each host keeps the digest a live native object was opened with in memory. Natives do not survive a host restart, so nothing needs persisting. - In staging, the recording `self` marks which objects call `self.open`. Commit restarts those whose recorded digest is not the new one. **Base bundle:** exclude host packages and `src/**`. A base change already restarts the affected hosts (§14.2). Do include the loader version and esbuild options, because a different build changes code without changing any source. ### 2. Correctness: how behaviour can change without the digest changing | Way it can change | Matters? | Digest | Prevention | |---|---|---|---| | Closure over facet-level variables or functions | yes | caught only if the factory's module is the facet, which is coarse | `defineNative` must be top-level in a `native/` module; lint forbids `self.open` outside `native/` | | Values passed in from the facet (`serveRadius(host, {onStatus})`) | yes | missed by a declaration-module digest | `defineNative` takes no facet arguments, and the facet calls `host.serve(X)` with nothing else (lint) | | Registries and host values | only if captured in `open` | not code | read them at use through readables; capturing an entry in `open` keeps it stale. Hard to lint, so document it | | References to other objects (`self.get`) | no | n/a | they are contract-versioned calls; deps already reopen on change | | Dynamic `import()` | yes | string literals appear in the metafile as `dynamic-import`; computed paths are missed | already forbidden by the §14.5 static check | | Shared helpers and cross-plugin files | yes | caught (they are bundled) | false positives only | | Bundled `node_modules` (global plugins) | yes | caught; hashes cached by mtime and size | — | | External host packages and `src/**` | yes | not counted | covered by the base restart | | JSON and `?url` assets | yes | caught (they are metafile inputs) | — | | Files read or spawned via `import.meta.url` | yes | **missed** | see below | | Runtime config (`radius.json`, `P2_RADIUS`, env file) | data, not code | n/a | watch it, or use `retryOn`/deps | | Module-level state in a native module | yes | n/a | lint: no top-level mutable state in `native/` | **Out-of-bundle files:** `terminals/detached.ts` spawns `main.ts` and the ghostty wasm, which never enter the server graph. It already keeps its own hand-written list in `terminalHostVersion()`, and that list can drift. Generate it from a metafile instead, or let `defineNative({ codeFiles })` add those files to the digest. **Module-level state:** a kept native keeps running old-generation code. After a reload, the bundle is re-evaluated, so new pure code sees fresh module state while the native's callbacks see the old state. An example is `models/worker.ts` `settingsSources` (a top-level WeakMap). The same split breaks `instanceof` against a kept native's class, so lint that too. **GC note:** esbuild puts all of a facet's modules in one scope. A kept native therefore pins its opening generation's whole script, but only that one generation, so the alive set does not grow over reloads. The leak detector should expect it. ### 3. Granularity: plugins and how often digests would change Commit counts are for the facet closure versus the hypothetical native-only closure. Caveat: the git history covers only Oct 9–16 (65 commits, mostly a migration). The ratio is the useful signal. | Plugin | Native module(s) | Facet vs native commits | Cost of a restart | |---|---|---|---| | radius | `native/connection.ts`: `RadiusBrain` + `RelayAccess` (brain.ts, relay-access.ts). The `RequestResolvers` logic moves to a pure object | 11 vs 4 | drops the relay WebSocket, including the phone's own connection | | terminals | `native/service.ts` (service, detached, protocol) plus the per-session view lease; `codeFiles` = the `terminalHostVersion` list | 14 vs 5 | terminal WebSockets drop; detached hosts survive | | codemode | `native/sandbox.ts` (`CodemodeSandbox` is a host package) + calls.ts. Extension wiring stays pure | 11 vs 1 | kills running scripts | | workspace (worker) | `native/watch.ts`: only the watch. `writer` is plain state; `viewer` is a liveness lease | environment.ts (890 lines) changed 6×; split watch from file logic | cheap re-watch | | envs | `native/store.ts` (Observer) and `native/ssh.ts` (manager, remote-env, config) | 15 vs 2 | SSH connections close | | models (server + worker) | today `openResources` pulls in model-config and provider-config parsing | 16 vs 9 (`model-config.ts` alone changed 18× since April) | low (file watcher) | | search | `native/db.ts` (SQLite) | 13 vs 3 | trivial | | login | `native/logins.ts` | `logins.ts` 4 | aborts an in-progress login | | logs | `captureProcessLogs` + timer. The `collect` logic now inside `open` becomes a routine | n/a | cheap | For models, the rule should be: the native is only the Observer, and parsing becomes a pure derive. That would cut its digest changes to about 1–2. The same pattern applies to logs. **Layout rules that make the digest useful:** - Natives stay thin: `open`, `close` and `expose`. - All logic goes in pure objects that `self.get` the native. - No other plugins' contracts in a native closure unless it needs them. ### 4. Alternatives | Option | Correctness | Simplicity | Connection drops while iterating | |---|---|---|---| | Call-site module closure (the proposal) | good | needs a call-site locator (stack traces or an AST scan) | every facet edit, so the same as coarse today | | **Declaration module closure + lints** | good; gaps closed by lint, plus `codeFiles` | one helper, one manifest field | only real native-code edits | | Author-bumped `codeVersion` | bad: a forgotten bump means you reload and still run old code, with no signal | simple | minimal, but untrustworthy | | Separate natives facet per plugin | same as mine, but all natives of a plugin together | needs a second facet per surface and changes discovery | envs store+ssh and codemode sandbox+calls restart together | | Factory source text + imports | misses helpers and mutable module state; needs free-variable analysis (re-implementing the bundler) | complex | — | | Coarse: restart all natives on any reload | correct except out-of-bundle files | simplest | worst: radius drops the phone and codemode kills the script doing the reload | The rule I recommend has the same cost as the proposal, closes the closure and argument gaps by construction rather than by luck, and cuts restarts about 3–10× for radius, codemode, search and envs. For models and logs the gain comes only after their parsing and polling move out of `open`. ## Additional findings and reasoning produced during the investigation ### Correctness and layout — recommendations still open - The native-holder object is the restart unit, not merely its socket or its `open` function. If resolver, extension or business logic shares its factory, an edit to that logic restarts the holder as well. Keep native-holder objects small and isolate behavior in pure objects. - Radius concretely mixes `RadiusBrain`/`RelayAccess` ownership with the `RequestResolvers` contribution. Moving only `brain.ts` is insufficient if the digest still roots at `radius/server.ts`. Put the native declaration in a separate module and put resolver registration in a pure object. - Codemode similarly mixes sandbox ownership, `RunningCalls`, settings-derived extension enablement and `Extensions` registration. Extension wiring should be replaceable without closing the sandbox. - A helper such as `serveRadius(host, options)` does not solve closure tracking: facet callbacks or other values passed through `options` can change while the helper's module graph does not. A predeclared native descriptor with no arbitrary facet arguments avoids this hidden dependency. - A callback created inside the native factory or `open` is covered by that factory's module and imports. The problematic callback is one injected from outside that graph, or fetched from a registry and then retained. - A registry entry read at open-time can retain another plugin's old function after that plugin changes, without changing the native's digest. Read current registry values at use-time instead. This is a dataflow/lifetime issue, not something the import graph proves absent. - Host-provided code is trusted base code for this purpose, but changing runtime host values is separate from changing host code. The digest does not establish what runtime values were used to open a connection. - Bundled shared helpers must remain included. Excluding cross-plugin contracts to reduce churn is unsafe without proving those modules contain only inert declarations: contracts can include real runtime logic. Type-only imports are the safe way to avoid a runtime edge when only types are needed. - `terminals/server.ts` currently pulls `sessions/contract.ts` into the native digest through Catalog use. A separate service-holder module can avoid some such irrelevant edges; a session-specific holder may still genuinely need Catalog. - For JSON, text, wasm and other assets, inclusion depends on how the loader resolves and loads them. Assets supplied as esbuild inputs can be hashed; arbitrary runtime filesystem reads cannot. `?url` is a loader convention, not itself an esbuild guarantee. - Computed imports, `require` and reflective code loading should be rejected rather than guessed at. Loader D already proposes these graph restrictions. - Runtime configuration is not native code. `radius.json`, `process.env.P2_RADIUS`, env configuration and model settings need their own observation/reconciliation policy. A file digest cannot say whether a changed URL, credential or provider setting should reopen a native. - The native module and its helper graph should have no mutable module-level state. A kept native runs closures from its opening generation while newly evaluated code has fresh module state; this can silently split caches, callback tables or identity checks. - The cross-plugin `instanceof` warning also applies within one plugin across reload generations: a native created with the old class is not an instance of the newly evaluated constructor. Treat native references as opaque/structural values rather than testing them against reload-local classes. ### Granularity and false positives — findings and open recommendations - A module digest is conservative, not semantic equivalence checking. Comments, unrelated e - apps/p2/docs/p3-reviews/dsh-compare.md (33117 bytes)
# dsh-compare — DeepSeek Harness comparison research subagent ## Status and scope This archives the findings, review points, open questions and recommendations produced earlier in this conversation. It is not a new owner-approved design. The report below is preserved verbatim where possible; supplementary observations from the research commentary follow it. Evidence paths refer to `/home/badlogic/research/deepseek-harness` at `d743267388641bc76f17c45ce8b4c231aed1d32c` unless otherwise stated. Chord comparisons refer to `origin/private:apps/p2/docs/target-design.md`, `plugin-loading.md` and `questions.md` as consulted during the research. **Owner-decided requirements known from the supplied target design:** in-place plugin reload on every host without restarting it; remote objects with principals; plugins working across p2 versions; running calls and jobs finish with old code rather than being drained by reload; same-version resources kept; resource replacement/handover rather than an object restart; immutable pinned registry tables; no cross-host transaction; base/runtime code changes restart affected hosts. The target explicitly rejects drain-based handover, object restart for state changes, late-bound registry entries and `restart_required`. The owner's assessment supplied in the task was that objects plus references have no issues or footguns, whereas the other concepts—especially natives—do. **Still open / recommendations only:** every proposal below to remove `self.resource`, move natives into base supervisors, restart native-owning objects, make host installation all-or-nothing, or change Questions semantics. No owner acceptance of those proposals occurred in this conversation. Some intentionally conflict with the recorded target and would require a new owner decision. No later p3 decisions are known to this subagent. ## Original report **DeepSeek Harness (DSH) vs Chord: natives, reload, state** Read-only research; I made no edits. DSH checkout HEAD is `d743267` (2026-10-09). DSH's `native/` directory holds native addons (Landlock, flock) and is unrelated to "natives" in our sense. ### 1. Natives and their lifecycle DSH runs on Cordis, a plugin framework. In Cordis everything a plugin does is an *effect* inside a *fiber* (one running plugin instance). When the fiber is disposed, its effects are undone in reverse order and awaited until they have actually stopped (docs/cordis-primer.md:13, docs/defensive-patterns.md:19). Natives follow one pattern throughout DSH, though it is a convention and nothing enforces it: - **A few long-lived services own the natives:** processes in `ctx.subprocess`, PTYs in `ctx.terminals`, background work in `ctx.jobs`, plus LSP and MCP connections and session persistence. Ordinary plugins such as tools hold opaque branded ids and call methods. "The service owns process coordinates and lifetime; consumers own what a process means" (packages/subprocess/subprocess/README.md:98). - **A native's lifetime belongs to its owner, not to the code that created it.** Spawning takes the exact owning `Agent`. The service then attaches its cleanup to that agent's scope: `owner.ctx.effect(...)` in packages/terminal/terminal/src/index.ts:322-332 and packages/jobs/jobs-local/src/index.ts:610-627. "Registrations outlive producer and controller fibers" (packages/jobs/jobs/README.md:73). - **Access is fenced by the owner.** Another agent gets `FOREIGN_SESSION` (packages/terminal/terminal/README.md:52-56). - **Reconnecting lives inside the owner.** The MCP client has a connection supervisor that keeps numbered client generations, retries with backoff and rebuilds its registrations; consumers only see tools come and go (packages/mcp/mcp-client/README.md:93, :118, :129). - **Execution world:** the filesystem and subprocess services are swapped together for remote providers. Handles expose no path or pid (.agents/notes/implemented/architecture/2026-09-11-subprocess-control-pipe.md:21). **The generalization we lack:** natives are never values in plugin code. They live in owner services, are reached by id, and each one is bound to an owner identity. A native's lifetime is the shorter of the owning service's fiber and the owner's scope. A consumer reload keeps natives, because the owner's code did not change (subprocess README:113). A reload of the owning service kills them. ### 2. What DSH constrains that we allow - **No native survives a code change of its owner.** There is no `replace`, no resource version and no handover. - **Dependents always restart.** A fiber's activation epoch is the list of its providers' fiber uids (vendor/cordis/src/fiber.ts:611-622). A provider reload changes that list, so every consumer is torn down and set up again (fiber.ts:625-640). Their own note calls it "correct, if heavy" (2026-07-23-client-plugin-loading-model.md:95). - **No in-memory state survives a reload.** The fiber is fresh, and so is React state (same note, :97). State that must last goes into the session event log or the storage domains. - **No mixed versions.** A reload batch is all-or-nothing: if activation fails, the old modules and plugins are restored (packages/boot/hmr/src/index.ts:678-720). - **One Node host process,** plus browser clients that call `@Remote` methods. Ids are resolved to host objects (`agentId` → `Agent`) by lookups (docs/api-gateway.md:11). There are no remote objects and no principals beyond the owner fence. - **`restart-required` exists:** package version changes and bootstrap modules need a restart (packages/boot/plugin-manager/README.md:52, :140). What this buys them: none of our replace ordering, `deps` handover, stale handles, "read `.value` at use", derive rebinding or rollout coverage. Those cases simply never happen. ### 3. Code changes and reload DSH does reload in place (Cordis HMR on the host, a `client-hmr` plugin in the browser). It does not swap code under live state. A reload means: 1. Dispose the plugin's fibers and wait for them to settle (hmr/src/index.ts:686-687). 2. Plug in the new module with the same config, which cascades to every dependent. 3. On failure, roll back. The costs are accepted openly: - Reloading the agent-loop provider "stops and drains every live handle it made" (packages/core/agent/README.md:117). Sessions are later resumed from the log, with interrupted turns closed. - Reloading the persistence backend makes writes "fail loudly until the sessions restart" (2026-08-27-handle-based-session-persistence.md:35). - Reloading the terminal service closes every terminal (terminal README:81). - Running job and tool closures keep their old code until they settle. This is the same as our jobs. ### 4. State, persistence, background work, prompts - **State:** - The append-only session log is the only durable truth; projections are derived from it (docs/architecture.md:118-126). - Non-session data lives in versioned storage domains (packages/storage/README.md:12). - Format changes are adjacent migrations that never overwrite a committed generation. - **Background work:** - Jobs are in-process and "die with the harness process" (jobs README:55). The README admits the contract cannot go cross-process (:126). - Schedules are durable records. After downtime only the latest due occurrence is delivered (docs/subsystems/schedule.md:146). - **Automatic work does not resume by itself.** Goal "activation" is kept out of replay, so resume or fork needs a human action first (docs/glossary.md:27). - **Questions and approvals:** - While the tool call is open, the call itself is the question. After the call returned or the process died, the question is "continued" and an answer comes in as a new message (packages/interaction/user-questions/README.md:35). - No pending promise needs to survive anything. - Approval fails closed: `unavailable` counts as a denial (docs/subsystems/approval.md:20). ### Comparison | Topic | Chord target | DSH | |---|---|---| | Native holder | resource in any object | owner service only | | Plugin holds | `.value` (the native) | id plus methods | | Native owner | the object that opened it | explicit `Agent` scope | | Code change | keep / replace / reopen | restart | | Dependents | `deps` graph | restart (epoch) | | Old code | kept natives, stale handles | only in-flight closures | | Mixed versions | per facet | none (rollback) | | Reconnect | routine | inside the supervisor | | Pending question | resource store | open call, else a message | | Hosts | many, rollout | one process | | Gives up | nothing (complex) | kills live state on provider reload | ### 5. Which ideas remove which of our problems - **Natives only in owner services; plugins hold ids.** Removes "old code inside kept natives" and "natives shared across plugins": sharing is a served type, which is already our rule. It also removes the reviewers' worry about plugin class instances holding behaviour: those are code, rebuilt on reload, not natives. - **Explicit owner at spawn** (`spawn(spec, { owner })`). Removes "natives born in requests": a request-born native belongs to the calling object identity or to a job, never to a code generation. It also gives keep-alive a single source: the host counts live natives per owner (DSH: `hasOwnerActivity`, terminal README:85; the jobs registry reporting session activity, jobs README:83). - **Supervisor with generations.** Removes "reconnecting natives" from plugin code; consumers read a status readable. - **Restart instead of replace.** Removes "replace and dependents" entirely. In our world the cascade already exists: a reference ends with an error and is re-held (§6). Dependents become references, the one part the owner calls footgun-free, instead of `deps`. - **All-or-nothing per host with rollback.** Removes "mixed versions within a host". - **Owner services in runtime code.** If the generic supervisors (process, PTY, socket or connection, SQLite) live in the base bundle as host values, a plugin reload never touches a native, and a change to a supervisor is a host restart, as base changes already are. - **Question = open call; after loss = input.** Removes the Questions store as a kept resource with `replace: old => old`. - **Registry readiness:** Cordis keeps consumers pending until their injected services exist, and does a startup-only audit (2026-09-09-consumer-owned-startup-strictness.md:19). This roughly matches our start barrier; no new idea here. Their registry entries are reversible effects that go away and come back on reload, so there is a gap, which we deliberately avoided. - **Durable phases across reloads:** DSH freezes each request and kills live agents on a core reload. Our immutable pinned registry tables are already better here; keep them. **Ideas that don't fit p2:** - **Cordis's in-process cascade** cannot cross hosts or principals. Our reference error plus re-hold is the remote form of the same thing. - **Killing live agents on reload** conflicts with "live reload on every host". The fix is to keep natives out of plugins, not to accept the kill. - **Owner fencing by `Agent` object identity** does not cross the wire. We need object identity plus principal. - **Single process with no rollout and `restart-required`** contradicts our goals. That is exactly where their simplicity comes from. ### Proposal: a constrained model for natives 1. **Remove `self.resource` from plugin code.** A native exists only inside a supervisor: host values from the base bundle (process, PTY, socket or connection with reconnect, SQLite or plugin database, sandbox, watch). Plugins get handles: opaque ids plus methods plus a status readable. 2. **Every native has an owner** passed at creation: an object identity or a job id. It closes when the owner ends. Keep-alive and presence count natives per owner. 3. **A plugin reload never touches natives.** It reruns factories and restarts routines. Behaviour that used to sit inside resources (terminal logic, search, radius) is plain code around a handle id, so code changes need no version bump. 4. **A plugin that truly needs its own native** (rare) serves it as an object type. A code change restarts that object, and consumers re-hold through references. No `replace`, `deps`, resource versions, ordering rules or stale handles. 5. **Per-host install is all-or-nothing** with rollback; drop "facet keeps its previous version". 6. **Pending waits (Questions, logins) belong to the asking call.** A plugin reload or crash fails the call retryably, and a re-ask with the same `key` reattaches. The questions themselves are documents; resolvers are never kept. This deletes most of §12 and part of §15, and reduces the native problem to objects plus references. The cost is a small set of supervisors in the runtime, which has to be designed well because changing it needs a restart. Any plugin native that does not fit a supervisor is restarted on reload, as DSH does. ## Additional observations and questions from the research commentary These record material produced before the final report, including qualifications that the report compressed too aggressively. They are research observations or open design questions, not owner decisions. ### Lifecycle effects and dependency cascades - Cordis is the framework associated with Koishi. Its shared context holds services; services are supplied by plugins. Plugins contribute services, events and reversible registrations. There is no need to import concrete implementations to find services. - Reloading a plugin means disposing all its fibers and waiting for quiescence, then re-plugging with the same config—essentially a kill-and-restart where natives die. Required `inject` dependencies make consumers pending until providers exist. When a provider disappears or changes fiber uid, its dependents unload and reactivate. - The epoch is just the concatenation of injected services' provider fiber uids, so when a provider reloads with a new uid, every consumer's epoch shifts and triggers a full unload-and-reload cascade through dependents—no handover or versioning needed, just a clean restart. - Tools register themselves into `ctx.tools` as registry effects. Reloading a tool plugin removes and re-adds its registration without necessarily restarting the agent loop, since the loop injects the registry, not every individual contributing plugin. Reloading a foundational service such as sessions is a different matter. - This distinction is important for registry readiness: dependency readiness is not the same thing as an immutable, gapless contribution table. Cordis restart semantics do not solve the pinned Durable phase problem. - Open investigation question raised during research: does a tool plugin reload cancel an in-flight tool call, or does that call continue on its captured definition while the registry changes? The report inferred continuation for old closures; no universal tool-specific cancellation guarantee was established by the inspected RE - apps/p2/docs/p3-reviews/durable-deferred-commits.md (10872 bytes)
# Durable: deferred storage of throttled commits Status: **proposal for review** (2026-10-24). Replaces the `pi.run`/`pi.progress` split (`durable-run-progress.md`, kept for its reviews). A pi change in `packages/durable`, upstream first. No schema change, no change for renderers. ## Problem Durable commits a streaming answer's partial (`pi.live.generation.message`) and running tool output (`pi.live` tool slots' `output`, `details`, `diagnostics`) about every 100 ms per conversation. Each commit is a full storage transaction: SQLite writes ~17.7 KB per commit (page granularity) for a ~150-byte change, and p3's store adds an outbox record per commit. With 32 streams (a real case): ~270-320 commits/s, ~4.7 MB/s of disk writes (~17 GB/hour), ~11% CPU on slayer (Fable, `/tmp/group-commit-fable.md`; Astra, `/tmp/drp-review-astra.md`). CPU is mostly the Session's in-memory work, which stays; disk and outbox are what this proposal reduces. Group commit was measured and rejected (2-4x less disk only at >= 8 streams, no CPU gain, a few ms latency on every commit, early adoption needed). ## Idea Only the two throttles (the generation's partial throttle in `harness/generation.ts`, the tool output throttle in `harness/tool.ts`) mark their commits **deferrable**. Such a commit is adopted in memory and published to watchers at once, as today, but not stored: its ops wait, folded, in a `PendingOps` per document (Chord's `@earendil-works/chord/delta`, upstream `d8d99950a`). They are stored by the first of: 1. the next **normal** commit that writes the **same document** (it stores the held ops first, in its own storage transaction); 2. a Session **timer**, at most ~1 s after the first held write (one storage transaction for every held document); 3. `close()`. A deferrable commit that writes anything else (an entry, a task, another document, a document creation, migration or retirement) is stored at once: the mark is a permission, not an order. ## Why storage stays consistent - Held ops belong to one document, `pi.live`, and only to its streaming fields. - Everything that consumes or clears streamed content writes `pi.live` in the same commit: converting a partial into an aborted entry deletes `generation`; ending an answer clears the partial and sets `tools`; settling a tool builds its result from the slot's output and sets the slot `done`; steering, run start and run end change `run`. So each of these commits stores the held ops first, together with its own changes. - Commits that do not write `pi.live` (a task's own state, a display-only entry, a submission record, another document) do not depend on the partial; they need not flush it. - Guarantee: every document in storage is a state it really had; every commit that writes a held document stores it together with its own changes. Interleaving is low: control changes of `pi.live` happen at phase boundaries (run start, answer end, tool start and end, run end); streaming happens in between. - Order: held ops are older than the normal commit that stores them, so they come first in the stored delta; with folding, a later write of the same field absorbs the held one (deleting `generation` drops the held appends). ## Adoption, sequence numbers, publication All on the Session's line (one commit at a time), as today. 1. **A held commit:** memory adoption only (the tracker's pointer swap and ops), no change to stored-revision counters (`storedVersion`, `deltasSinceBase`) or record metadata; its ops go into the document's `PendingOps`; the document is pinned in the Session's cache until stored (no idle eviction, also with retention 0). Publication is tagged: ```ts type CommitPublication = | { readonly kind: "stored"; readonly seq: Seq; readonly changes: readonly CommitChange[] } // as today | { readonly kind: "memory"; readonly after: Seq; readonly changes: readonly DocumentCommitChange[] }; // held ``` A held commit only ever contains document changes, hence `DocumentCommitChange`; `after` is the last stored seq. The view mount and document watches apply its ops as usual; listeners that care about durability (replication, anything keyed by seq) skip `memory`. 2. **A normal commit writing a document with held ops:** `take()` the held ops, fold this commit's ops for that document onto them, one `document.change` write; storage commits as today; `adopt(seq)` adopts this commit's own ops in memory and updates the stored counters for the whole delta; the publication carries only this commit's own ops (the held ones were published already); the pin is released. 3. **The timer or `close()`:** a store-only commit on the line, one `document.change` per held document, one storage transaction; adoption updates stored counters only (no memory change, no publication). It takes a seq like any storage write; listeners see a gap, which Durable allows (it compares seqs, never counts them). So `Transaction.adopt` splits into adopting in memory and recording what was stored; today's path runs both, a held commit only the first, a flush only the second. ## Restrictions and edges - The option applies only to documents with `history: "latest"` and `fork: "initial"` (`pi.live` qualifies): "as of" reads and forks of the current state read storage by seq and would miss held ops. - Creation, migration and retirement of a document are never held. - Checkpoints: `checkpointWhen` counts stored deltas; a base write includes everything held. - The timer is a foreground timer (Workers refuse background timers), its deadline from the first held write; a failed flush fails the Session (as any storage failure). - Other readers of storage (p3's outbox and replicas, an export, a second process) lag up to ~1 s for held fields; an export that must include them flushes first. ## Crash and resume After a crash or a Cloudflare eviction, held ops are lost with the process; storage has `pi.live` with control as of its last normal commit and streaming fields as of the last flush (up to ~1 s older, or absent if the stream started less than a second before). | What was running | Resume | Difference from today | | --- | --- | --- | | generation streaming | converts the stored partial into an aborted entry, clears it (normal commit), resends | the aborted entry up to ~1 s shorter; none if the stream was under 1 s old (today: 100 ms); its usage up to ~1 s low | | deferred response being polled | continues from the task's checkpoint | none | | replay-safe tool | reruns, progress cleared by a normal commit | none | | tool without replay | ends `interrupted` with the stored output | output up to ~1 s shorter (accepted lossy recovery input) | | compaction | reruns from its checkpoint | none | | clean `close()` | stores everything held first | none | | restart counter (#11139) | partial commits never counted as progress | none | At open nothing is held; every resume step that consumes progress is a normal commit writing `pi.live`, so a second crash cannot convert the same partial twice. ## Expected effect 32 streams: storage transactions for partials and tool output drop from ~300/s to at most one per conversation per phase boundary or per second; disk and outbox accordingly (estimate: about 10x less; to be measured). CPU about the same (the 100 ms in-memory work stays). One stream: ~10x fewer storage transactions for streaming. ## Changelog Durable's `CHANGELOG.md`: the tagged `CommitPublication` is a breaking change for `subscribeCommits` consumers (a migration note: skip or handle `kind: "memory"`); the crash window for streamed content grows from ~100 ms to ~1 s. ## What the reviewers check (against Durable's code, in full) 1. The consistency argument: is there any commit that stores data derived from `pi.live`'s streaming fields without writing `pi.live` in the same commit? Any reader of storage (scheduler, resume, inspect, forks, copies, events, view builds after eviction) that would see an inconsistent combination? 2. The adoption split, the tagged publication, the flush path, pinning, the timer, `close`, failures: feasible as described in `session.ts`, `transaction.ts`, the documents and storages? Every consumer of `CommitPublication` and `seq`. 3. Resume: every path in the table, plus double crashes. 4. Interleaving: measure on a real Durable conversation with tools (parallel tools included) how many normal commits write `pi.live` per second against held ones; and the disk, outbox and CPU effect at 1 and 32 streams. 5. Size of the change and anything simpler. ## Decided (owner, 2026-10-24), after the reviews (`/tmp/ddc-review-{astra,fable,opus}.md`) - **Spec change accepted:** Durable's spec §1 ("all visible progress is durable; no volatile publication path") and the §13 non-goal change: a commit may be marked deferrable; what watchers see can be up to its window ahead of storage. A stream that crashes before its first flush leaves no aborted entry, and that attempt's usage is not recorded. - **Public option, per commit, tagged:** `commit(change, context, { persistence: { kind: "deferred", withinMs? } })`; without `withinMs` the default is `settings.progress.persistWithinMs` (1,000); without the option, immediate. No internal flag: any task (built-in or third-party) may use it. The two throttles (generation partials, tool output) use it. - **A deferrable commit is held only** when every write changes an existing document defined with `history: "latest"` and `fork: "initial"`; creation, migration and retirement of a document are never held; anything else in the commit makes it immediate (the mark is a permission). - **Stored by the first of:** a normal commit that **reads (`tx.doc`) or writes** the document (held ops first, in its storage transaction); the document's deadline (the earliest `withinMs` deadline of its held commits), on a **foreground** timer through the injected `setTimer`; `close()` (after `writeAtClose`, before documents are cleared, in `try/finally`); and before a tracker with held ops is dropped (`unloadDocuments`, a version-mismatch reload) or its document retired. - **Held ops leave the queue only at the last step before `Storage.commit`** (a failure earlier loses nothing); a failed flush fails the Session; the `PendingOps` overflow (10,000 ops) stores a base. - **Pinning:** a document with held ops is not evicted. - **Publications:** held commits are delivered to internal observers (`observeCommits`: the view mount, document watches; tasks and submissions ignore them) tagged memory-only; `subscribeCommits` stays durable-only and carries exactly what was stored (no breaking type change for it). - **Changelog:** a Breaking/Changed entry for the spec change with a migration note (opt-in; what a crash loses). - apps/p2/docs/p3-reviews/durable-run-progress.md (8797 bytes)
# Durable: split `pi.live` into `pi.run` and `pi.progress`, with deferred persistence for progress Status: **proposal, owner wants it (2026-10-23), pending a full review by Astra and Fable**. A breaking change in pi's Durable (`packages/durable`), upstream first; pairs with pi #11327 (windowed view, `runFailure`, removing `watchEvents`) and #11139 (a run cut off by repeated crashes ends visibly). ## Problem Durable commits a streaming answer's partial every ~100 ms per conversation into `pi.live`, a persisted document. Each partial commit is a full durable write: a storage transaction, a delta row, an outbox record (p3's store syncs it to Radius). With 32 subagents streaming in one session (a real case) that is ~320 durable commits per second, almost all overwritten 100 ms later. Measured by `rqlite-eval` (`/tmp/rqlite-eval.md`, real Durable Session on SQLite, 32 streams): 100 ms persistence 9.8% CPU, 5.6 MB/s disk (20 GB/hour), 79 KB/s outbox raw; 1 s persistence 1.6% CPU, 0.6 MB/s, 16 KB/s. Per partial commit ~640 us, ~310 us of it the SQLite transaction (`is-9265`'s profile). Persisting the partial less often is not possible today, because `pi.live` mixes two kinds of fields: control that Durable itself decides with (`run`, tool slot identity and status, compaction status) and fields only clients display (the partial message, tool output, backoff times). Deferring writes of a mixed document either risks losing control state in a crash (trace: the final commit deletes `run` and settles the submissions; if `pi.live`'s part were lost, the resumed host sees `run` still set, `submissions.ts` treats the conversation as busy, nothing ends that run) or needs a flush rule that, applied session-wide, would flush ~10 times per second with 32 streams and win nothing. ## Proposal ### Two documents, every field in exactly one ```ts /** pi.run: what Durable decides with. Persisted with every commit, as today. */ type RunState = { run?: { taskId: TaskId; inputs: SubmissionId[] }; // present exactly while busy runFailure?: { taskId: TaskId; reason: string; error: string }; // why the last run ended unanswered (#11327, #11139) generation?: { attempt: number }; // an attempt is running tools?: { callId: string; name: string; status: "pending" | "running" | "done"; taskId?: TaskId; entry?: EntryId }[]; nestedTools?: { callId: string; name: string; status: "pending" | "running" | "done"; parentCallId: string; parentTaskId: TaskId; taskId: TaskId; arguments: JsonObject; // what the call runs with summary?: { isError: boolean; durationMs?: number; usage?: Usage; error?: string }; }[]; compactions?: { taskId: TaskId; reason: CompactionReason; blocking: boolean; attempt: number }[]; }; /** pi.progress: for display only. Persisted deferred (at most once a second). */ type ProgressState = { message?: AssistantMessage; // the partial answer retry?: { at: number; error: string }; // generation backoff deferred?: { pollAt: number }; // a deferred response being polled calls: { [taskId: string]: { // model-issued and nested calls, by tool task id output?: string; droppedBytes?: number; droppedLines?: number; details?: JsonValue; diagnostics?: ToolDiagnostic[]; } }; compactions: { [taskId: string]: { retry?: { at: number; error: string } } }; compactionFailure?: { taskId: TaskId; reason: CompactionReason; error: string }; }; ``` - Progress is keyed by **task id**, not call id: a nested call's id is `<parentCallId>/<key>` and a model-issued id from OpenAI Responses (`call_id|item_id`) may contain `/`, so call ids are not unique across both kinds by construction; task ids are (minted per session), and progress only exists while a call runs, when it has its task. - Renderers read both from one view value and join by task id; nothing is duplicated, so nothing can disagree. - `pi.live` disappears (the name meant "live state"; `pi.run` is control, `pi.progress` display). ### Deferred persistence: a generic document option ```ts export const ProgressDoc = defineDoc<ProgressState>({ kind: "pi.progress", …, persist: { deferMs: 1000 } }); ``` The Session knows nothing about `pi.progress`; its rule is generic: - Every commit is adopted and published at once, as today (watchers and the view see every 100 ms update). - Writes to documents whose definition has `persist.deferMs` are **held** per document, their ops folded (the leaf index of Chord's pending queue: appends to one text join, sets replace sets); all other writes go to storage in the commit's storage transaction as today. - One Session timer stores every held document's folded ops in one storage transaction, at most `deferMs` after the first held write. A commit that writes a held document anyway may carry its held ops in the same transaction (optional; storage is fresher at no extra transaction). - `close()` stores everything held. A base write of a held document (checkpoint, retirement) includes its held ops. - No commit ever has to store another document's held writes for consistency: held writes touch only their own document, so storing them later gives the same stored state. ### Crash and resume - Control (`pi.run`, entries, tasks, submissions) is always stored with its commit. - `pi.progress` in storage is up to ~1 s behind, or absent if the host died within the first second of a stream. - Resume converts the stored partial (`pi.progress.message`) into an aborted entry as today; if it is absent, there is no aborted entry and the request is resent anyway. ### Migration of existing sessions - `pi.run` is created from an existing `pi.live` (its control fields) when a conversation is first opened by the new version; `pi.live` is retired. - Every view-only field is cleared when a run ends (`endRun` deletes `generation`, `tools`, `nestedTools`), so a session at rest loses nothing; `pi.progress` starts empty and is created by its first write. - A session upgraded in the middle of a run (it crashed or stopped mid-stream and resumes on the new version) loses its old partial preview and tool output; resume may read the old `generation.message` once to write the aborted entry. - Clients reading `pi.live` break (public protocol, spec §9.3): p2's renderers, aron-cf's integration. Accepted (owner). ### Changelog: breaking change with a migration guide (owner) Durable's `CHANGELOG.md` gets an entry under `### Breaking Changes` with a migration guide inside it: what moved where (a field table `pi.live.<field>` -> `pi.run.<field>` or `pi.progress.<field>`), how a renderer joins tool slots with their progress by task id (code before and after), what the view exposes (`docs["pi.run"]`, `docs["pi.progress"]`), what happens to stored sessions on first open (and mid-run), the new `persist: { deferMs }` option for extensions' own documents, and what a crash now loses (up to ~1 s of progress, or the whole progress of a stream's first second). ## What the reviewers check (in full, against Durable's code) 1. Every reader and writer of each `pi.live` field (`generation.ts`, `tool.ts`, `compaction.ts`, `live.ts`, `submissions.ts`, `events.ts`, `view.ts`, `inspect()`, the task graph, tests): is each field placed right (control vs display)? In particular: does Durable read `generation.retry`, `generation.deferred`, a compaction's `retry` or a tool slot's `output`/`details` itself (resume, scheduling, replay)? Then it is control and stays in `pi.run`. 2. The Session mechanics: adoption and publication of a held write without a storage seq (today adoption is driven by storage's `adopt(seq)` callback and every publication carries a seq); the documents engine accepting one stored delta that covers several in-memory revisions; checkpoints (`checkpointWhen`), the "complete base whenever nothing runs" rule of `pi.live`; retirement; `close`, eviction and `writeAtClose`; `StoreStorage` and the outbox (p3's store). 3. Crash and resume: every resume path that reads progress (`convertPartial`, tool replay clearing progress, compaction retries), and what an absent `pi.progress` means for each. 4. Migration: creating `pi.run` from `pi.live` on open (where, when, atomically), mid-run upgrades, forks (`fork: "initial"` today), copies (`document.copy`). 5. The view (#11327 and today's mount: `MOUNTED` documents), `watchEvents` until it is removed, and p2's renderers. 6. Numbers: confirm the win at 32 streams with the real code path (commits, CPU, disk, outbox), and what remains. 7. Anything simpler that gets the same result. - apps/p2/docs/p3-reviews/pl-agent.md (40996 bytes)
# pl-agent — models, prompt, code mode, agent, logs, developer tools, PowerShell, web and TUI design review Status labels in this document: - **DECIDED**: explicitly decided by the owner in `redesign-decisions.md` or its porting guidelines. - **RESOLVED / SUPERSEDED**: the later p3/Chord v2 design removes the reviewed mechanism or adopts the recommendation. - **OPEN**: explicitly listed as open in `redesign-decisions.md`, or still not answered by the decisions known to this reviewer. - **RECOMMENDATION**: proposed by this review, not known to be accepted. This records the design findings and recommendations from two rounds: the review of `target-design.md` at `6559589bd`, then the ownership/native-object review after `redesign-decisions.md` at `5abe3f6d6`. It intentionally excludes the later functionality map. ## Round 1: adversarial review of the original target design The reviewed plugins were models, prompt, codemode, coding-tools, agent, web-shell, logs, dev-tools and powershell, including their interactions with login, sessions, chat, subagents, environments, Durable, web and TUI runtime services. ### Summary verdict and cost **SUPERSEDED estimate, still useful as scope evidence.** The initial estimate was about 16 files and 650–850 changed lines. models was the largest part (about 300 lines: `resources.ts` restructured, methods replacing the opaque member, derives). Then prompt (about 150), logs (about 80), codemode (about 50), powershell (about 40), dev-tools (about 30), and agent, coding-tools, web-shell and the TUI/web registrations (about 10–20 each). Login and sessions added about 100 lines for removing models' opaque native, and subagents about 40 lines for removing the shared document write. The original verdict was: > **The design can express these plugins, with conditions.** Findings 1, 4 and 6 are API gaps (logic refresh of kept natives, overlap policy, rebinding derives over native change sources, whole-value replacement); without them, reloads of models, prompt and logs succeed without changing behaviour. Findings 2, 3, 5 and 7 are lifecycle and cross-plugin cases that need small additions: a close reason, deferred client releases, rebuilding dependent facets on reload, and handing over the credential locks. The later p3 design deliberately replaces rather than migrates p2, and replaces resources with fixed-lifetime native objects, so the implementation estimate and several mechanisms are superseded. The concrete traces remain relevant where noted below. ### 1. Kept resources silently keep old plugin logic **Status: RESOLVED by the decision that native objects restart on plugin reload; the underlying restart cost moved to later findings.** > A kept resource keeps running the code that opened it, so a reload reports `installed` and changes nothing. Concrete traces: - models: `Resources` (`models/resources.ts:41`) wires its watcher to its own class method `load` (`resources.ts:67-79`). `load` calls `applyModelConfig`, which builds providers and patches `models.getAuth` with closures from that code (`provider-config.ts:673-691`). `FileCredentials` and `FileModelsStore` are plugin classes too. - Trace: fix a bug in `provider-config.ts`, then reload models. The resource keeps its version, so it is kept. The next `models.json` change runs the old `load` with the old `applyModelConfig`. The fix only lands after a restart or a version bump. - prompt: the same problem exists through closures in `prompt/worker.ts:84-106`, `environment.ts:89-97` and `context-files.ts:64`. A fix to skill discovery never applies; even a session move reuses the old closure (`worker.ts:154-158`). - logs: the `setInterval(collect)` closure in `logs/server.ts:21-67` keeps running the first version of `collect`. - Each such resource pins its first code generation, working against loader D's generation-GC goal. - The old rule “keep logic out of the resource” could not actually be met by models: pi-ai's `MutableModels` is a native whose content is plugin closures. Original recommendation: > Add a resource mode `version: "code"`: every reload of the plugin runs the new code's `replace(old)`, which keeps the natives (file locks, watcher) and rebuilds the logic parts. Add a static check that warns when a resource's `open` returns an instance of a class defined in the plugin. That recommendation is superseded. The owner instead decided that natives are fixed for one object lifetime and native objects restart on reload. This guarantees new code but makes expensive or disruptive restart behavior explicit. ### 2. Replacing or restarting the code-mode sandbox can hang Durable work **Status: OPEN. `redesign-decisions.md` explicitly lists close-with-reason as raised and not yet discussed. The new design makes the trigger more common: every reload of the native-owning plugin restarts it.** > Replacing the codemode sandbox hangs running scripts instead of failing them. `codemode/calls.ts:17-23` records the reason: when the sandbox ends a run on its own, Durable stores no outcome, and the call waits forever. Original trace under the resource design: 1. Bump the `sandbox` resource version (`codemode/worker.ts:22`). 2. Resource replacement closes the old sandbox. 3. `calls` depends on harness and worker, not on the sandbox (`worker.ts:27-34`), so `RunningCalls.cancel` does not run. 4. The tool call never settles, the conversation remains busy, and the worker never goes idle. The p3 trace is now: 1. Reload the plugin while a code-mode script is running. 2. The native object is restarted and closes `CodemodeSandbox`. 3. Durable sees that the sandbox abort was not initiated through the invocation's own controller; its current behavior stores nothing so a future worker can replay it. 4. The current invocation can remain waiting forever unless close explicitly settles or aborts the Durable task. Related problem: - On a worker stop, code mode must **not** abort and commit a terminal outcome, because the next worker should replay the interrupted script. - On a plugin reload/removal, the running call should be deliberately aborted and settled. - Today this distinction is inferred through `worker.liveness.stopping` (`codemode/worker.ts:31-33`). Original recommendations: - Under the resource design, make `calls` depend on `sandbox`, so calls close and settle before the sandbox closes. - Pass `close(value, { reason: "replace" | "remove" | "stop" })`. - Replace the raw `setTimeout` poll in `reported` (`calls.ts:67`) with a Durable watch. Current recommendation: - Chord v2 native close needs a reason, at least `reload`, `end` and `stop`. - The code-mode native owner must abort the Durable tasks it serves on reload/end, but not on worker stop. - Define how a native owner accounts for and settles non-Chord work, because “running work in native objects is aborted” currently describes Chord work, not Durable task invocations hidden behind a plugin extension. ### 3. Client renderer reloads create false departures **Status: OPEN. `redesign-decisions.md` explicitly lists “UI renderer reload vs. leaving” and presence behavior as open.** > A web or TUI facet reload releases every reference below it, and the design reads a release as someone leaving. Concrete traces: - web-shell registers the root component (`web-shell/web.tsx:46`). Giving `App` a new component identity remounts the whole tree. Every `useObject` cleanup calls `ref.release()` (`runtime/web.tsx:177-181`). - Login: `ProviderLogin`'s viewing cleanup runs `abandon` (`login/server.ts:134-141`), so reloading the shell can kill an OAuth login in progress. - Questions: with `all` and a live audience, a reloading owner appears to leave. If everyone else has answered, the question can resolve with that owner in `missing` even though the owner did not intentionally depart. - Presence: viewer counts briefly fall and rise. - TUI: when a slot factory changes, the runtime disposes its surface (`runtime/tui.ts:473-479`). `ModelState` releases and re-holds `Chat` (`models/tui.ts:73`). - TUI notices: `NoticeWatch` resets `seen` and `first` (`logs/tui.ts:164-165`), so old notices are summarized again after reload. Recommendation: > While a client install runs, defer releases until after the next render commit, and let a re-hold of the same `(type, input)` take over the old remote hold. Neither `host.references` nor the linger bit should see this as a departure. This also needs a decision for full web reloads/TUI worker swaps, where the old and new clients are different runtime instances but represent one logical viewer origin. ### 4. `routines.on` needs explicit overlap semantics **Status: OPEN. `redesign-decisions.md` explicitly lists `routines overlap: "latest"` as open.** > `routines.on` does not say what happens when a new value arrives while a handler runs; PowerShell and prompt need the old run aborted. Concrete traces: - PowerShell probes an unreachable remote in a loop until its own “moved” signal aborts (`powershell/worker.ts:32-37,44-53`). - Prompt closes the old watcher after a location move (`prompt/worker.ts:154-160`). - A session is on unreachable remote A, then moves to a local Windows machine. If `on` queues handlers, the new handler waits behind A's endless retry, so PowerShell remains stale and is not offered. - On reload, serialized routine restart can wait behind the same handler until its deadline. - If prompt-watch logic moves into a restarted routine, its “first load done” marker cannot live in that routine. Otherwise every prompt-plugin reload makes subsequent input wait for another remote scan, up to the existing 60-second load timeout. Recommendation: > `self.routines.on(source, fn, { overlap: "latest" | "queue" })`. `latest` aborts the running handler's `ctx.signal` when a new value arrives. Durable “first load done” markers belong in state that survives routine restart. ### 5. Agent copies and writes another plugin's Durable document **Status: RESOLVED / DECIDED in the p3 porting guideline. Core-enforced conversation policy is written at child creation; plugins do not share private documents.** > agent copies another plugin's Durable document and writes to it. Concrete trace: - `agent/tool.ts:17` imports `Subagents` and `resolveChoice` from `subagents/tools.ts`, a loader-D cross-plugin copied module. - It writes `registry.oneShots` into the subagents-owned document (`agent/tool.ts:82-90`). - If subagents bumps `Subagents` to version 2 and only subagents reloads, agent's bundle still has the version-1 copy; `tools.agent` throws on the newer document. - Reverse case: agent reloads with a version-2 copy while the subagents worker facet rejects and remains at version 1. agent writes a v2 document; subagents' old code then throws on its own document, including `subagentCwdFollowers`, which relocation/start calls (`runtime/worker.ts:476-526`). The relocation fails and can leave recovery to a worker restart. Original recommendations: - Expose a host-local recorder owned by subagents, or add a Durable commit hook, so agent no longer imports/writes the document. - Rebuild every facet whose bundle metadata includes a changed cross-plugin file, even when only one plugin is named. - More generally, forbid copied cross-plugin modules from defining Durable documents or tasks unless all consumers form one reload unit. Owner decision: - Core owns per-conversation policy (`readOnly`, `followsSessionCwd`, and equivalent creation metadata). - Agent/subagents write that core format in the creating commit. - Chat and relocation enforce it directly. - `ReadOnlyConversations` and `SessionCwdFollowers` policy registries go away. The one-shot listing metadata still needs one owner; it should not return to a plugin-shared private document. ### 6. Old derives over changing natives and whole-root document writes **Status: native-source half RESOLVED by fixed-lifetime natives and native-object restart. Whole-root replacement remains UNKNOWN/OPEN unless Chord v2's document API has since specified `set`.** Original finding: > Derives over a changing native do not work as specified, and documents cannot replace their root value. Concrete traces: - models' `summary` and `snapshot` follow changes inside `Resources` (`models/server.ts:31-39`, `config.ts:51-59`). A derive over the resource handle only recomputes when the handle rebinds, not when the native changes internally. - Taking a readable out of `.value` was the old resource footgun; `questions.md` nevertheless illustrated `self.derive([pending.value.readable], …)`. After replacement, that derive could stay attached to old data silently. - PowerShell's `windows` and dev-tools' `offered` replace scalar roots. The document draft design described property assignment, `push` and `Object.assign`, but not replacement of a boolean root. - `Object.assign` is also not enough for replacement semantics because it does not delete keys absent from the next object. Original recommendations: - Derive sources shaped as `(handle, value => value.changes)`, rebound as part of replacement. - Add `handle.set(ctx, value)` and `tx.set(handle, value)` for whole-value replacement. The new native design removes replacement/rebinding of a native during one object lifetime. Native owners still need explicit source readables for internal changes. The scalar/root-document write question remains. ### 7. Models exposes opaque natives across plugin boundaries **Status: DECIDED. No natives across plugins; consumers use owner methods/state. Restart behavior during login remains OPEN as a consequence.** Original finding: > Natives cross from models to login and sessions; the design forbids that, and the replacement methods fail mid-login when models reloads. Concrete traces: - login constructs `Logins` with `ModelConfig.resources` (`login/server.ts:62`) and calls `models.login`, `refreshModels`, `modelStore.delete`, `reload` and `publish` (`login/logins.ts:41,130-146,199-206`). - sessions reaches through the same opaque resource to validate a model (`sessions/server.ts:223`). - `HarnessConfig` similarly returns a native `Models` object to core Harness, chat, agent and subagents. - Under the old replacement design, an OAuth flow waiting for the browser could lose its credential write when models' old `Resources.close()` closed `FileCredentials` and its lock (`resources.ts:287-295`). Original recommendation: - Replace the opaque member with methods: `hasModel`, `setDefault`, `setEnabledModels`, login/logout/refresh and related operations. - Use callbacks only for local methods such as provider prompts/notices. - Hand locked files over during replacement, and resolve the current credential store at write time. - Clarify whether host-owned registries such as `HarnessConfig` are an allowed native bridge; otherwise move model runtime into core. Owner decisions now known: - No natives across plugins. - Local methods may take callbacks. - - apps/p2/docs/p3-reviews/pl-chat.md (45082 bytes)
# pl-chat: target design, reload, Chord v2 and p3 reviews This records the reviews, findings, questions and recommendations produced in this conversation, excluding the functionality map. The two review reports below are preserved verbatim apart from their heading levels. They are historical reports, not a claim that every proposal still applies. Additional review points from the discussion follow them. ## Decision and open-status index Known owner decisions come from `packages/chord/docs/decisions.md` at `5abe3f6d6`. Proposed object boundaries and fixes below are recommendations, not owner decisions. | Point | Known status | |---|---| | Build p3 next to p2, isolated data, no legacy migration; Chord v2 inside p3 initially | Owner decided | | Pure facet, natives only in served objects through `self.open`; no resources, replace or deps | Owner decided; supersedes the first report's resource proposals | | Native objects restart and abort running work on reload; expensive natives split into separate plugins | Owner decided | | Whole-host rollback for load/staging errors, including live-object factories | Owner decided; resolves first report F1's setup-throw version mix | | Named derive with its previous value retained across factory re-runs | Owner decided; resolves F3's missing mechanism | | Jobs retain intent/run/finish and add awaitable `outcome(ctx)` | Owner decided; job lifetime and dedupe-retention questions remain open | | Ownership principle; methods return data, objects passed as addresses; no cross-plugin natives | Owner decided | | Conversation policy written in the creating commit; remove external read-only/cwd-following policy registries | Owner decided | | Registries remain behaviour extension points, including send routers and input gates | Owner decided | | Single-shot Durable watches | Mechanism changes resolve retained-watch reuse; proposed split into a separate feed object/plugin remains a recommendation | | Draft live representation, withdrawal/restoration atomicity and retention | Open recommendations; no known owner decision adopting this review's particular design | | SessionDeleted, file formats, optional references, UI renderer reload versus leaving, shared Durable docs | Explicitly open in redesign-decisions.md | | Keep-alive calculated by worker; presence/link-drop/linger details | Explicitly open in redesign-decisions.md | | Proposed Catalog, Session, Conversation, Composer, Downloads and Artifacts boundaries | Review recommendations, not adopted decisions | | Cross-owner chat digest concurrency, byte transfer overhead, relocation environment addresses | Open review questions/recommendations | | Precise migration estimates in report 1 | Historical only: superseded by the decision to build p3 without legacy migration | ## Original review against target-design.md and plugin-loading.md, 6559589bd I read target-design.md, plugin-loading.md and questions.md in full on origin/private `6559589bd`, all 31 files of the six plugins, and the runtime they use (`worker.ts`, `durable.ts`, `commits.ts`, `contribute.ts`, `chord.ts`, `tui.ts`). I also checked pi-durable's `WatchHandle`. Nothing was edited; the only scratch files are under /tmp/plrev. **Verdict:** the design holds for these six plugins, but five gaps need fixing first. conversations, send-file and the artifacts server, web and TUI facets port almost unchanged. The problems are all in chat, chat-mode, drafts and the artifacts worker. ### Sketches in the new design ```ts // chat/worker.ts export default defineFacet((host, self) => { const harness = host.value(HarnessKey), envs = host.value(EnvironmentsKey), liveness = host.value(LivenessKey); host.serve(Conversation, ({ conversation }, _p, self) => { const id = conversation as ConversationId; const conv = self.resource("conversation", { version: 1, open: async ({ signal }) => (await harness.conversation(id, durableContext(signal))) ?? fail(`Unknown ${id}`) }); const view = self.source("view", { deps: [conv], // MISSING API, F2 open: ({ signal }) => watchSource(conv.value.watch(durableContext(signal))) }); const window = self.derive([view], (v, prev) => v && windowView(v, prev)); // needs prev, F3 const model = self.derive([view, host.registry(HarnessConfig).table], (v, t) => modelMeta(v, t)); const last = self.state(LastShell); // singleton: {id, data} | null const shell = self.job(ShellJob, { concurrency: 1, run: async ({ command, includeInContext, author }, ctx) => { const agent = await conv.value.agent(durableContext(ctx)); const out = new OutputBuffer({ maxBytes: SHELL_MAX_BYTES, maxLines: SHELL_MAX_LINES, retain: "tail" }); const r = await envs.at(agent.cwd).exec(command, { ...opts, onOutput: (t, _c, i) => { out.push(t, i.skipped); ctx.progress({ output: out.snapshot(), truncated: out.truncated }); } }, durableContext(ctx.signal)); const data = shellData(command, author, includeInContext, out, r, ctx.signal.aborted); await conv.value.submit({ type: "write", entry: shellEntry(data), requestId: `${author.id}:shell:${ctx.jobId}` }, durableContext(ctx)); // admission only, no .wait (F4) return data; }, finish: async (done, tx) => { (await tx.draft(last)).value = { id: done.id, data: done.ok ? done.result : errorData(done) }; }, }); self.routines.on(view, (v, ctx) => last.value && placed(v, last.value.id) && last.change(ctx, (d) => { d.value = null; })); return { view, window, model, shell: self.derive([shell.running, last], liveShellOf), // ShellState | null, as Chat v5 runShell: async ({ requestId, author, ...a }, ctx) => { for (const gate of host.registry(InputGates).table.value.values()) await gate(ctx); return { shellId: (await shell.start({ ...a, author }, ctx, { key: `${author.id}:${requestId}` })).id }; }, abortShell: ({ shellId }) => ({ aborted: shell.abort(shellId) }), submit, write, stop, dequeue, dequeueOwn, history, configure, compact, reset, abortCompaction, costs, // unchanged }; }, { linger: 30_000 }); host.serve(Chat, (input, principal, self) => { /* as today, minus the "viewer" resource (-> host.references); call = liveness.call */ }); host.serve(ChatCosts, /* unchanged */); }); // chat/contract.ts: SendRouters / TranscriptEntries get { version: 1 }; Conversation unchanged. // chat/web.tsx: self.register(Slots, "main", Main) // chat/tui.ts: self.register(TuiSlots, "chat", (rt) => new ChatSurface(rt)); // for (const [k, c] of Object.entries(CHAT_COMMANDS)) self.register(TuiCommands, k, c); // ChatSurfaceKey = defineSurface<ChatApi>("chat") (14.3) ``` ```ts // chat-mode/worker.ts export default defineFacet((host, self) => { const harness = host.value(HarnessKey); host.serve(ChatModeSetting, ({ conversation }, _p, self) => { const doc = self.source("doc", { open: ({ signal }) => watchSource(harness.watchDoc(ChatModeDoc, conversation, durableContext(signal))) }); // replaces CommitSignal + task const view = self.get(Conversation, { conversation }).view; return { enabled: self.derive([doc], (d) => d?.enabled ?? false), waiting: self.derive([view], waiting), // once per conversation, F8 set: ({ enabled }, ctx) => commitChatMode(harness, conversation, enabled, ctx), // Durable doc, as today }; }, { linger: 30_000 }); self.register(SendRouters, "chat-mode", routeChatMode); // body unchanged host.serve(ChatMode, (i, p, self) => { const s = self.get(ChatModeSetting, i); return isOwner(p) ? { variant: "owner", enabled: s.enabled, waiting: s.waiting, set: s.set } : { variant: "reader", enabled: s.enabled, waiting: s.waiting }; }); }); // web: self.register(Slots, "composer:chat-mode", ChatModeChip); self.register(TranscriptEntries, k, v) for each display // tui: same registrations; chat.setCanReply via ctx.surface(ChatSurfaceKey) // drafts/worker.ts: one persisted singleton (F5), not keyed const DraftsDoc = defineState("drafts", { version: 1, persist: true, schema: Type.Object({ next: Type.Number(), byKey: DraftsByKeySchema }), initial: () => ({ next: 1, byKey: {} }) }); host.serve(DraftStore, (_i, _p, self) => { const doc = self.state(DraftsDoc); return { drafts: self.derive([doc], (d) => d.byKey), add: ({ conversation, items }, ctx) => self.tx(ctx, async (tx) => { const d = await tx.draft(doc); for (const { author, text } of items) (d.byKey[draftKey(conversation, author.id)] ??= []).push({ id: d.next++, text }); }), claim: ({ conversation, authorId, ids }, ctx) => self.tx(ctx, async (tx) => { /* filter, delete key when empty, return texts */ }), }; }, { lifetime: "host" }); // Drafts (per author): self.derive([store.drafts], (all) => all[key] ?? []) unchanged // conversations: server unchanged (Hosts -> host value); worker: self.register(Extensions, "p2-conversations", extension); // PeerConversation uses host.value(LivenessKey).call; web: self.register(Slots, "chat-attribution", Attribution) // send-file/worker: self.register(Extensions, "p2-downloads", createSendFileExtension(host.value(WorkerInitKey).dataDir, session)) // (the template-plus-late-execute trick goes away); SentFiles unchanged; tui: renderers + command, tool views via ChatSurfaceKey // artifacts/worker: Extensions as send-file; ArtifactList: self.derive([self.source("doc", watchDoc(ArtifactDoc, id))], d => d?.artifacts ?? []) // artifacts/server: const dir = host.value(ServerInfoKey).dataDir; // self.register(Routes, "GET /artifact", serve); self.register(Routes, "HEAD /artifact", serve) // (the "artifacts" resource is not a native: drop it) ``` ### Findings, most severe first **F1. Two plugins on different versions in one host can write wrong history; I challenge the decision on this.** The decision assumes "mixed versions within a host are the same situation as during a rollout". That is not true for `local` types: they never cross hosts, and they are not validated. - Trace: commit `e16667441` added `maxImages` to `Conversation.submit` and chat-mode's use of it in one change, with no version bump (`chat/contract.ts:170`, `chat-mode/digest.ts:173`). Commit `79c5e20d5` added `entry` the same way (`contract.ts:166`). - Suppose chat's worker setup throws on such a reload. Chat keeps its old code; chat-mode moves on, and its router calls the old `Conversation.submit` with `entry: chat.mention` (`chat-mode/worker.ts:164`). - The coverage check admits it: an extra optional property is "covered" by an open TypeBox object. Old `submit` drops `entry` and writes a `p2.input` entry whose text is the digest, without `included`. - Durable entries are permanent. `unmentioned` (`digest.ts:68-84`) never finds a boundary again, so every later mention re-quotes up to 100 messages. With `maxImages` dropped, every image of the digest is inlined. - Fix: for a facet setup that throws, also reject the facets that hold that plugin's local types in the same revision on that host. Alternatively, make the coverage check require the provider to declare every property the caller's arguments declare, so this fails with `contract_mismatch` instead of being dropped silently. **F2. There is no declaration for a source backed by a subscription, and Durable watches work only once.** - A Durable `WatchHandle` has one `start` and a terminal `stop` (`durable/src/types.ts:902-904`). `watchSource` allows one subscription (`runtime/durable.ts:112-121`). - Chat holds the watch as a resource and subscribes to it (`chat/worker.ts:158-176`). Under §12/§15, a reload keeps the resource (same version) and re-runs the factory and routines. - The new subscription throws "supports one subscription", or the old unsubscribe has already called `watch.stop()`. Either way every transcript freezes after a chat reload. - The pattern `CommitSignal` uses in chat-mode and artifacts (`chat-mode/worker.ts:88`, `artifacts/worker.ts:62`) attaches a handler inside resource `open`, which 14.5 forbids. §8 names "source readables via derive" but gives no API for one. - Fix: add `self.source(name, { deps, open })`. It acquires the native per subscription and is restarted with routines. The derive keeps its last value until the new source delivers. Add a p2 `watchDoc` helper on top of it. **F3. Derives have no previous value, and a reload resets the chat window for every viewer.** - `windowStart` keeps the previous start within a 1.5× slack (`chat/window.ts:32-44`, using the `shown` memo at `worker.ts:167`). That is why new entries are small appends rather than a window shift on every commit. - A pure `derive(sources, fn)` cannot express this. A closure memo is lost on every factory re-run, and so is the module-level `entrySizes` cache (`window.ts:17`). - After a reload, each live conversation re-stringifies its whole active range (MBs on long sessions), possibly inside the one-tick publish. Every viewer then gets a window that may have moved, up to 512 KiB each. - Fix: `self.derive(sources, (values, previous) => …)`, keyed by derive name so `previous` survives factory re-runs. **F4. Shell as a job: object lifetime, output display and deduplication are unspecified.** - **Lifetime:** `Conversation` is a referenced object with `linger: 30_000` (`chat/worker.ts:505`). Today, closing the object aborts the shell (`:186-195`). The design does not say whether a running job keeps its object alive. If it does not, `finish` has no object once the last tab has been closed for 2 min plus 30 s. - **Display:** today the finished output stays visible until the `p2.shell` entry is placed (`:290-322`). When the agent is busy, that happens at the run's next boundary, which can be minutes away. If `run` waits for placement, `concurrency: 1` refuses the next `!cmd` with SHELL_BUSY. If it does not wait (§10's example), the output disappears until the entry appears. - **Deduplication:** today the dedupe map keeps 100 keys (`:333-341`). The design does not say how long a finished run's key is kept, and §21 asks for "shell dedupe survives". - Fix: running or finishing runs count as a reference to their object. Add `dedupe: { keep: N }` to `defineJob`. `liveShell` derives from `running` plus a singleton `LastShell` document, which a routine on `view` clears once the entry is placed (sketch above). **F5. The drafts plan in §21 contradicts §7, and Stop is not atomic across Durable and Chord.** - §21 says drafts are "persisted, keyed `conversation:authorId`". But keyed documents have no live readable, and `Drafts.drafts` must be live: web (`chat/web.tsx:362-370`), TUI (`tui.ts:340-348`), derive (`drafts/worker.ts:49`). - Stop - apps/p2/docs/p3-reviews/plan-review4-summary.md (5316 bytes)
# Review round 4 (Fable, Astra): the full plan as of `c991bdf02`, summary for the owner Both reviewed every decision of 2026-10-21 as one design. Verdict: keep one public `WatchHandle` and one small synchronous publication mechanism; drop the `replicatedState` bridge; defer IndexedDB. Nothing below is decided. | # | Sev. | Finding | Proposed fix | Source | | --- | --- | --- | --- | --- | | W1 (decided 2026-10-21: adopt and mark in the step, one synchronous flush after) | blocker | "Derives and the wire encoder run inside the commit step" is unbuildable: the step is short and shared by every owner; a throwing derive after the backend commit cannot roll back; 30 encodes block every owner. Moving only Durable's adoption leaves its views behind (published later, `session.ts:765-818`); flushing each listener separately tears (`view` new, `graph` old). Durable's views scan every mounted conversation per commit | The step only **adopts and marks** (pointer swap, dependents dirty, ops appended by reference to each hold's batch and each watch's queue); a **flush** right after the step, before any listener or await, recomputes dirty derives in dependency order, encodes one batch per link and schedules watches; a throwing derive fails only its member (`broken`); changes dispatched once per conversation/document id; the same publication boundary for in-memory commits, registry changes and inbound wire batches (no store seq); Durable's `advance` called synchronously (no microtask), ideally inside `tx.adopt`; a `publishing` flag makes reentrant `self.tx`/`watch`/`release` throw | Fable 1, 2; Astra 1, 7 | | W2 (decided 2026-10-21: explicit retirement in Chord and Durable; routines.on runs with watch.value first) | high | Durable's watch is not generic: retirement inferred from `value === null` (`observation.ts:228,266`); `start()` never delivers the acquisition value | explicit retirement event; `routines.on` runs once with `watch.value`, then drains | Astra 2 | | W3 (decided 2026-10-21) | medium | a stopped watch keeps its listener, detach/report/replace closures and cancellation refs (`observation.ts:279`): a retained stopped handle pins the Session and plugin closures | clear on termination; GC tests count retained revisions per watch, also for stopped-but-referenced handles | Astra 5; Fable 3 | | W4 (decided 2026-10-21: two kinds, Watchable<Json> with ops and NativeWatchable) | major | `Watchable<T extends Json>` conflicts with `self.writable` and composed registries (functions, natives) | `Watchable<T>` unbounded; exposable members are `Watchable<Json>` (Q14's check); native watchables send `[["r", v]]` (owner's proposal) | Fable 5; Astra 6 | | W5 | major | design §24, Q14 and §12 still name `replicatedState` and `cell.ts` | text fix with the decided model | Fable 4 | | W6 (decided 2026-10-21: connect, load, resume) | major | worker start: `load` (barrier) before `connect`; a host-lifetime store with `self.get(Catalog, {})` (a server type) parks 60 s | connect first, then load; or forbid remote `self.get` in host-lifetime factories (use `ctx.get` in routines) | Fable 2 | | W7 (decided 2026-10-21: automatic, host kind and type only) | major | P3 address must never contain the session id, or a split or imported copy orphans every Chord document | owner strings carry the host kind and type, never an instance | Fable 3 | | W8 (decided 2026-10-21: every str() column as UTF-16 BLOB; revisit note: Armin) | high | flat columns make unindexed text queryable, but only indexed text is UTF-16 BLOB: orders differ, lone surrogates lost, adding an index changes results; `:to \|\| substr(...)` yields TEXT; BLOB offsets must be code-unit aligned | one lossless representation for every queryable text column; casts and alignment in the backend spec and conformance suite | Astra 3; Fable 8 | | W9 (decided 2026-10-21: no compaction at first) | high | "keep the last write per row" compaction loses data with `update` and bulk operations | no compaction at first: ordered operation batches | Astra 4 | | W10 (decided 2026-10-21: read_after_bulk) | major | `tx.get` overlay over pending `updateWhere`/`deleteWhere` needs a JS evaluator on every backend | after a bulk op in a transaction, `tx.get` on that table throws `read_after_bulk` | Fable 6 | | W11 (decided 2026-10-21: per-column JSON versions, row v for eager steps only) | major | migrations: a `rows` step ships one record per row (1M rows → 1M records); resuming needs durable progress | record a step as one operation when expressible in `set`; per-row only for pure JS transforms; persist step and last key atomically with each chunk | Fable 7; Astra 8 | | W12 (decided 2026-10-21) | minor | in-memory (P2) commits have no publication id or order | the owner's next publication id, never a store seq, never the commit queue | Fable 9 | | W13 (decided 2026-10-21) | minor | `mintId` over five tables; who flushes a link's batch; `count` is an aggregate | state: max over all tables at open; the flush unit (W1); keep `count` | Fable 10, 11 | | W14 (decided 2026-10-21: a, b, d cut; c kept; e done) | cut | IndexedDB backend; the `replicatedState` family as public API (Chord and Durable); `stopped` and `cancelled` as one reason; derive `previous`; per-row `rows` as the default | | Fable 4; Astra keep/cut | - apps/p2/docs/p3-reviews/rel-sol.md (12747 bytes)
# rel-sol — focused adversarial reload and rollout review Scope: `target-design.md` §15 and `plugin-loading.md` §§4.1, 7.4, 7.5, on pi-private `6559589bd`; read-only review against real p2 plugins and runtime. Assumes §§7–14 as given. This records review conclusions and recommendations, not implemented behavior. References to an "owner decision" below mean the decision is stated as decided in the target design; the proposed fixes and open questions are **not** known to have been decided by the owner. ## Verdict **§15 is not sound enough to build as written.** The clean rollout path is plausible, but the start barrier, registry admission, removal, and overlapping-install rules need tighter definitions. These are design findings, not claims about the current loader. ## Findings (ordered by severity) 1. **Durable work can remain blocked after removal or a failed first load.** A subagent reporter can commit its `report` checkpoint, then `subagents` is removed and the worker restarts. The next phase has no `p2.subagent-reporter` definition, so its answer is never delivered. A pinned snapshot protects a phase already running, not one resumed after a crash. The start barrier also permits an “optional” facet to fail before Durable resumes. See target-design.md §15 (lines 446–452), `plugins/subagents/tools.ts:118–164`, and `packages/durable/src/harness/scheduler.ts:1229,1366–1381`. **Fix:** treat definitions needed by unsettled Durable tasks as required; block removal or retain their definitions until the tasks settle, and check this before `resume()`. **Status:** Open. The decision to allow optional-facet failure and to remove types in place is stated in §15; this necessary Durable-specific exception is not resolved there. 2. **The start barrier can deadlock against its own worker admission.** The subagents host-lifetime contributor holds the server’s `Subagents` object (`plugins/subagents/worker.ts:35–39`). Today the worker admits its IPC link with `control.ready()` *before* awaiting `host.ready()` (`src/hosts/worker.ts:98–105`). If “wait until every contributor is live” and “route nothing until it reports” also delay that link admission, the hold cannot complete, so the worker cannot report. This applies to a worker spawned mid-rollout too. See target-design.md §§14–15 (lines 417–418, 446–457). **Fix:** specify an early transport/control admission distinct from application routing; gate external calls, not the link needed for bootstrap holds. **Status:** Open. The §14 start barrier and §15 install-before-routing policy are stated design decisions; their ordering needs to be reconciled. 3. **Publishing registry entries before their resources are ready bypasses the object gate.** Radius registers a request resolver that immediately reads `connection.value` (`plugins/radius/server.ts:46–48`); HTTP invokes resolvers directly from the registry (`src/hosts/server.ts:493–500`). Between publish and resource replacement/open, a request can invoke a new resolver against an unready resource. Retaining the old resolver is not necessarily safe either if its resource has been replaced and its handle is stale. See target-design.md §15 (lines 434–440). **Fix:** make registry invocation readiness-aware, or keep the old contributor *and its usable resources* until the replacement is ready, then switch exposure together. **Status:** Open. The publish/after division and no-entry-gap goal are stated decisions; direct registry calls need a rule that actually enforces them. 4. **A rejected facet can break a same-host registry contract without a binding failure.** For example, change chat’s `SendRouters` interface while changing `chat-mode`; if `chat-mode` setup throws, its old router remains, while new chat can read and invoke it directly (`plugins/chat/worker.ts:588–596`; `plugins/chat-mode/worker.ts:131–169`). The object-type bind rule does not protect that function call. Section 13 describes adapting old entries, but §15 does not say that an incompatible retained entry refuses the *new registry owner’s* publication. See target-design.md §§13, 15 (lines 395–401, 429–438). **Fix:** validate the complete proposed table, including retained entries, against its owner version; reject the dependent publication unless an adapter covers each old entry. Report the resulting mixed-version state, not just the facet that threw. **Status:** Open. Per-facet setup rejection and mixed versions within one host are expressly accepted in §15, but the claim that the same contracts protect every cross-plugin interaction needs this validation, including old registry readers as well as old entries. 5. **Removal contradicts the promise that running work finishes.** A chat shell is launched after its method returns; ending its object closes `shells` and aborts the command (`plugins/chat/worker.ts:185–195, 320–347`). If chat is removed while it runs, the command may already have changed files, while its subsequent state write or transcript entry can fail (`:270–315`). A retry can repeat the external effect without the original result being recorded. See target-design.md §15 (lines 449–452). **Fix:** either refuse removal while such work is active, or define removal as cancellation with a durable, visible terminal result. Do not apply the ordinary “running work finishes” claim to removal. **Status:** Open. In-place removal and non-draining reload are stated decisions, but their interaction for already-running work is unspecified. 6. **Successive installs need explicit per-host serialization.** Nothing in §15 or plugin-loading.md §7.5 orders R2 against an R1 whose asynchronous *after* phase is still reopening resources. R2 can publish and open its gate; an R1 completion can then replace a resource or report after R2, leaving the host’s advertised revision different from its live code. A server restart during a rollout makes late reports particularly important to discard. See target-design.md §15 (lines 429–440, 455–467); plugin-loading.md §7.5. **Fix:** serialize publish/after per host, retain only the latest desired complete code set, and fence every completion and report by revision *and incarnation*. **Status:** Open. Complete-code-set revisions with server incarnation are stated decisions; install ordering and stale-completion handling are not. 7. **`installed` at an older revision has no mismatch rule.** The stated cases cover a provider ahead, behind *and installing*, rejected, or at the same revision (`target-design.md:468–476`). In rapid rollouts, a client at R2 can meet a worker still reporting `installed` R1 before its R2 install starts. This is neither “behind and installing” nor final; parking without a known target relies only on the deadline. **Fix:** responses must include the provider’s desired/pending revision as well as its installed revision. Define a behind-but-not-installing state: initiate reconciliation if reachable, otherwise fail or time out with a specific reason. **Status:** Open. Parked calls, mismatch status and deadlines are stated decisions; this state is missing from the comparison table. 8. **The readiness gate opens before eager singleton migration.** Section 15 orders “then the readiness gate opens” before migrating exposed singleton documents (`target-design.md:439–444`). A newly admitted call or subscription can therefore reach a v2 member while its stored value is still v1, despite the stated rule that v2 operations never run on v1 state. **Fix:** migrate eager singletons before opening that object’s gate; if one migration fails, expose its specified per-member error when the gate opens. **Status:** Open. Lazy migration of unloaded/keyed documents and per-member failure for exposed singletons are stated decisions; the eager migration/gate order needs correction. ## Additional review points, open questions and boundary cases - **Clean reconnects, new workers and late clients.** The revision/incarnation scheme and install-before-routing rule address ordinary reconnects and late clients **if** enforced for every new worker, web page and TUI host. Current `src/runtime/server.ts:380–389` chooses a worker's init plugins at spawn, and `src/hosts/web/main.tsx` and `src/hosts/tui-client.ts` use their own client connection/bundle lifecycle. A worker spawned during rollout must be tagged with the complete desired revision, admit its bootstrap link as in finding 2, reconcile it, and withhold application routing until it has reported. A client that was disconnected, hidden, or installed late must reconcile the current complete set on reconnect; a server restart must force reinstallation across incomparable incarnations. **Status:** Intended by §15 (owner-design decision), but the admission/progress ordering in findings 2, 6 and 7 remains open. No separate defect is claimed for an ordinary successful reconnect. - **Partial rollout and rejected facets.** A server *load* error is specified to stop the rollout before publishing; a setup rejection *after* load can leave that server on mixed facet versions while workers and clients advance. Likewise a worker rejecting a facet is not necessarily a reason to stop the whole rollout. These are deliberate no-cross-host-transaction choices. The provider's reported revision must not pretend to name the complete code set when it retained an old facet, and the rejection status and plugin-specific reason must travel to affected bindings. Validate retained registry entries, shared document assumptions and old readers before treating this as contract-safe (finding 4). **Status:** Mixed versions/no votes decided in §15; safe detection/reporting of same-host incompatibility open. - **Project installs and base rollouts.** Project revisions install per worker, outside the base rollout (`plugin-loading.md` §7.4), while the worker also receives base facets from the server (`§4.1`, `src/runtime/server.ts:380–389`). An overlapping project install and base R1/R2 install needs a single per-host publication order and an explicit representation of both desired sets; comparing only the base revision must not mistake a project facet's state for a fully reconciled worker. **Status:** Separate project revision is stated in §7.4; combined ordering/identity is open. No concrete project-plugin failure was established in this review. - **Routines and parked calls.** §11 allows a routine's old handler to finish, with a deadline before starting its replacement. A worker routine waiting on a server call that is parked for the worker's pending install can delay that install itself. The deadline makes the wait finite, but an old handler that later resumes may duplicate a new routine's reconciliation unless the operation has its own idempotency key. The subagents reporter uses submission request IDs (`plugins/subagents/tools.ts:127–133,155–163`), an example of the required protection. **Status:** The routine deadline and reconciliation rule are stated decisions; ensure the install path does not wait on a handler blocked on that same install, and document per-operation deduplication where appropriate. This is a review question, not a separately established finding. - **Resources and old code.** §12 expressly allows a replaced native to error for old running work and says old handles become stale. This limits §15's broad “running calls and jobs finish with their old code” language: they finish execution under old code, but need not succeed or preserve an external operation after resource replacement. Code mode's running scripts depend on a sandbox resource (`plugins/codemode/worker.ts:22–40`); removing code mode has an explicit cancellation implementation (`plugins/codemode/calls.ts`). **Status:** Resource replacement semantics decided in §12. Open recommendation: state the limitation and require a terminal result or a refusal for effects that cannot safely be interrupted, especially removal (finding 5); do not promise uninterrupted work solely from code pinning. - **Preparation versus publication.** Explicit reload, never on save; no factory trial, migration dry run or live-consumer check at prepare; install means code published rather than every object live; per-object failed/backoff/retry are stated owner-design decisions (§15). These are not themselves defects in this review. They make clear post-publish failure states, per-member migration errors, validated registry tables and readiness gates necessary rather than optional preflight complexity. **Status:** Decisions stated in §15; concrete gaps above remain open. Read-only review: no implementation changes or tests were made.
- apps/p2/docs/p3-reviews/rev-ho-fable.md (43626 bytes)
# rev-ho-fable — adversarial prototype and target-design reviewer This file records the findings, review points, open questions, and recommendations I produced during the Chord v2 / p3 design reviews. It intentionally includes superseded designs and prototype-specific findings so the reasoning is not lost. Status labels: - **[DECIDED / SUPERSEDED]**: the owner subsequently chose a different design or explicitly addressed the point. - **[ADDRESSED]**: a later prototype/design revision addressed the specific issue, though the prototype was not merged. - **[OPEN]**: no later decision in this conversation clearly resolved it. - **[KEEP]**: implementation property or test coverage worth preserving. --- ## 1. Drain/handover prototype, first adversarial review Branch reviewed: `chord-handover`, initial local handover model. ### 1.1 Re-entry after one closure member drained — blocker **[ADDRESSED in the prototype; drain design later SUPERSEDED]** `object.ts:1667` admitted any root with work in *any* handover-group member. Members handed over concurrently, so A's steps (migrate, close exclusive, open) could run while B still drained. B's old call could then call A and be admitted to A's old implementation mid-steps. Concrete trace: A and B both bumped. A drained instantly and began `close conn1`. B's old call did `ctx.get(A).write`; A v1 ran after its resource was revoked, `conn.value` threw `stale`, and `count.change` wrote `{n:99}` to the old cell. The staged `{m:1}` was then installed and the write silently disappeared. Recommendation: drain the whole closure before any member's steps, or stop admitting re-entry to a member once its drain finishes. The prototype adopted a closure-wide quiet point and re-entry only until then. ### 1.2 A second effect swap during pending asynchronous cleanup — major **[ADDRESSED; serialized routines retained in current target]** A second `swap()` could find no active run and start immediately while the first asynchronous cleanup still ran. The first completion then bailed because its swap generation was stale, leaving `stalled` permanently true. New and old setup effects could overlap. Trace: v1 setup had a stubborn asynchronous cleanup; reload v2 reported pending; reload v3 started v3 while v1 cleanup remained; after v1 settled, `stalled` remained true. Recommendation: keep a per-slot `#stopping` promise, chain subsequent swaps on it, and clear stalled state when that cleanup settles regardless of later swaps. This became the current rule that routines are serialized per routine on reload. ### 1.3 Derivations installed from prepare-time values — major **[ADDRESSED; trials later SUPERSEDED]** The prototype precomputed derivations during prepare and installed that value at publication. Old code could write source cells while prepare waited on another object. Trace: x v2 derived `n*2+1000`; while y's `init` blocked prepare, x v1 changed `n` from 1 to 5. Publication installed 1002 instead of 1010 and it stayed stale until another write. Recommendation: recompute synchronously at publication before mutation, or attach without the prepare-time value. ### 1.4 New effect handler received old-shaped buffered state — major **[ADDRESSED; state is now transactional documents]** State-source events buffered across a handover were delivered to the new handler before the migrated reset. Trace: observations were `v1:{n:1}`, then `v2:{n:13}`, then `v2:{m:13}`. New code acted on v1 data. Recommendation: queued values from readable state are not events; drop them on swap and resubscribe for the current snapshot. Only real event sources buffer data. ### 1.5 An operation started by old code after publication got a new stamp — major **[ADDRESSED; operations replaced by jobs]** A pre-publication call could call `job.start` after publication. The operation was stamped with the current publication even though it ran old code, so old resources closed before the operation settled. Trace: old call started v1 operation after compatible v2 publish; old `conn1` closed while the operation ran; it later failed `stale` instead of returning 1. Recommendation: stamp detached work with the publication of the declaration/code generation that created it, not the wall-clock publication at `start`. ### 1.6 Failed handover could not revert to last-live code — major **[ADDRESSED; drain transitions later SUPERSEDED]** A migration staged v2, a later exclusive open failed, then reloading v1 was rejected as version lowering from staged v2 even though the last-live v1 value remained intact. Recommendation: plans start from the last-live version; reverting to that version discards staged transition state. ### 1.7 Peer-held references blocked breaking rollout — major **[ADDRESSED; current rollout uses revision/status mismatch handling]** Connection-owned v1 references were treated as local live consumers. A server provider could not bump to v2 while any worker held v1, so the rollout never began even though that worker was meant to install v2 in the same rollout. Recommendation: peer holds do not refuse local publication. End/reacquire them with a retryable `contract_pending`/revision state. Current design instead fails/waits at bind by revision. ### 1.8 Removal accepted while changed code still declared the removed dependency — minor **[ADDRESSED in prototype; removal semantics remain an OPEN consistency point below]** Removal checks skipped changed types and ran before their trial factories. A changed holder could install with a now-gone `self.get(B)` reference. Recommendation: check prospective declarations after factories run, or refuse lookup of a type removed from the prospective service table. ### 1.9 Handover wait deadline ignored drain deadline — minor **[ADDRESSED; drain design SUPERSEDED]** Waiting calls used `readyTimeout` only, although a reload could specify a longer drain plus grace. Calls timed out while the handover was still within its own deadline. Recommendation: fence wait budget must cover drain, step, activation, and grace budgets. ### 1.10 Reconnecting children received redundant full reloads — minor **[ADDRESSED; current target uses complete-set revisions]** Every reconnect reinstalled named child modules, restarting effects even when the child already ran them. A connection open before hello could also receive both rollout install and hello install. Recommendation: carry an artifact revision in hello/install and skip equal revisions. ### 1.11 First-round positive observations - Publication counter ownership and `releaseAfterOwners` were coherent for ordinary old calls and effect work. - `linkedSignal` detached listeners correctly, avoiding generation leaks. - Whole-suite generation GC test was meaningful: it tracked closure sentinels with `FinalizationRegistry`. - Re-entry across links preserved root ids correctly in the prototype. - Retry progress avoided double migration/open/close in covered paths once staged-state logic was corrected. --- ## 2. Drain/handover prototype, second review and facet object The first-round tests were rerun after fixes. Findings 1–7 were confirmed fixed; removal was correctly refused; wait budgets and revision skipping were fixed. ### 2.1 Facet-object activation failure was silent and produced an orphan repair — major **[Prototype-specific; OPEN as a startup/error-handling test requirement]** The facet hold was constructed before `object.start()`, so `RefImpl.ready` captured the initial resolved readiness. `install()` resolved while the facet object was still activating; a failed resource open was reported nowhere. Failure forgot the object from `host.objects` but retained it in `facetObjects`. A fixed reload could make that orphan `live` outside the host tables: registrations stayed absent and `host.close()` did not close its resource. Recommendation: start before creating readiness-dependent holds, and either retain failed host/facet objects as failed live identities or drop and recreate them consistently. Add startup-failure/retry/close tests. ### 2.2 Failed host-lifetime object was never pinned again — major **[Prototype-specific; OPEN as retry semantics]** A gone bootstrap ref remained in `hostObjects`, causing reload preparation and repinning logic to treat it as present. Later external `get` could create a transient object, but its host registration disappeared when released. Recommendation: a gone bootstrap ref is absent; reload must reacquire and pin it. ### 2.3 Prepare deadlocked against an old-code activation — major **[DECIDED: factory trials removed]** Old activation waited for `whenReloadDecided`; the trial's new factory held that same object and waited for its readiness. Reload timed out. Trace: Y v1 activation blocked in old open; X v2 trial declared `self.get(Y)`; Y waited for reload decision, reload waited for Y ready. Recommendation at the time: do not await changed old activations in trial hydration, or let them restart at publish. Final design removed factory trials entirely. ### 2.4 Old served-object call creating a sibling during facet handover deadlocked — major **[DECIDED: drain/closure handover rejected]** A draining old call did `ctx.get` for a new sibling served by the same facet. Sibling activation waited for the facet's handover fence; the fence waited for the old call. It ended only at the drain deadline. Recommendation at the time: activate under old service and join the closure, or reject immediately rather than deadline-stall. Transactional reload removes this closure mechanism. ### 2.5 Registry entry called without ctx deadlocked the closure — major **[DECIDED: no late-bound registries; immutable tables]** A contributor called a registry helper with a synchronous signature and no `ctx`. The wrapper created a fresh root with a never-aborting signal, then waited on its own contributor's fence. Production default wait approached 72 seconds. Recommendation at the time: no-ctx calls either use old code during drain or fail at the drain deadline. Current design uses immutable old registry tables rather than late-bound wrappers. ### 2.6 A synchronous late-bound entry became a Promise only during reload — major **[DECIDED: no late binding]** `defineRegistry<() => string>` returned a plain string normally, but while admission waited it returned an async IIFE/Promise. Runtime behavior violated its type only during reload. Recommendation: entries must be consistently async, or synchronously fail while unavailable. Current immutable-table design removes the wrapper. ### 2.7 Stalled served object blocked every future reload of its facet with no cancel API — major **[DECIDED: no drain/stalled model]** One signal-ignoring call left every closure member updating forever. A repair reload was refused `handover_running`; host API had no way to force-dispose host-principal objects despite the design calling stalled work cancellable. Recommendation at the time: explicit `host.cancel(address)`/force disposal, or repair reload may force-dispose stalled referenced members. Current no-drain design eliminates this state. ### 2.8 Facet setup could not hold a type from a later facet in the same load — minor **[OPEN: still a requirement in plugin-loading]** `host.load` claimed load order independence, but facet setup was synchronously run in each `install`; `self.get` of a type served by a later facet failed `unknown_type`. Recommendation: two-pass load—register every facet's services first, then activate facet objects/eager objects. ### 2.9 New bump `init` saw unmigrated sibling state — major **[DECIDED: cells/init model replaced by transactional documents]** A v2 `init` ran before staged migrated values were installed. It read another v2-declared cell and received its v1 shape. Trace: `settings` migrated `{n:7}`→`{m:7}`; `cache` v2 init read `settings.value.m`, but observed `{n:7}` and silently stored `undefined`. Recommendation: run init against a staged prospective view or disallow such dependency. Transactional documents now migrate on access and old definitions throw. ### 2.10 Other second-round observations - Facet generation GC held across repeated parallel and exclusive reloads: only the current sentinel survived. - Served objects joining during facet handover eventually activated on new code; releasing a member during handover disposed it after install. - A finite exclusive open exceeding the step deadline was closed and made the object failed, while retry reopened it unbounded at prepare and succeeded. This contradicted “past deadline = stalled” and showed inconsistent first-attempt/retry semantics. --- ## 3. State-design reviews before transactional documents ### 3.1 Every cell has init; optional migrate review **[SUPERSEDED by transactional documents]** 1. **Memory-only reset was too easy.** Once every bump had `init`, forgetting `migrate` silently reset drafts/UI state. Recommendation: require explicit `migrate` or `reset: true`. 2. **Mirror stale after resource replacement without cell bump.** `db` v1→v2 could change rows while mirror cell kept old value because init only ran on a cell bump. Recommendation: cell deps on resources, rerunning init after dependency replacement. 3. **Mirror catch-up gap.** Init read source at T1; effect subscribed at T2. External changes between them were lost. Recommendation: subscribe before catch-up or return `{value,since}` cursor. 4. **Init-only siblings had no ordering.** One init could read a sibling still uninitialized; parallel init retried forever. Recommendation: declaration-order init or explicit deps. 5. **Paged clients lacked cache invalidation.** Live window could change shape/revision while cached older pages remained old or stale after edits. Recommendation: window carries `historyRevision`; cache keys include revision and contract fingerprint. 6. **Retry/revert side effects.** Init could write into a source, later handover fail, and revert to old code that did not know the write occurred. “Idempotent” must mean stable-id upsert, and docs must state init writes are not rolled back. ### 3.2 `init({previous, from})` review **[SUPERSEDED]** The API itself was sensible for idempotent migration into a source. However: - Bumped init reran on retry while added-cell init was staged/reused, creating inconsistent semantics. - Init could call another closure member and deadlock until the step deadline; guidance needed to forbid calls into closure siblings/dependents. - Most importantly, init ran as new code over unmigrated sibling cells (finding 2.9). --- ## 4. “Reload without drain / version fences” proposal review **[DECIDED / SUPERSEDED by transactional document store]** Verdict then: simpler and right direction, but broken if applied to mutable cells and raw natives without retaining bounded ownership waits and serialized effect swaps. ### 4.1 Exclusive close under non-cooperative work produced native errors, n - apps/p2/docs/p3-reviews/rev-ho-opus.md (51511 bytes)
# rev-ho-opus — adversarial reviewer of Chord v2 / p3 state, jobs, reload and rollout This file records the findings, review points, open questions and recommendations I produced in the review conversation. It excludes the functionality map. Status labels reflect later owner decisions or later designs known in that conversation; they are not a fresh re-review of current HEAD. Status legend: - **Decided / superseded:** the owner later chose a design that replaces the reviewed one. - **Decided / adopted:** the later target design explicitly adopted the recommendation. - **Confirmed fixed:** a later prototype revision and regression test addressed the finding. - **Open:** no later owner decision or verified fix was established in the conversation. - **Historical:** useful implementation evidence from a design that is no longer the target. --- ## 1. Drain/handover prototype: first adversarial review **Status: Decided / superseded.** The owner later rejected drain-based handover. These findings remain useful traces of why. ### 1.1 Blocker: a drained object could still run old code during the rest of its handover **Finding.** `object.ts:1667` admitted any call whose call tree had work in *any* object of the handover group. Each object's handover ran independently (`reload.ts:568`), not in dependency order. Once B had drained, a call from A, which was still draining, could enter B's old code while B migrated and swapped resources. **Concrete trace / test.** `/tmp/chrev/reentry.test.ts`: B drained, migrated `{n:1} -> {m:1}`, closed `conn1` and waited on `conn2`. A's call then reached B v1: its write to `n` produced `{n:11}`, its resource read threw `stale`, and B ended at `{m:1}`. The write was lost although the reload reported `installed`. **Recommendation.** Admit re-entry only while the whole closure is draining. Drain the closure to one common quiet point before any member runs migration/close/open steps, then execute steps in dependency order. **Later status: Confirmed fixed in the prototype.** The rewritten test retained the re-entrant write (`{m:11}`), and no read happened after resource close. ### 1.2 Major: registry calls escaped contributor admission and ownership **Finding.** `host.ts:617` returned raw registry values. A route or tool call did not become work of the contributing object. Neither handover drain nor `releaseAfterOwners` waited for it. **Trace.** `/tmp/chrev/registry.test.ts`: a running route saw `close conn1`, then `route v1 ERR stale`, for both an exclusive handover and a parallel resource replacement. **Recommendation.** Wrap function entries so invocation enters the contributor's admission and begins stamped work with the caller's root id/context. **Later status: Confirmed fixed for function-valued entries; object-valued entries remained a later finding.** ### 1.3 Major: queued `run()` closures were stamped when they started, not when admitted **Finding.** A closure queued before publication could begin afterward with the new publication stamp and therefore own none of its old declarations. **Trace.** `/tmp/chrev/queued.test.ts`: `v1 handled 1 with conn1`, then `close conn1`, then queued `v1 handled 2 ERR stale`. **Recommendation.** Reserve and stamp work when `run(fn)` admits the closure, or drop queued closures from a stopped generation. **Later status: Confirmed fixed.** The old resource closed only after the queued closure ran. ### 1.4 Major: every failed object blocked every unrelated reload **Finding.** `#todo` retried failed objects in every reload. A failed exclusive open therefore rejected an unrelated facet change during prepare. **Trace.** `/tmp/chrev/failedblocks.test.ts`: while port 9090 remained busy, reloading an unrelated `other` facet was rejected with `EADDRINUSE`. **Recommendation.** Retry failed objects only when their type/plugin is in the batch, or isolate retry failure to that object and continue the reload. **Later status: Confirmed fixed.** ### 1.5 Major: reverting to last-live code could not repair a failed handover **Finding.** Staged v2 state was treated as current, so reverting to v1 was rejected as a version decrease even though v2 had never gone live. **Trace.** `/tmp/chrev/revert.test.ts`: after a failed v2 port open, reload to v1 failed with `version of state count lowered from 2 to 1`. **Recommendation.** Plan from last-live state. A staged copy is valid only for exactly its target version/schema and should be discarded for a different target. **Later status: Confirmed fixed.** ### 1.6 Major: a handover step that hung was never reported and could never be repaired **Finding.** Exclusive `open` and `init` after drain had no deadline. The object remained `updating`, not `stalled`; reload never resolved, later reloads got `handover_running`, and a child's serialized install queue wedged permanently. **Trace.** `/tmp/chrev/hang.test.ts`: an exclusive open that never resolved left `reload()` pending and the object `updating, stalled:false`. **Recommendation.** Give each post-drain step an aborting deadline and stalled reporting; do not let a child install queue wait forever on a post-publication handover. **Later status: Partly fixed for one-member closures; multi-member closure remained broken in round two.** ### 1.7 Major: a second reload during async effect cleanup started a replacement under it and left `stalled` stuck **Finding.** A stopped run had already been cleared, so the second swap saw nothing to wait for and started immediately. The superseded first swap returned without clearing its stalled flag. **Trace.** `/tmp/chrev/doubleswap2.test.ts`: `setup v3` occurred before `cleanup done v1`; the second report said `installed`; `stalled` stayed true forever. The loop variant also showed the stuck flag. **Recommendation.** Keep the pending cleanup promise on the slot and chain every later start after it. Clear stalled state when the cleanup settles or a swap is superseded. **Later status: Confirmed fixed for loops and plain setups.** ### 1.8 Minor: old activations delayed publication and could livelock reload **Finding.** Prepare waited for activating objects of changed types. A slow activation delayed reload up to `readyTimeout`; a steady stream could prevent publication forever, despite a `CodeChanged` restart path existing. **Recommendation.** Publish without waiting and abort/restart old-code activations synchronously at publication. **Later status: The direct wait was fixed; a prepare/activation dependency cycle remained a later finding.** ### 1.9 Missing design pieces - Facet setup was not represented as a facet object, so objects sharing facet closures were not handed over with it. - Dependents of a failed provider stayed live rather than failing with it. - Handovers ran concurrently rather than in dependency order. **Recommendation.** Introduce the anonymous facet object, include served objects when its declarations change, propagate provider failure, and order close/open by the resource/reference dependency graph. **Later status: Implemented in the prototype, then superseded with the drain design itself.** ### 1.10 What held in the first prototype Calls racing publication, waiting calls, stale-write refusal after install, parallel-resource ownership by stamped work, cell version checks and dry runs, no repeated migration on a plain retry, and the repeated-reload generation checks held under the tests examined. --- ## 2. Drain/handover prototype: second adversarial review **Status: Decided / superseded.** The owner later rejected the drain design. These findings came after closure-wide drain, dependency order and the facet object were added. ### 2.1 Major: sequential closure steps let one member freeze every later member **Finding.** Members' close/open steps were awaited one at a time. A member whose open ignored its signal prevented all later members from reaching their own steps and prevented the reload report from completing. **Traces.** - `/tmp/chrev/r2facet.test.ts`: served object `a` hung opening a cursor after a facet DB replacement. `reload()` never reported; served object `b` remained `updating`; later reloads were refused. - `/tmp/chrev/r2seq.test.ts`: six independent served objects each took 60 ms, below a 100 ms step deadline. Sequential processing took 366 ms and the last two waiting calls hit `handover_timeout`. **Recommendation.** Execute independent members concurrently once their providers are ready. Report every member blocked behind a stalled provider. Size call waiting by the longest dependency path, not the number of members. ### 2.2 Major: registry entry code ran with the caller's context, not work-owned child context **Findings.** - The handover deadline aborted the entry's internal work record, but not the context passed to entry code. A streaming route honoring `ctx.signal` never stopped. - New code reached through a long-lived caller context retained the caller's pre-migration stamp, so legitimate writes were rejected as stale. **Traces.** - `/tmp/chrev/r2entryabort.test.ts`: the reload reported pending, the route kept running, and `host.close()` hung. - `/tmp/chrev/r2entrystale.test.ts`: a new route after migration got `stale: a write from work that started before its state was migrated`. **Recommendation.** Replace the entry's context argument with `ctx.child(contributor, work)`, exactly as object method calls do. ### 2.3 Major: object-valued registry entries bypassed admission and ownership **Finding.** Only top-level function values were wrapped. A tool object such as `{ name, execute }` was returned raw. **Trace.** `/tmp/chrev/r2registry.test.ts`: the old resource closed while `execute` ran; `execute` then got `stale`. **Recommendation.** Wrap declared callable properties of object entries, or constrain registries to function values. ### 2.4 Major: removing a facet closed facet resources under served-object work **Finding.** `dropFacetObject` released its host hold immediately. The facet object disposed and closed resources before served objects completed forced disposal. **Trace.** `/tmp/chrev/r2facet.test.ts`: `close db1` occurred while a served call was pending; the call resumed with `not_ready`. **Recommendation.** Keep the facet hold until every served object has disposed; release it afterward. ### 2.5 Major: revision skip after a rejected child install permanently stranded missing modules **Finding.** A child that rejected R2 could successfully install only R3's delta and then claim R3. Reconnect skipped because revisions matched, although R2's module was never installed. **Trace.** `/tmp/chrev/r2revision.test.ts`: module `a` failed transiently in R2, module `b` installed in R3; reconnect installed nothing and the worker had only `b` while the server desired `a,b`. **Recommendation.** Clear/withhold the child's revision after rejection, track revision per connection, or send the complete desired module set for every revision. ### 2.6 Minor/major: prepare deadlocked with an old-code activation **Finding.** The activation waited for reload decision, while a trial/new host object waited for a reference to that activation. **Trace.** `/tmp/chrev/r2activation.test.ts`: reload rejected after `readyTimeout`, naming the activating target reference. **Recommendation.** A candidate acquisition of an old-code activating object must restart it immediately with candidate code, or prepare must not await readiness of references to changed types. ### 2.7 Minor: no-context entry call could wait for its own handover **Finding.** A closure member calling a registry entry without passing a context got a new root and a never-aborting signal. It parked behind its own fence. **Trace.** `/tmp/chrev/r2registry.test.ts`: still waiting after two seconds; it could wait roughly the full 60-second handover timeout. **Recommendation.** Carry root identity through async-local context, or reject/cancel no-context entry waits at the drain deadline. ### 2.8 Facet-generation GC held `/tmp/chrev/r2gc.test.ts` reloaded facet objects five times with both parallel and exclusive resources; only the current generation remained. This part held. --- ## 3. State §7 design reviews before transactional documents ### 3.1 `init({ previous, from })`: live previous value was unsafe **Status: Decided / superseded by transactional documents.** **Finding.** Passing the live last-live object let `init` mutate state outside transactions. If `init` later failed, reverting served corrupted old state with no state event. **Trace.** `/tmp/chrev/r2init.test.ts`: `init` drained `previous.items` then threw. Revert served `{ items: [] }` instead of `['a','b','c']`. **Recommendation.** Pass a structured clone or frozen value. ### 3.2 `init` calling another closure member could deadlock **Status: Decided / superseded.** `init` ran in a fresh root after re-entry closed. Calling a sibling/self that had not completed handover parked until deadline and failed repeatedly. The author could not know closure membership statically. **Recommendation.** Enforce at runtime: fail fast when an init root calls an updating closure member, or permit only already-installed providers. ### 3.3 An init deadline could be too short for data-moving I/O **Status: Decided / superseded.** Moving a large in-memory value to SQLite under the ordinary drain deadline could fail every retry. If I/O committed after abort, retry repeated it. **Recommendation.** Give init its own longer deadline and require one transaction plus idempotence. ### 3.4 Mirror cell could lose or duplicate source events around init **Status: Decided / adopted in later design as source readables / derives.** **Trace.** First start: init reads m1–m4, m5 commits before subscription, so m5 is lost. Reload: an event m5 buffers while init reads through m5; replay appends m5 twice. **Recommendation.** Mirrors come from atomic readable sources, or use sequence-numbered events with subscribe-before-catch-up and drop events at or below the read position. ### 3.5 Memory-only data could be silently dropped **Status: Decided / superseded by versioned transactional documents.** Because every cell had init, a bump without migrate silently reset drafts. Returning `undefined`, renaming or removing a cell had the same effect. **Recommendation.** Declare mirror versus memory state, require migrate for memory state, and require an explicit drop/retire declaration. ### 3.6 Paging needed stable cursors, gap detection and cache epochs **Status: Partly adopted by the real Chat window/history design; general contract remained a review recommendation.** **Traces.** Offset paging while the window advances causes skips/duplicates. A disconnected client whose cache ends at m149 can reconnect to window m400–m449 without knowing m150–m399 are missing. Schema/source rewrites leave cached pages stale. **Recommendation.** Stable monotonic entry ids; window - apps/p2/plugins/agent-plugins/skills/writing-p2-plugins/example/counter/contract.ts (576 bytes)
// A session's shared counter: its count (state), who looks at it now (state), waves (an event). import { arr, int, obj, str } from "@earendil-works/pi-schema"; import { method, param, remote, serverHost } from "@p2/app/v1"; export const Counter = remote("counter.board", { version: 1, params: { session: param.str() }, hostAddress: serverHost, variants: { player: { state: { count: int(), present: arr(str()) }, events: { waved: obj({ from: str() }) }, methods: { add: method(obj({ by: int({ minimum: -100, maximum: 100 }) })), wave: method() }, }, }, }); - apps/p2/plugins/agent-plugins/skills/writing-p2-plugins/example/counter/package.json (60 bytes)
{ "name": "counter", "private": true, "type": "module" } - apps/p2/plugins/agent-plugins/skills/writing-p2-plugins/example/counter/server.ts (3156 bytes)
// The counter's server side: one store keeps every session's count (persisted) and emits waves; each person's (and // the session's agent's) Counter object shows the session's count and who is there, and adds and waves for those who // may send. import { int, obj, str } from "@earendil-works/pi-schema"; import { canSend, canView, defineFacet, defineState, event, forbidden, isWorker, keyed, local, method, untilAborted, } from "@p2/app/v1"; import { Counter } from "./contract.ts"; const Counts = defineState("counter.counts", { schema: obj({ value: int() }), version: 1, initial: () => ({ value: 0 }), keyed: true, persist: true, checkpointWhen: () => true, }); const Store = local("counter.store", { version: 1, params: {}, members: { counts: keyed(obj({ value: int() })), waved: event(obj({ session: str(), from: str() })), add: method(obj({ session: str(), by: int() })), wave: method(obj({ session: str(), from: str() })), }, }); export default defineFacet((host) => { host.serve(Store, (_params, self) => { const counts = self.state(Counts); const waved = self.events<{ session: string; from: string }>(); return { counts, waved, add: async ({ session, by }, ctx) => { await counts.change( session, (draft) => { draft.value += by; }, ctx, ); return null; }, wave: async ({ session, from }) => { waved.emit({ session, from }); return null; }, }; }); const references = host.references(Counter); host.serve(Counter, ({ session }, principal, self) => { // The session's own agent (its worker) may use it too: the plugin's tool. const agent = isWorker(principal) && principal.session === session; if (!agent && !canView(principal, session)) throw forbidden("the session's viewers only"); const store = self.get(Store, {}); const name = principal.kind === "guest" ? principal.name : principal.kind === "worker" ? "agent" : principal.kind; const count = self.derive([store.counts.at(session)], ([doc]) => doc.value); const present = self.derive([references], ([list]) => [ ...new Set( list .filter((each) => each.params.session === session) .map((each) => (each.principal?.kind === "guest" ? each.principal.name : (each.principal?.kind ?? "?"))), ), ]); // Waves of this session only, re-emitted from this object's own source. const waved = self.events<{ from: string }>(); self.onLive(async (ctx) => { const subscription = await store.waved.on(({ value }) => { if (value.session === session) waved.emit({ from: value.from }); }, ctx); try { await untilAborted(ctx); } finally { subscription.stop(); } }); const mayAdd = () => { if (!agent && !canSend(principal, session)) throw forbidden("only people who may send here"); }; return { variant: "player", members: { count, present, waved, add: async ({ by }, ctx) => { mayAdd(); await store.add({ session, by }, ctx); return null; }, wave: async (ctx) => { mayAdd(); await store.wave({ session, from: name }, ctx); return null; }, }, }; }); }); - apps/p2/plugins/agent-plugins/skills/writing-p2-plugins/example/counter/web.tsx (1686 bytes)
// The counter's pane: view kind "counter", beside the transcript (a view) or in it (`inline`, a tool card's). import { BACKGROUND_CONTEXT, contribute, defineFacet, liveIn } from "@p2/app/v1"; import { type PaneProps, useObject, useReadable, useStatus, ViewKinds } from "@p2/app/v1/web"; import { useEffect, useState } from "react"; import { Counter } from "./contract.ts"; function CounterPane({ session, editable, inline }: PaneProps) { const ref = useObject(Counter, { session }); useStatus(ref); const board = liveIn(ref, "player"); const count = useReadable(board?.count); const present = useReadable(board?.present) ?? []; const [wave, setWave] = useState(""); useEffect(() => { if (!board) return; let stopped = false; let stop = () => { stopped = true; }; void board.waved .on(({ value }) => setWave(`${value.from} waved`), BACKGROUND_CONTEXT) .then((subscription) => { if (stopped) subscription.stop(); else stop = () => subscription.stop(); }); return () => stop(); }, [board]); return ( <div style={{ padding: inline ? 0 : "var(--x)" }}> <p className="counter-count">{count ?? "…"}</p> <p className="muted">Here: {present.join(", ")}</p> {editable && board && ( <p> <button type="button" onClick={() => void board.add({ by: 1 }, BACKGROUND_CONTEXT)}> +1 </button>{" "} <button type="button" onClick={() => void board.wave(BACKGROUND_CONTEXT)}> Wave </button> </p> )} <p className="muted">{wave}</p> </div> ); } export default defineFacet((host) => { contribute(host, "counter.web", (self) => { self.contribute(ViewKinds, { counter: { Pane: CounterPane } }); }); }); - apps/p2/plugins/agent-plugins/skills/writing-p2-plugins/example/counter/worker.ts (1936 bytes)
// The counter's tool for the session's agent: `counter_add`, for code mode only (scripts call it as // `tools.counter_add({ by })`), in the code-mode namespace `counter` its extension declares: the prompt lists the // namespace with its description, `describeNamespace("counter")` gives its instructions and tools. import { defineExtension } from "@earendil-works/pi-durable"; import { int, obj } from "@earendil-works/pi-schema"; import { contribute, defineFacet, expectVariant } from "@p2/app/v1"; import { defineP2Tool, Extensions, type P2Tool, workerOf } from "@p2/app/v1/worker"; import { Counter } from "./contract.ts"; export default defineFacet((host) => { contribute(host, "counter.tools", (self) => { const worker = workerOf(self); const extension = defineExtension<P2Tool>({ name: "counter", namespace: { name: "counter", description: "The session's shared counter (the Counter view).", instructions: "counter_add changes the count for everyone in the session; open the view with view_open.", }, tools: [ defineP2Tool({ name: "counter_add", description: "Adds `by` to the session's shared counter (the Counter view); returns the new count.", parameters: obj({ by: int({ minimum: -100, maximum: 100 }) }), replay: "unsafe", // Out of the model's tool list: code mode's scripts call it. Leave it out to offer it to the model too. callers: ["tools"], execute: async ({ by }, _api, context) => { const count = await worker().root(async (ctx) => { const session = worker().init.values.session.id; const board = expectVariant(await ctx.get(Counter, { session }), "player"); await board.add({ by }, ctx); return board.count.value; }, context); return { output: [{ type: "text", text: `The count is ${count}.` }] }; }, }), ], }); self.contribute(Extensions, { [extension.name]: extension }); }); }); - apps/p2/plugins/agent-plugins/skills/writing-p2-plugins/SKILL.md (7311 bytes)
--- name: writing-p2-plugins description: Write, apply and show a p2 plugin (shared or persistent views, multiplayer games, boards, editors several people use, server-side objects, tools for the agent). Use when the user wants something multiplayer, persistent or with rules inside p2; html artifacts are for single-player pages. --- # Writing p2 plugins A p2 plugin is a directory of TypeScript files that p2 builds and loads into its server, every session's worker and the page. Plugins you write live in the **user plugin root**, `<data>/plugins/` on the p2 server (the `local:` directory this skill's subagent works in). They survive p2 updates; a built-in plugin wins a name clash. Use a plugin for anything multiplayer, persistent or with rules: games, shared boards, editors several people use. An html artifact (`artifact_write` with kind `html`, a page you write) is for single-player pages. ## Where to work Work on the server's own files: as a subagent located at `local:<data>/plugins` (subagent `env: null`, `cwd` the root), or in a session there. Write files with the ordinary `write`/`edit`/`read`/`bash` tools. Saving applies nothing; `reload_plugins` does. ## Layout ``` <name>/ lowercase letters, digits, dashes; not a built-in plugin's name package.json { "name": "<name>", "private": true, "type": "module" } (name = directory) contract.ts the types the other files share (no facet: imported by them) server.ts runs on the server (shared state, rules) worker.ts optional: runs in each session's worker (tools for the agent) web.tsx optional: runs in the page (React panes) skills/<skill>/SKILL.md optional: skills for agents on the server's machine (see below) ``` Imports: only `@p2/app/v1` (any facet), `@p2/app/v1/server` (server), `@p2/app/v1/worker` (worker), `@p2/app/v1/web` (web), `@earendil-works/pi-schema`, `@earendil-works/pi-durable` (worker, for `defineExtension`), `react` (web), and the plugin's own files by relative path. No other packages, no top-level await, no `import.meta`. `pluginFile(path)` from `@earendil-works/chord/plugins` gives a facet the absolute path of a file of its own plugin. ## The pieces - **Contract** (`contract.ts`): `remote(name, { version, params, hostAddress: serverHost, variants })`. Each variant has `state` (JSON schemas from `@earendil-works/pi-schema`), `methods` (`method(args, result)`) and `events` (fire-and-forget values for motion: pointers, drags, game inputs; never kept or replayed). Every change of a contract's shape bumps its `version`. - **Server object** (`server.ts`): `export default defineFacet((host) => { host.serve(Contract, (params, principal, self) => ({ variant, members })) })`. Check the principal: `canView`/`canSend(principal, session)` for people, `isWorker(principal) && principal.session === session` for the session's agent. Persistent data lives in a local store served once (`local(name, { version, params: {}, members })` with `self.state(defineState(name, { schema, version, initial, keyed?, persist: true, checkpointWhen }))`, changed with `handle.change(key, draft => ..., ctx)`); each person's object reaches it with `self.get(Store, {})` and shows derives (`self.derive([sources], fn)`). Data that belongs to one session is keyed by the session's address (`keyed: Session` from `@p2/app/v1`, keys `Session({ id })`): a session's export, import and delete carry it. Events: `self.events<T>()`, `.emit(value)`; listening to a reference's event member: `await ref.member.on(listener, ctx)`. Presence: `host.references(Contract)` lists who holds which object now. - **Pane** (`web.tsx`): `contribute(host, "<name>.web", (self) => self.contribute(ViewKinds, { <kind>: { Pane } }))`. The pane gets `PaneProps` (`session`, `item` (the view's item: what it shows, e.g. a board's ID), `inline` (true in the transcript, false beside it), `view` (beside the transcript), `editable`, `select`, `annotations`) and holds objects with `useObject(Contract, params)`, `liveIn(ref, variant)`, `useReadable(member)`. - **One script per plugin:** each plugin's web facet is built into a script of its own, and a file it imports from another plugin is copied into it. Never share module state (a `Map`, a `let`, a store, a React context) through such a file: the other plugin's copy is another instance. Use a registry the owning plugin serves, or `sharedContext(name, initial)` from `@p2/app/v1/web` for a React context. A plugin's web change applies in open pages in place (no reload). - **Listable items** (`web.tsx`, optional): a plugin keeping several items per session (boards, games) contributes an artifact kind, `self.contribute(ArtifactKinds, { <kind>: { title, icon, useItems, useCount, view, useRemove? } })` (`ArtifactKinds` from `@p2/app/v1/web`): the artifacts button counts them, its sheet lists them, and a row opens the item as a view of kind `view` with the item's ID. - **Tools** (`worker.ts`): `defineExtension({ name, namespace, tools: [defineP2Tool({...})] })` contributed to `Extensions`. A plugin's tool is **code-mode only by default**: give it `callers: ["tools"]`, so it stays out of the model's tool list and scripts call it (`await tools.<name>(args)` in `codemode`). The extension's `namespace` (`{ name, description, instructions? }`) groups its tools in code mode: the prompt lists the namespace's name and one-line description, and `describeNamespace(name)` gives its instructions and tools. To offer a tool in the main list instead, leave out `callers`; do that only for a tool the agent needs in most turns. - **Skills**: a worker facet contributes its plugin's skills directory: `self.contribute(PluginSkills, { <name>: { dir: pluginFile("skills") } })` (`PluginSkills` from `@p2/app/v1/worker`). Agents on the server's machine see them in `<available_skills>` after the environment's and the project's skills; agents on other environments do not (the files stay on the server). ## Apply and show 1. `reload_plugins` (no arguments: every changed plugin and every plugin of the user root; or `{"plugins": [names]}`) builds and rolls out to the server, every session's worker and the page (open pages reload onto the new bundle by themselves). It reports per plugin: `applied`, `removed`, `failed` with the errors as `file:line:column: message`, or `refused` with the reason (an invalid `package.json`, a built-in's name), and errors logged meanwhile. A failed build applies nothing; the old code keeps running. Fix and reload again. 2. Show the pane: in code mode, `await tools.view_open({ kind, title, item? })` (`kind` the name your web facet registered, `item` what it shows). A kind no loaded plugin registers shows "Unknown view kind". Edits reload in place: objects people hold go on with the new code. ## Complete example `example/counter/` next to this file is a complete, tested plugin: a shared counter with presence (who has the pane open), a wave event, a pane and a code-mode tool in its namespace. Copy it to `<root>/counter/` (`cp -r`), run `reload_plugins`, then in code mode `await tools.view_open({ kind: "counter", title: "Counter" })`. Read its files before writing your own. - apps/p2/README.md (28891 bytes)
# p2 > [!WARNING] > # EXPERIMENTAL > **p2 is a prototype. It will be gone soon.** It is not supported, its data formats and APIs change without notice, > and it will be replaced. Do not build on it or run it for anything you depend on. A self-hosted Pi Durable UI built from independent [Chord](../../packages/chord) plugins. Owners have full agent and shell control on that server and its workers; this is not server-side filesystem isolation. Downloaded TUI facets run in a locked-down worker on the visitor's machine. ## Run Requires Node 22.19+ and the monorepo's installed dependencies and model data. From the repository root: ```sh npm install --ignore-scripts npm run hydrate:model-data cd apps/p2 npm start ``` Open http://127.0.0.1:8788 and paste the sign-in token the server prints at startup (`p2: sign-in token (…): <token>`). `PORT` changes the port; `P2_DATA_DIR` changes the default data directory `~/.pi/p2`. `P2_FAKE=1 npm start` uses a fake provider for demos, but tools still act on real files. See [development notes](docs/p2-notes.md) for dev mode, architecture and testing. Code mode scripts can orchestrate one-shot agents (`tools.agent`); [docs/workflows.md](docs/workflows.md) has workflow patterns with example scripts (fan-out with adversarial verification, drafts plus a judge, fix until a check passes, search until nothing new). ## Signing in on this machine Loopback alone proves nothing: on a machine with several users, any of them reaches `127.0.0.1:8788`. The owner is whoever can read the server's private data directory (0700; its `owner-key` is 0600): - **Browser:** each browser profile signs in once: paste the one-time token the server prints at startup (valid ten minutes) into the sign-in page, or open a fresh link from `npm run owner-link -- http://127.0.0.1:$PORT`. Redeemed on loopback, the token gives a local owner session (`access: "local"`): it may mint owner links and use loopback provider sign-ins. The session cookie lasts 30 days and each app load renews it, so a browser in use stays signed in. A browser without it gets the sign-in page (`/health` stays open). - **TUI, `npm run owner-link`, `scripts/verify-sessions.ts`:** they sign in by themselves with the owner key of the data directory (`P2_DATA_DIR`, default `~/.pi/p2`), which must be the running server's: run them as the server's user, with its `P2_DATA_DIR`. Otherwise they say which directory did not match (the TUI then asks for an owner link). The key never leaves the client: the server sends a one-time challenge, and the client answers with an HMAC of it and the server's port, so whatever listens on that port (another user's process while p2 is down) cannot replay the answer. - **Radius** authenticates its relayed requests itself and needs none of this. **Remaining risk:** browser cookies are not port-scoped, so a browser holding the session cookie sends it to any `127.0.0.1:<port>` page it opens, including another local user's server. That cookie is a session, not the key: **Sign out all devices** ends it (and every other session). Use a separate browser profile for untrusted local servers. ## Using the owner UI from another device The server listens only on loopback. Use an HTTPS tunnel or reverse proxy preserving the public Host header, and list its hostname in `P2_ALLOWED_HOSTS` (comma-separated for several hosts). 1. On the host, with `PORT` matching its running server: ```sh npm run owner-link -- https://your-domain ``` The base defaults to `P2_PUBLIC_URL`, or `http://127.0.0.1:$PORT` (8788 when unset). The command signs in as the local owner with the data directory's `owner-key` (see above), so only the server's user can run it. 2. Open the printed `https://your-domain/owner#<token>` link on your other device. The page removes the fragment, POSTs its token on the same origin, then redirects to `/`. The fragment never reaches HTTP/proxy logs. Links work once and expire after ten minutes. Treat their output as a private credential: do not publish it or put the token in a query/path. You can also use **Devices → Copy token** in a local owner's UI, or the private `<P2_DATA_DIR>/owner-token.txt` startup token, and paste it at `/owner`. Owners get distinct signed, HttpOnly, SameSite=Lax cookies, Secure on a remote HTTPS origin. Cookies last 30 days (server-side expiry too), are port-scoped, and survive restarts through the private `owner-key`. Devices signed in on a remote origin are remote owners (`access: "device"`), so provider login and SSH-key upload work but owner-token minting does not; only local owners can mint invitations. Guest sharing is independent and session-limited. Radius remains a separate authenticated owner transport; a proxy marker is never authentication. **Devices → Sign out on this device** revokes this cookie, including its live connections (a local session's links stay: local sessions share one principal); other devices stay signed in. **Sign out all devices** rotates the signing key, ends every owner session, remote and local, with their live connections and outstanding links; local clients sign in again with the new key. Individual revocations persist in private `owner-revoked.json`. Already-admitted agent work continues. The first deployment of signed cookies rotates the former raw-key cookie secret once (`owner-cookie-version`): old raw-key cookies intentionally stop working. Generate fresh owner links after deploying. Outstanding invitations do not survive server restarts; a fresh headless token replaces `owner-token.txt`. Compatible plugin reloads retain them. ## Providers and SSH keys **Providers** is available to every owner (local, associated device or Radius), never guests. Provider-owned pi-ai flows support API keys and OAuth; credentials stay in the server's private `auth.json`. Remote connections prefer device-code methods where offered. For a browser-callback flow, complete sign-in in the provider tab and paste its code or final redirect URL into Providers. If OpenAI ChatGPT lands on an unreachable localhost page, copy its entire address from the address bar and paste it; the page need not load. Anthropic displays a code to paste. The provider's registered callback URL is unchanged. **Envs → Upload private key** also works for every owner, locally or remotely, never guests. Keys must be passphrase-less and at most 16 KiB; they are stored privately under `<data>/ssh/keys/`, never returned to a browser. Key generation and fetching an existing public key are available to every owner too, never guests. See [remote environments](docs/remote-envs.md). **Discord** (sidebar → More) sets up bots; a session's sheet → Discord links it to a channel: people there wake the sandboxed, locked session by mentioning the bot, and its answers come back as replies. See [Pi on Discord](docs/discord.md). API keys, pasted login codes and uploaded SSH keys pass through your tunnel or relay. Use a trusted transport. Keep the data directory, owner-key/tokens/revocations, provider credentials and SSH keys private; never commit them. ## Deploy behind a proxy (no HTTP basic auth) Keep p2 loopback-only. Preserve the public Host header and forward WebSocket upgrades. Configure `P2_ALLOWED_HOSTS=your-domain` and `P2_PUBLIC_URL=https://your-domain`. Overwrite `X-P2-Proxy-Secret` on **every** request with a constant marker, for example `1`. The historical header name remains, but its value has no authority and `P2_PROXY_SECRET` is no longer used. Any marked/forwarded localhost request is rejected rather than receiving device privileges, even when a Unix-socket bridge makes its network peer `127.0.0.1`. Allowed public hosts are remote regardless of the marker; without an owner/guest cookie visitors get only a minimal sign-in page and no application assets, sessions, WebSockets or file/upload access. Share links still work without owner sign-in. Example Caddyfile behind an existing HTTPS front end (the Unix-to-loopback bridge is unchanged): ```caddyfile :80 { reverse_proxy unix//p2/p2.sock { header_up X-P2-Proxy-Secret 1 flush_interval -1 } } ``` Remove `basic_auth` and the old secret injection. Do not rewrite the public Host to localhost. Ensure the public front end actually uses HTTPS; p2 treats allowed public hosts as HTTPS origins. Restart p2 to apply runtime/auth changes, then reload the proxy and mint a fresh owner link. Do not run competing servers against the same data dir. ## TUI sign-in The TUI launcher requires **Node >=26.5.1**; the server still supports Node 22.19+. On slayer use Node 26.11.1: `PATH="$HOME/node-v26.11.1-linux-x64/bin:$PATH" LD_LIBRARY_PATH="$HOME/node26-lib" npm run tui` (or invoke `~/.local/bin/node26 --import ./src/source.ts src/hosts/tui.ts`). ```sh npm run tui # local owner on http://127.0.0.1:$PORT (8788 when unset) npm run tui -- https://your-domain # prompts for an owner link if no valid cookie is saved npm run tui -- https://your-domain --owner-link 'https://your-domain/owner#<token>' npm run tui -- --radius # through Radius: your only online server, or --radius <server-id> ``` With `--radius` the TUI reaches a server signed in with `npm run radius:login`, from anywhere and without a tunnel or owner link. The first run signs the TUI in to Radius by device code (saved in `~/.pi/p2-tui/radius.json`); Radius admits only the server's owner. For now each request is its own Radius connection, so a new server bundle shows up within 10 seconds rather than one (see [p2 notes](docs/p2-notes.md#tui-through-radius)). On loopback the TUI signs in as the local owner with the data directory's owner key (see [Signing in on this machine](#signing-in-on-this-machine)), without a link prompt, and saves nothing. Without a readable key, or against a server with another data directory, it says why and asks for an owner link; a proxy marker cannot use local sign-in. For remote URLs (HTTPS required), get an owner link on the host, then pass `--owner-link` or paste it at the prompt. The TUI stores only its cookie per origin in `~/.pi/p2-tui/config.json` (0600, directory 0700; `P2_TUI_CONFIG_DIR` overrides it). It never stores the consumed invitation. `P2_TUI_OWNER_LINK` is an optional alternative to an argument. Links must match the selected origin and put the token in the fragment only. Expired/revoked credentials prompt again on the next launch. Basic-auth flags, environment and saved passwords are no longer used. If an active remote TUI is revoked, restart it and sign in with a fresh link. ### TUI isolation The server esbuilds one self-contained ESM containing pi-tui, the Chord client, the runtime and its selected `plugins/*/tui.ts` facets, only when their TUI dependencies are already selected (they never expand backend selection). The launcher fetches the authenticated `/__p2_tui` manifest and hash-addressed bundle; it never discovers or imports TUI facets from the visitor's checkout. Successful plugin reloads rebuild the bundle. A changed hash starts a fresh worker; only after readiness does the launcher swap, restore UI state and terminate the old worker. `/reload` requests a fresh worker too. Network drops reconnect to the same server. The unrestricted launcher alone owns terminal input/resize, WebSocket, sign-in and local cookie storage. A plain Worker runs with `execArgv: ["--permission"]`, `env: {}` and V8 resource limits. Its trusted bootstrap drops `fs.read`, `fs.write`, `worker`, `net`, `child`, `addon`, `wasi` and `inspector` **before** importing the bundle from an in-memory data URL. Worker stdout/stderr are captured and discarded. Heap snapshots/object queries, worker environment-data/BroadcastChannel access, parentPort exposure and process-signaling/report controls are also disabled. The entire worker fs/fs-promises function facade is closed: inherited numeric descriptors must not read stdin or bypass stdout/stderr capture. Permission drops still protect JSON/require/file module loading. Keybindings, theme and display settings are the visitor's own: the launcher reads pi's `~/.pi/agent/keybindings.json`, `settings.json` and `themes/*.json` plus `~/.pi/p2-tui/prefs.json`, validates a whitelisted subset and posts it to the worker as data (no paths, env or commands); the server's `TuiPreferences` (`<P2_DATA_DIR>/tui.json`) only supplies defaults below them and never privileged bindings. The worker never reads files. Clipboard writes, opening URLs, downloads, the external editor and Ctrl+Z are launcher actions under a grant (the visitor's own key, or a confirmation in a launcher-drawn screen showing the exact data); server code cannot pick a visitor path, read the clipboard or open a browser by itself. All worker output passes a tight 64 KiB/frame terminal filter: printable text, CR/LF/tab, SGR, cursor/erase, cursor visibility, synchronized output, bracketed paste, pi-tui's alternate screen and scroll region. OSC (including clipboard/title/hyperlinks), DCS, APC/PM/SOS, kitty graphics, C1 controls, unknown modes and incomplete escapes are stripped. Chord frames are relayed unchanged over the launcher's fixed-origin connection. **Accepted risk: denial of service.** Malicious server code may crash or exhaust its TUI process. V8 limits bound heap, not every native/ArrayBuffer allocation; a launcher watchdog samples process RSS and ArrayBuffers every 100 ms and terminates the worker above 512 MiB RSS or 128 MiB main-thread ArrayBuffers. RSS covers all threads; ArrayBuffer accounting is thread-local. Sampling is not a hard OS quota: rapid allocations/native work can beat it, and memory exhaustion can affect the host. Node/runtime vulnerabilities remain out of scope; keep Node patched. There is no OS sandbox. The intended authority boundary denies machine reads/writes, network, child processes, inherited environment and terminal escapes; a connected server retains full authority over its own workers. ### Owner testing on slayer From `/home/badlogic/pi/apps/p2` (no server restart needed): ```sh ~/.local/bin/node26 --import ./src/source.ts src/hosts/tui.ts # Loopback http://127.0.0.1:8788: local owner by the data directory's owner key, no prompt. npm run owner-link -- https://p2.mariozechner.at ~/.local/bin/node26 --import ./src/source.ts src/hosts/tui.ts https://p2.mariozechner.at # Paste the private printed link at the prompt. Ctrl+D exits. ``` Do not publish the owner link or paste it into logs. A valid saved remote cookie skips the prompt on later launches. ## Configuration Edit `<P2_DATA_DIR>/settings.json` as a file, or in the Settings window (the gear beside the sidebar's theme switch, owner only), which saves each change to the file at once. The window also holds Providers, MCP servers, Environments, Ports, Devices, Discord, Voice and Logs. Valid changes hot-reload without a restart, and in-flight requests keep their configuration. Set the default model and - apps/p3/docs/slices/10-agents.md (3666 bytes)
# Slice 10: Agents **Goal.** Subagents, one-shot agents, code mode, reaching other sessions (`conversations.reach`). p2's `plugins/subagents`, `plugins/codemode`, `plugins/conversations` as precedent. **Depends on:** slices 3, 8. **From the archive:** `archive/rundown/book.md` (Subagents and Places, The Worker and the People), `archive/locks/design.md` (`conversations.reach`, `subagents.*`). **Read first:** `decisions.md` ("Tool access", "Artifacts and views": plugin tools are code-mode only, by namespace); Chord decisions on subagents (`context: "inherit"`) and on Chord from code-mode scripts (the agent's principal, open); design.md §24 (code mode on reload); `archive/status-2026-11-09.md` (subagents choose from the caller's tools; instruction scopes); `packages/codemode/`. The brief is filled out when the slice is next. **Also read (docs audit, 2026-11-14; "Chord dec." is `packages/chord/docs/decisions.md` by entry title and line, "archive" is `../archive/`, "research" is `/home/badlogic/chord-notes/research/p3-people-2026-11-13/` on slayer):** - **Chord dec.:** 2413 (inherit, tree), 3552 (code mode scripts calling Chord), 110 (code mode sandbox close/R16). - `status-2026-11-09.md:60-62, 108-111`; `merged.md` §2 #17, §4 Q10; `handoff.md:65-67`; decisions "Artifacts and views" (code-mode-only plugin tools, namespaces from configuration); `rundown/book.md:1495-1509`. - **p2:** `apps/p2/plugins/agent/`, `agent-plugins/`, `apps/p2/plugins/subagents/contract.worker.ts` (`resolveTools`), `apps/p2/docs/agent-plugins.md`. **Open or to carry (docs audit):** - Agent-written plugins need the server's machine (a subagent with `env: null` and the plugins directory as cwd, `archive/rundown/book.md` 1495-1509); Chord from code-mode scripts needs a principal for the agent (Chord dec. 3552). ## Worker memory: what to build with code mode (owner, 2026-11-18) Measured by mem-audit on the realistic session worker of slice 26's memory test (Node 26.11.1, the committed worker flags), with code mode as p2 runs it; private: `/home/badlogic/chord-notes/research/p3-memory.md`, "Code mode and remote environments". USS, MiB, medians: | | idle | after the first script | after 20 scripts | after the burst | peak | runs over 55 | | --- | --- | --- | --- | --- | --- | --- | | no code mode | 21.7 | – | – | 36.2 | 51.2 | 0 of 6 | | p2's code mode (a worker thread per script) | 23.0 | 37.4 | 44.4 | 51.7 | 57.0 | 3 of 3 | | QuickJS inline, in the worker's thread | 21.8 | 38.2 | 38.6 | 47.5 | 52.8 | 0 of 3 | **To build (decided):** 1. **The sandbox runs inline** in the worker's own thread, not a worker thread per script: about 10 MiB less after the first script, 28 MiB less with five scripts at once, and a script takes 14 ms instead of 30. A runaway script holds the worker's thread until QuickJS's interrupt budget stops it; that budget is the guard. 2. **Opened on first use** (1.2 MiB less while unused). Not closed when idle: that frees nothing measurable and a reopen costs 21 to 37 ms. One sandbox per worker (per conversation costs the same: QuickJS compiles once per process). 3. **Look at what Durable keeps per code mode call:** after 21 code mode turns with 63 nested calls the worker holds about 8 MiB more heap. Inline code mode plus an SSH environment peaks at 51 to 58 MiB, about one worker per run over the 55 MiB budget; trimming those records is the first place to find the missing 3 MiB. If that fails, the budget for code mode sessions is decided again (60 MiB was proposed). 4. The release must ship code mode's sandbox with the worker's bundle (inline needs no separate worker file). - biome.json (1040 bytes)
{ "$schema": "https://biomejs.dev/schemas/2.3.5/schema.json", "linter": { "enabled": true, "rules": { "recommended": true, "style": { "noNonNullAssertion": "off", "useConst": "error", "useNodejsImportProtocol": "off" }, "suspicious": { "noExplicitAny": "off", "noControlCharactersInRegex": "off", "noEmptyInterface": "off" } } }, "formatter": { "enabled": true, "formatWithErrors": false, "indentStyle": "tab", "indentWidth": 3, "lineWidth": 120 }, "files": { "includes": [ "scripts/model-catalog-protocol.ts", "scripts/model-catalog-protocol.test.ts", "packages/*/src/**/*.ts", "packages/*/test/**/*.ts", "packages/ai/catalog/**/*.ts", "packages/coding-agent/examples/**/*.ts", "packages/agent/examples/**/*.ts", "!**/node_modules/**/*", "!**/test-sessions.ts", "!**/models.generated.ts", "!**/*.models.ts", "!packages/mom/data/**/*", "!!**/node_modules", "!!apps" ] }, "plugins": ["./scripts/biome/model-type-comparison.grit"] } - CONTRIBUTING.md (5032 bytes)
# Contributing to pi This guide exists to save both sides time. ## Philosophy First things first: **pi's core is minimal**. If your feature does not belong in the core, it should be an extension. PRs that bloat the core will likely be rejected. Pi's core exists to be minimal and to be extensible so that it can be influenced and manipulated by extensions. Even hook points for extensions however should be well considered and discussed to avoid adding unmaintainable bloat and complex interactions. ## The One Rule **You must understand your code.** If you cannot explain what your changes do and how they interact with the rest of the system, your PR will be closed. Using AI to write code is fine. Submitting AI-generated slop without understanding it is not. If you use an agent, run it from the `pi` root directory so it picks up `AGENTS.md` automatically. Your agent must follow the rules and guidelines in that file. ## Contribution Gate All issues and PRs from new contributors are auto-closed by default. Issues submitted Friday through Sunday are not guaranteed to be reviewed. If something is urgent, ask on Discord: https://discord.com/invite/3cU7Bz4UPx Maintainers review auto-closed issues daily and reopen worthwhile ones. Issues that do not meet the quality bar below will not be reopened or receive a reply. Approval happens through maintainer replies on issues: - `lgtmi`: your future issues will not be auto-closed - `lgtm`: your future issues and PRs will not be auto-closed The command must be at the start of the reply (optionally after one or more `@username` mentions) or at the end. `lgtmi` does not grant rights to submit PRs. Only `lgtm` grants rights to submit PRs. ## Quality Bar For Issues If you open an issue, you must use one of the two GitHub issue templates. If you open an issue, keep it short, concrete, and worth reading. - Keep it concise. If it does not fit on one screen, it is too long. - Write in your own voice (do not use an LLM to generate text, if you must, follow up with a clearly AI labeled comment). - State the bug or request clearly. - Explain why it matters. - If you want to implement the change yourself, say so. If the issue is real and written well, a maintainer may reopen it or reply with `lgtmi` or `lgtm` in the command position described above. ## Blocking If you ignore this document twice, or if you spam the tracker with agent-generated issues, your GitHub account will be permanently blocked. If you send a large volume of issues through automation, your GitHub account will be permanently blocked. No taksies backsies. ## Before Submitting a PR Do not open a PR unless you have already been approved by a maintainer using `lgtm` in the command position described above. Before submitting a PR: ```bash npm run check ./test.sh ``` Both must pass. Do not edit `CHANGELOG.md`. Changelog entries are added by maintainers. If you are adding a new provider to `packages/ai`, see `AGENTS.md` for required tests. ## Questions? Ask on [Discord](https://discord.com/invite/nKXTsAcmbT). ## FAQ ### Why are new issues and PRs auto-closed? pi receives more issues than the maintainers can responsibly review in real time. Many reports do not meet the quality bar in this guide or do not follow CONTRIBUTING.md. Some are slung at the repository mindlessly via an agent instead of being reviewed and shaped by the person submitting them. Auto-closing creates a buffer so maintainers can review the tracker on their own schedule and reopen the issues that meet the quality bar. ### Why are weekend issues lower priority? We triage the tracker during working hours. That means more issues can accumulate over the weekend. Anything submitted Friday through Sunday may be missed or given lower priority in the Monday review queue. If a problem is urgent, ask on Discord and include the short version, a repro, and the relevant logs. ### Why do some issues get no reply? A reply is maintenance work too. Low-signal issues, unclear reports, duplicates, and issues that do not follow this guide may be closed without discussion. This keeps time available for reproducible bugs, thoughtful requests, and contributors who have done the work to make their report actionable. ### Why not let AI triage everything? AI can help group duplicates, summarize reports, and spot missing information. It is not trusted to make final maintainer decisions. Polished AI-generated issues can still be wrong, misleading, or expensive to investigate. Human review remains the final gate. ### Is this hostile to contributors? No. It is a guardrail against burnout and tracker spam. Short, concrete, reproducible issues are welcome. Thoughtful contributions are welcome. Automated slop, entitlement, and large volumes of low-effort reports are not. ## Where can I learn about plans? Earendil uses RFCs to discuss larger changes. Not all of them are public, but quite a few are. They can be found at [rfc.earendil.com](https://rfc.earendil.com/keyword/pi).
- INSTRUCTIONS.md (177 bytes)
--- name: dynamic-resources description: Example skill loaded from resources_discover --- # Dynamic Resources Skill This skill is provided by the dynamic-resources extension.
- LICENSE (1069 bytes)
MIT License Copyright (c) 2025 Mario Zechner Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions: The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software. THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
- package.json (5127 bytes)
{ "name": "pi-monorepo", "private": true, "type": "module", "workspaces": [ "packages/*", "apps/*", "packages/chord/test/examples/61-tutorial", "packages/coding-agent/examples/extensions/with-deps", "packages/coding-agent/examples/extensions/custom-provider-anthropic", "packages/coding-agent/examples/extensions/gondolin" ], "scripts": { "clean": "npm run clean --workspaces --if-present", "build": "cd packages/telemetry && npm run build && cd ../schema && npm run build && cd ../chord && npm run build && cd ../tui && npm run build && cd ../codemode && npm run build && cd ../mcp && npm run build && cd ../ai && npm run build && cd ../durable && npm run build && cd ../env && npm run build && cd ../agent && npm run build && cd ../coding-agent && npm run build", "build:offline": "cd packages/telemetry && npm run build && cd ../schema && npm run build && cd ../chord && npm run build && cd ../tui && npm run build && cd ../codemode && npm run build && cd ../mcp && npm run build && cd ../ai && npm run build:offline && cd ../durable && npm run build && cd ../env && npm run build && cd ../agent && npm run build && cd ../coding-agent && npm run build", "build:native:darwin": "npm --prefix packages/tui run build:native:darwin", "build:native:linux": "npm --prefix packages/tui run build:native:linux", "build:native:win32": "npm --prefix packages/tui run build:native:win32", "check": "node scripts/biome-check.mjs --write && npm run check:pinned-deps && npm run check:runtime-deps && npm run check:ts-imports && npm run check:entry-graphs && npm run check:install-lock:coding-agent && tsc --noEmit && tsc --noEmit -p packages/coding-agent/examples/tsconfig.json && npm run check:browser-smoke", "check:browser-smoke": "node scripts/check-browser-smoke.mjs", "check:pinned-deps": "node scripts/check-pinned-deps.mjs", "check:runtime-deps": "node scripts/check-runtime-deps.mjs", "check:package-install": "node scripts/check-package-install.mjs", "check:released-changelogs": "node scripts/check-released-changelogs.mjs", "check:install-lock:coding-agent": "node scripts/generate-coding-agent-install-lock.mjs --check", "check:ts-imports": "node scripts/check-ts-relative-imports.mjs", "check:entry-graphs": "node scripts/check-entry-graphs.mjs", "generate:models": "npm --prefix packages/ai run generate-models", "hydrate:model-data": "npm --prefix packages/ai run hydrate-model-data", "check:model-data": "npm --prefix packages/ai run check:model-data", "generate:model-catalog": "npm --prefix packages/ai run generate-model-catalog", "diff:model-catalog": "node scripts/diff-model-catalog.mjs", "update:model-catalog-pin": "node scripts/update-model-catalog-pin.mjs", "eval": "npm run eval --workspace=@earendil-works/pi-evals --", "check:model-catalog": "node scripts/publish-model-catalog.mjs --input .artifacts/model-catalog --dry-run", "check:model-catalog-compat": "node scripts/check-model-catalog-compat.mjs", "profile:tui": "node scripts/profile-coding-agent-node.mjs --mode tui", "profile:rpc": "node scripts/profile-coding-agent-node.mjs --mode rpc", "test": "npm run test:scripts && npm run test --workspace packages --if-present", "test:scripts": "node --test scripts/*.test.mjs scripts/*.test.ts", "test:mcp-conformance": "node packages/coding-agent/test/mcp-conformance/run.ts", "version:patch": "npm version patch --workspace packages --no-git-tag-version --no-workspaces-update && node scripts/sync-versions.js && npm install --package-lock-only --ignore-scripts", "version:minor": "npm version minor --workspace packages --no-git-tag-version --no-workspaces-update && node scripts/sync-versions.js && npm install --package-lock-only --ignore-scripts", "version:major": "npm version major --workspace packages --no-git-tag-version --no-workspaces-update && node scripts/sync-versions.js && npm install --package-lock-only --ignore-scripts", "version:set": "npm version --workspace packages --no-workspaces-update", "prepublishOnly": "npm run clean && npm run build && npm run check", "publish": "npm run prepublishOnly && node scripts/publish.mjs", "publish:dry": "npm run prepublishOnly && node scripts/publish.mjs --dry-run", "release:local": "node scripts/local-release.mjs", "release:p3:local": "node --disable-warning=ExperimentalWarning apps/p3/scripts/release-local.ts", "pack:packages": "node scripts/pack-packages.mjs", "install-lock:coding-agent": "node scripts/generate-coding-agent-install-lock.mjs", "release:patch": "node scripts/release.mjs patch", "release:minor": "node scripts/release.mjs minor", "release:major": "node scripts/release.mjs major", "release:fix-links": "node scripts/release-notes.mjs fix-github-releases", "prepare": "husky" }, "devDependencies": { "@biomejs/biome": "2.3.5", "@biomejs/wasm-nodejs": "2.3.5", "@types/node": "22.19.19", "esbuild": "0.28.2", "husky": "9.1.7", "pnpm": "11.15.1", "typescript": "7.0.2", "yaml": "2.9.0" }, "engines": { "node": ">=22.19.0" }, "version": "0.0.3", "overrides": { "protobufjs": "7.6.6" } } - packages/coding-agent/examples/extensions/dynamic-resources/SKILL.md (177 bytes)
--- name: dynamic-resources description: Example skill loaded from resources_discover --- # Dynamic Resources Skill This skill is provided by the dynamic-resources extension.
- packages/coding-agent/examples/extensions/subagent/agents/planner.md (896 bytes)
--- name: planner description: Creates implementation plans from context and requirements tools: read, grep, find, ls model: claude-sonnet-4-5 --- You are a planning specialist. You receive context (from a scout) and requirements, then produce a clear implementation plan. You must NOT make any changes. Only read, analyze, and plan. Input format you'll receive: - Context/findings from a scout agent - Original query or requirements Output format: ## Goal One sentence summary of what needs to be done. ## Plan Numbered steps, each small and actionable: 1. Step one - specific file/function to modify 2. Step two - what to add/change 3. ... ## Files to Modify - `path/to/file.ts` - what changes - `path/to/other.ts` - what changes ## New Files (if any) - `path/to/new.ts` - purpose ## Risks Anything to watch out for. Keep the plan concrete. The worker agent will execute it verbatim.
- packages/coding-agent/examples/extensions/subagent/agents/reviewer.md (933 bytes)
--- name: reviewer description: Code review specialist for quality and security analysis tools: read, grep, find, ls, bash model: claude-sonnet-4-5 --- You are a senior code reviewer. Analyze code for quality, security, and maintainability. Bash is for read-only commands only: `git diff`, `git log`, `git show`. Do NOT modify files or run builds. Assume tool permissions are not perfectly enforceable; keep all bash usage strictly read-only. Strategy: 1. Run `git diff` to see recent changes (if applicable) 2. Read the modified files 3. Check for bugs, security issues, code smells Output format: ## Files Reviewed - `path/to/file.ts` (lines X-Y) ## Critical (must fix) - `file.ts:42` - Issue description ## Warnings (should fix) - `file.ts:100` - Issue description ## Suggestions (consider) - `file.ts:150` - Improvement idea ## Summary Overall assessment in 2-3 sentences. Be specific with file paths and line numbers.
- packages/coding-agent/examples/extensions/subagent/agents/scout.md (1255 bytes)
--- name: scout description: Fast codebase recon that returns compressed context for handoff to other agents tools: read, grep, find, ls, bash model: claude-haiku-4-5 --- You are a scout. Quickly investigate a codebase and return structured findings that another agent can use without re-reading everything. Your output will be passed to an agent who has NOT seen the files you explored. Thoroughness (infer from task, default medium): - Quick: Targeted lookups, key files only - Medium: Follow imports, read critical sections - Thorough: Trace all dependencies, check tests/types Strategy: 1. grep/find to locate relevant code 2. Read key sections (not entire files) 3. Identify types, interfaces, key functions 4. Note dependencies between files Output format: ## Files Retrieved List with exact line ranges: 1. `path/to/file.ts` (lines 10-50) - Description of what's here 2. `path/to/other.ts` (lines 100-150) - Description 3. ... ## Key Code Critical types, interfaces, or functions: ```typescript interface Example { // actual code from the files } ``` ```typescript function keyFunction() { // actual implementation } ``` ## Architecture Brief explanation of how the pieces connect. ## Start Here Which file to look at first and why. - packages/coding-agent/examples/extensions/subagent/agents/worker.md (664 bytes)
--- name: worker description: General-purpose subagent with full capabilities, isolated context model: claude-sonnet-4-5 --- You are a worker agent with full capabilities. You operate in an isolated context window to handle delegated tasks without polluting the main conversation. Work autonomously to complete the assigned task. Use all available tools as needed. Output format when finished: ## Completed What was done. ## Files Changed - `path/to/file.ts` - what changed ## Notes (if any) Anything the main agent should know. If handing off to another agent (e.g. reviewer), include: - Exact file paths changed - Key functions/types touched (short list)
- pi-test.sh (2455 bytes)
#!/usr/bin/env bash set -euo pipefail SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)" # Check for --no-env flag NO_ENV=false ARGS=() for arg in "$@"; do if [[ "$arg" == "--no-env" ]]; then NO_ENV=true else ARGS+=("$arg") fi done if [[ "$NO_ENV" == "true" ]]; then # Unset API keys (see packages/ai/src/env-api-keys.ts) unset ANTHROPIC_API_KEY unset ANTHROPIC_OAUTH_TOKEN unset OPENAI_API_KEY unset GEMINI_API_KEY unset GROQ_API_KEY unset CEREBRAS_API_KEY unset XAI_API_KEY unset OPENROUTER_API_KEY unset ZAI_API_KEY unset MISTRAL_API_KEY unset MINIMAX_API_KEY unset MINIMAX_CN_API_KEY unset AI_GATEWAY_API_KEY unset OPENCODE_API_KEY unset COPILOT_GITHUB_TOKEN unset GH_TOKEN unset GITHUB_TOKEN unset HF_TOKEN unset GOOGLE_APPLICATION_CREDENTIALS unset GOOGLE_CLOUD_PROJECT unset GCLOUD_PROJECT unset GOOGLE_CLOUD_LOCATION unset AWS_PROFILE unset AWS_ACCESS_KEY_ID unset AWS_SECRET_ACCESS_KEY unset AWS_SESSION_TOKEN unset AWS_REGION unset AWS_DEFAULT_REGION unset AWS_BEARER_TOKEN_BEDROCK unset AWS_CONTAINER_CREDENTIALS_RELATIVE_URI unset AWS_CONTAINER_CREDENTIALS_FULL_URI unset AWS_WEB_IDENTITY_TOKEN_FILE unset AZURE_OPENAI_API_KEY unset AZURE_OPENAI_BASE_URL unset AZURE_OPENAI_RESOURCE_NAME unset ANTHROPIC_AUTH_TOKEN unset ANTHROPIC_FEDERATION_RULE_ID unset ANTHROPIC_IDENTITY_TOKEN_FILE unset ANTHROPIC_ORGANIZATION_ID unset ANTHROPIC_SERVICE_ACCOUNT_ID unset ANTHROPIC_WORKSPACE_ID unset ANT_LING_API_KEY unset BASETEN_API_KEY unset CLOUDFLARE_API_KEY unset DEEPSEEK_API_KEY unset FIREWORKS_API_KEY unset GOOGLE_CLOUD_API_KEY unset KIMI_API_KEY unset META_API_KEY unset MOONSHOT_API_KEY unset NVIDIA_API_KEY unset QWEN_TOKEN_PLAN_API_KEY unset QWEN_TOKEN_PLAN_CN_API_KEY unset RADIUS_API_KEY unset TOGETHER_API_KEY unset TYPESAFE_API_KEY unset XIAOMI_API_KEY unset XIAOMI_TOKEN_PLAN_CN_API_KEY unset XIAOMI_TOKEN_PLAN_AMS_API_KEY unset XIAOMI_TOKEN_PLAN_SGP_API_KEY unset ZAI_CODING_CN_API_KEY echo "Running without API keys..." fi # --import takes a module specifier, so pass the resolver as a file URL (raw paths break on #, ?, %). RESOLVER_URL="$(node -p 'require("node:url").pathToFileURL(process.argv[1]).href' "$SCRIPT_DIR/packages/coding-agent/src/experimental/source-resolver.ts")" node --import "$RESOLVER_URL" "$SCRIPT_DIR/packages/coding-agent/src/cli.ts" ${ARGS[@]+"${ARGS[@]}"} - README.md (10679 bytes)
<p align="center"> <a href="https://pi.dev"> <img alt="pi logo" src="https://pi.dev/logo-auto.svg" width="128"> </a> </p> <p align="center"> <a href="https://discord.com/invite/3cU7Bz4UPx"><img alt="Discord" src="https://img.shields.io/badge/discord-community-5865F2?style=flat-square&logo=discord&logoColor=white" /></a> <a href="https://www.npmjs.com/package/@earendil-works/pi-coding-agent"><img alt="npm" src="https://img.shields.io/npm/v/@earendil-works/pi-coding-agent?style=flat-square" /></a> </p> > New issues and PRs from new contributors are auto-closed by default. Maintainers review auto-closed issues daily. See [CONTRIBUTING.md](CONTRIBUTING.md). # Pi Pi is a minimal, extensible agent harness that you can make your own. Adapt Pi to your workflows, not the other way around. Customize Pi with [extensions](packages/coding-agent/docs/extensions.md), [skills](packages/coding-agent/docs/skills.md), [prompt templates](packages/coding-agent/docs/prompt-templates.md), and [themes](packages/coding-agent/docs/themes.md). Bundle them as [Pi packages](packages/coding-agent/docs/packages.md) and share via npm or git. Pi ships with powerful defaults but skips features like sub-agents and plan mode. Ask Pi to build what you want, or install a package that does it your way. Use Pi [interactively](packages/coding-agent/docs/usage.md), automate it in [print or JSON mode](packages/coding-agent/docs/cli.md), control it over [RPC](packages/coding-agent/docs/rpc.md), or build apps with the [Pi TypeScript SDK](packages/coding-agent/docs/sdk.md). See [OpenClaw](https://github.com/OpenClaw/OpenClaw) for a real-world integration. ## Getting started Install the command-line interface: ```bash curl -fsSL https://pi.dev/install.sh | sh ``` On Windows: ```shell powershell -c "irm https://pi.dev/install.ps1 | iex" ``` The installer pins all dependencies and updates Pi with `pi update`. Alternatively, install directly with npm, which does not pin transitive dependencies: ```bash npm install -g --ignore-scripts @earendil-works/pi-coding-agent ``` Pi requires Node.js 22.19 or newer. The macOS, Linux, and Windows installers can install it if needed. Pi does not require dependency lifecycle scripts for a normal npm installation. Start Pi in the directory where you want it to work: ```bash cd /path/to/project pi ``` For a built-in AI provider, run `/login` inside Pi to connect a subscription or API key. Then give Pi a task. See the [documentation](https://pi.dev/docs/latest) for full setup and usage instructions, or [visit pi.dev](https://pi.dev) for demos. ## Run with Nix ```bash nix run github:earendil-works/pi/stable ``` `stable` points at the latest release. Install it with `nix profile add github:earendil-works/pi/stable` and update with `nix profile upgrade pi`. Use a release tag such as `github:earendil-works/pi/v1.0.0` to pin a version, or `github:earendil-works/pi` for unreleased changes on `main`. Nix builds Pi from source. Supports ARM64 and x86-64 on Linux and macOS. Use `nix build .` or `nix run .` to build or run your checkout. Nix builds are offline, so the bundled model data comes from a pi.dev model catalog revision pinned in `nix/model-catalog.json`. At runtime, Pi still overlays newer catalog data from pi.dev as usual. The Nix workflow replaces the pin on `main` when it no longer matches the checkout, for example after a provider is added or gains a new model type. To refresh it by hand: ```bash npm run update:model-catalog-pin ``` ## Packages This monorepo contains the Pi CLI and its supporting libraries. | Package | Description | |---------|-------------| | **[@earendil-works/chord](packages/chord)** | Object runtime for apps across processes: contracts, documents, a store, links, plugins with hot reload, HTTP, workers, web and terminal clients | | **[@earendil-works/pi-telemetry](packages/telemetry)** | Vendor-neutral telemetry contracts, reference adapter, conformance tests, and typed schemas | | **[@earendil-works/pi-ai](packages/ai)** | Unified multi-provider LLM API (OpenAI, Anthropic, Google, etc.) | | **[@earendil-works/pi-durable](packages/durable)** | Durable conversation, task, and document runtime | | **[@earendil-works/pi-agent-core](packages/agent)** | Agent runtime with tool calling and state management | | **[@earendil-works/pi-coding-agent](packages/coding-agent)** | Interactive coding agent CLI | | **[@earendil-works/pi-tui](packages/tui)** | Terminal UI library with differential rendering | For Slack/chat automation and workflows see [earendil-works/pi-chat](https://github.com/earendil-works/pi-chat). ## Permissions & Containerization Pi does not include a built-in permission system for restricting filesystem, process, network, or credential access. By default, it runs with the permissions of the user and process that launched it. If you need stronger boundaries, containerize or sandbox Pi. See [packages/coding-agent/docs/containerization.md](packages/coding-agent/docs/containerization.md) for three patterns: - **Gondolin extension**: keep `pi` and provider auth on the host while routing built-in tools and `!` commands into a local Linux micro-VM. - **Plain Docker**: run the whole `pi` process in a local container for simple isolation. - **OpenShell**: run the whole `pi` process in a policy-controlled sandbox. ## Contributing See [CONTRIBUTING.md](CONTRIBUTING.md) for contribution guidelines and [AGENTS.md](AGENTS.md) for project-specific rules (for both humans and agents). Longer term plans for Pi can also be found in [RFCs](https://rfc.earendil.com/keyword/pi). ## Development ```bash npm install --ignore-scripts # Install all dependencies without running lifecycle scripts npm run build # Refresh model data, then build all packages npm run build:offline # Rebuild using existing model data without network access npm run check # Lint, format, and type check ./test.sh # Run tests (skips LLM-dependent tests without API keys) ./pi-test.sh # Run pi from sources (can be run from any directory) ``` ### Using local packages outside the monorepo Build every public package into one coherent local artifact set: ```bash npm run pack:packages -- --out .artifacts/pi-packages ``` This refreshes model data before building `pi-ai`. To avoid network access when model data is already hydrated, pass `--offline-model-data`. Then configure an external project to consume one package and resolve all of its Pi dependencies from the same artifact set. npm is the default: ```bash node scripts/use-local-packages.mjs \ --manifest .artifacts/pi-packages/manifest.json \ --consumer ../my-project \ --package @earendil-works/pi-durable \ --package @earendil-works/pi-agent-core cd ../my-project npm install --ignore-scripts ``` For a pnpm project, point `--consumer` at the workspace root: ```bash node scripts/use-local-packages.mjs \ --manifest .artifacts/pi-packages/manifest.json \ --consumer ../my-project \ --package @earendil-works/pi-agent-core \ --package-manager pnpm cd ../my-project pnpm install --ignore-scripts ``` Repeat `--package` for each direct dependency. The command updates the consumer's `package.json` with content-addressed local `file:` references. It writes transitive overrides to `package.json` for npm or `pnpm-workspace.yaml` for pnpm. Keep the artifact directory available while installing or updating the consumer. Re-run both commands after changing Pi source. ## Building standalone binaries from release source GitHub releases include a versioned source archive covered by the release's `SHA256SUMS` file. Extract it and run the same build script used for the official standalone binaries: ```bash VERSION="<release-version>" tar -xzf "pi-${VERSION}-source.tar.gz" cd "pi-${VERSION}" ./scripts/build-binaries.sh --offline-model-data --platform linux-x64 --out "$PWD/out" ``` The archive includes release model data and native prebuilds. `--offline-model-data` uses that model data without refreshing provider catalogs. The script installs dependencies and builds the executable with its runtime assets; pass `--skip-install` if dependencies are already provided. ## Supply-chain hardening We treat npm dependency changes as reviewed code changes. - Direct external dependencies are pinned to exact versions. Internal workspace packages remain version-ranged. - `.npmrc` sets `save-exact=true` and `min-release-age=2` to avoid same-day dependency releases during npm resolution. - `package-lock.json` is the dependency ground truth. Pre-commit blocks accidental lockfile commits unless `PI_ALLOW_LOCKFILE_CHANGE=1` is set. - `npm run check` verifies pinned direct deps, native TypeScript import compatibility, and the generated coding-agent install lock. - The pi.dev installer installs from `packages/coding-agent/install-lock/`, generated from the root lockfile, to pin transitive deps. The npm package does not pin transitive deps. - Local release smoke tests and npm publication use the same tarball packer; npm publishes the validated tarballs rather than repacking workspace directories. - Local release installs, documented npm installs, and `pi update --self` use `--ignore-scripts` where supported. - CI installs with `npm ci --ignore-scripts`, and a scheduled GitHub workflow runs `npm audit --omit=dev` plus `npm audit signatures --omit=dev`. - Install lock generation has an explicit allowlist for dependency lifecycle scripts; new lifecycle-script deps fail checks until reviewed. ## Share your OSS coding agent sessions If you use Pi or other coding agents for open source work, please share your sessions. Public OSS session data helps improve coding agents with real-world tasks, tool use, failures, and fixes instead of toy benchmarks. For the full explanation, see [this post on X](https://x.com/badlogicgames/status/2037811643774652911). To publish sessions, use [`badlogic/pi-share-hf`](https://github.com/badlogic/pi-share-hf). Read its README.md for setup instructions. All you need is a Hugging Face account, the Hugging Face CLI, and `pi-share-hf`. You can also watch [this video](https://x.com/badlogicgames/status/2041151967695634619), where I show how I publish my `pi-mono` sessions. I regularly publish my own `pi-mono` work sessions here: - [badlogicgames/pi-mono on Hugging Face](https://huggingface.co/datasets/badlogicgames/pi-mono) ## License MIT <p align="center"> <a href="https://pi.dev">pi.dev</a> domain graciously donated by <br /><br /> <a href="https://exe.dev"><img src="packages/coding-agent/docs/images/exy.png" alt="Exy mascot" width="48" /><br />exe.dev</a> </p> - SECURITY.md (4227 bytes)
# Security Policy This document should guide you about understanding the security concept behind Pi and also where the boundaries are. In general Pi is a coding agent that runs locally within the security boundary of the user that is running it. It's the responsibility of the user to monitor its operations or to contain it within a container, virtual machine or other Sandbox solution. Pi treats the local user account and files writable by that account as inside the same trust boundary as the Pi process itself. If an attacker can modify files under the user's home directory, workspace, shell startup files, environment, or Pi configuration, they can generally influence Pi or other local developer tools. Reports that depend on such prior local write access are not security vulnerabilities unless they demonstrate how Pi grants that write access or crosses an operating-system privilege boundary. Pi relies on users installing trustworthy extensions and loading trustworthy skills and only to use pi within trusted repositories. This is because files like `AGENTS.md` or instructions in comments can be used to prompt inject the coding agent trivially and this cannot be protected against. ## Reporting a Vulnerability If you believe you found a security vulnerability in pi or another package in this repository, please report it privately by either: - Emailing `[email protected]`, or - Opening a private report through GitHub Security Advisories for this repository Please include: - A description of the issue and its impact - Steps to reproduce, proof of concept, or relevant logs - Affected package, version, commit, or configuration - Any known mitigations Do not open a public issue for security-sensitive reports. We will review reports and coordinate disclosure as appropriate. ## Scope Security issues in the distributed packages, command-line tools, APIs, and repository code are in scope as well as earendil operated infrastructure on `pi.dev`. ## Out Of Scope - Local code execution or sandboxing behavior (the Pi coding agent intentionally does not have a sandbox) - Behavior of pi extensions or skills installed by the user - Risks from working in untrusted repositories - Risks from installing untrusted extensions, skills, packages, or tools - Isuses caused by non trustworthy MITM proxies - Public internet exposure of a Pi installation - Prompt injection attacks - Exposed secrets that are third-party/user-controlled credentials - Reports requiring the ability to create, modify, delete, or replace files, directories, symlinks, environment variables, shell configuration, or other user-controlled local state on the target machine. This includes `~/.pi`, `~/.pi/agent/models.json`, workspace files, `AGENTS.md`, skills, extensions, extension configuration, dotfiles, and files synchronized through NFS, roaming profiles, or dotfile managers, unless the report shows how Pi itself grants that access. - Issues caused by intentionally weakened user configuration. - Resource/DOS claims that require trusted local input/config against the pi coding agent. - Reports about malicious model output. - User-approved or user-initiated local actions presented as vulnerabilities. ## Notes for Reporters The most useful reports show a current, reproducible security boundary bypass with demonstrated impact. Reports that only show expected local-agent behavior, prompt injection, or a malicious trusted extension/skill are not security vulnerabilities under this model. For example, a report showing that malicious contents written to a trusted Pi configuration file cause Pi to execute commands, load attacker-controlled tools, send credentials to an attacker-controlled endpoint, or otherwise change behavior is out of scope. When possible, include the exact affected path, package version or commit SHA, configuration, and a proof of concept against the latest release or latest `main`. For dependency reports, include evidence that the shipped dependency is affected and that the issue is reachable through Pi. For exposed-secret reports, include evidence that the credential is owned by Earendil or grants access to Earendil-operated infrastructure or services.
- SKILL.md (7311 bytes)
--- name: writing-p2-plugins description: Write, apply and show a p2 plugin (shared or persistent views, multiplayer games, boards, editors several people use, server-side objects, tools for the agent). Use when the user wants something multiplayer, persistent or with rules inside p2; html artifacts are for single-player pages. --- # Writing p2 plugins A p2 plugin is a directory of TypeScript files that p2 builds and loads into its server, every session's worker and the page. Plugins you write live in the **user plugin root**, `<data>/plugins/` on the p2 server (the `local:` directory this skill's subagent works in). They survive p2 updates; a built-in plugin wins a name clash. Use a plugin for anything multiplayer, persistent or with rules: games, shared boards, editors several people use. An html artifact (`artifact_write` with kind `html`, a page you write) is for single-player pages. ## Where to work Work on the server's own files: as a subagent located at `local:<data>/plugins` (subagent `env: null`, `cwd` the root), or in a session there. Write files with the ordinary `write`/`edit`/`read`/`bash` tools. Saving applies nothing; `reload_plugins` does. ## Layout ``` <name>/ lowercase letters, digits, dashes; not a built-in plugin's name package.json { "name": "<name>", "private": true, "type": "module" } (name = directory) contract.ts the types the other files share (no facet: imported by them) server.ts runs on the server (shared state, rules) worker.ts optional: runs in each session's worker (tools for the agent) web.tsx optional: runs in the page (React panes) skills/<skill>/SKILL.md optional: skills for agents on the server's machine (see below) ``` Imports: only `@p2/app/v1` (any facet), `@p2/app/v1/server` (server), `@p2/app/v1/worker` (worker), `@p2/app/v1/web` (web), `@earendil-works/pi-schema`, `@earendil-works/pi-durable` (worker, for `defineExtension`), `react` (web), and the plugin's own files by relative path. No other packages, no top-level await, no `import.meta`. `pluginFile(path)` from `@earendil-works/chord/plugins` gives a facet the absolute path of a file of its own plugin. ## The pieces - **Contract** (`contract.ts`): `remote(name, { version, params, hostAddress: serverHost, variants })`. Each variant has `state` (JSON schemas from `@earendil-works/pi-schema`), `methods` (`method(args, result)`) and `events` (fire-and-forget values for motion: pointers, drags, game inputs; never kept or replayed). Every change of a contract's shape bumps its `version`. - **Server object** (`server.ts`): `export default defineFacet((host) => { host.serve(Contract, (params, principal, self) => ({ variant, members })) })`. Check the principal: `canView`/`canSend(principal, session)` for people, `isWorker(principal) && principal.session === session` for the session's agent. Persistent data lives in a local store served once (`local(name, { version, params: {}, members })` with `self.state(defineState(name, { schema, version, initial, keyed?, persist: true, checkpointWhen }))`, changed with `handle.change(key, draft => ..., ctx)`); each person's object reaches it with `self.get(Store, {})` and shows derives (`self.derive([sources], fn)`). Data that belongs to one session is keyed by the session's address (`keyed: Session` from `@p2/app/v1`, keys `Session({ id })`): a session's export, import and delete carry it. Events: `self.events<T>()`, `.emit(value)`; listening to a reference's event member: `await ref.member.on(listener, ctx)`. Presence: `host.references(Contract)` lists who holds which object now. - **Pane** (`web.tsx`): `contribute(host, "<name>.web", (self) => self.contribute(ViewKinds, { <kind>: { Pane } }))`. The pane gets `PaneProps` (`session`, `item` (the view's item: what it shows, e.g. a board's ID), `inline` (true in the transcript, false beside it), `view` (beside the transcript), `editable`, `select`, `annotations`) and holds objects with `useObject(Contract, params)`, `liveIn(ref, variant)`, `useReadable(member)`. - **One script per plugin:** each plugin's web facet is built into a script of its own, and a file it imports from another plugin is copied into it. Never share module state (a `Map`, a `let`, a store, a React context) through such a file: the other plugin's copy is another instance. Use a registry the owning plugin serves, or `sharedContext(name, initial)` from `@p2/app/v1/web` for a React context. A plugin's web change applies in open pages in place (no reload). - **Listable items** (`web.tsx`, optional): a plugin keeping several items per session (boards, games) contributes an artifact kind, `self.contribute(ArtifactKinds, { <kind>: { title, icon, useItems, useCount, view, useRemove? } })` (`ArtifactKinds` from `@p2/app/v1/web`): the artifacts button counts them, its sheet lists them, and a row opens the item as a view of kind `view` with the item's ID. - **Tools** (`worker.ts`): `defineExtension({ name, namespace, tools: [defineP2Tool({...})] })` contributed to `Extensions`. A plugin's tool is **code-mode only by default**: give it `callers: ["tools"]`, so it stays out of the model's tool list and scripts call it (`await tools.<name>(args)` in `codemode`). The extension's `namespace` (`{ name, description, instructions? }`) groups its tools in code mode: the prompt lists the namespace's name and one-line description, and `describeNamespace(name)` gives its instructions and tools. To offer a tool in the main list instead, leave out `callers`; do that only for a tool the agent needs in most turns. - **Skills**: a worker facet contributes its plugin's skills directory: `self.contribute(PluginSkills, { <name>: { dir: pluginFile("skills") } })` (`PluginSkills` from `@p2/app/v1/worker`). Agents on the server's machine see them in `<available_skills>` after the environment's and the project's skills; agents on other environments do not (the files stay on the server). ## Apply and show 1. `reload_plugins` (no arguments: every changed plugin and every plugin of the user root; or `{"plugins": [names]}`) builds and rolls out to the server, every session's worker and the page (open pages reload onto the new bundle by themselves). It reports per plugin: `applied`, `removed`, `failed` with the errors as `file:line:column: message`, or `refused` with the reason (an invalid `package.json`, a built-in's name), and errors logged meanwhile. A failed build applies nothing; the old code keeps running. Fix and reload again. 2. Show the pane: in code mode, `await tools.view_open({ kind, title, item? })` (`kind` the name your web facet registered, `item` what it shows). A kind no loaded plugin registers shows "Unknown view kind". Edits reload in place: objects people hold go on with the new code. ## Complete example `example/counter/` next to this file is a complete, tested plugin: a shared counter with presence (who has the pane open), a wave event, a pane and a code-mode tool in its namespace. Copy it to `<root>/counter/` (`cp -r`), run `reload_plugins`, then in code mode `await tools.view_open({ kind: "counter", title: "Counter" })`. Read its files before writing your own. - test.sh (2801 bytes)
#!/usr/bin/env bash set -euo pipefail # Isolate user resources, credentials, temporary files, and tool configuration. temp_parent="${TMPDIR:-/tmp}" temp_parent="${temp_parent%/}" test_root="$(mktemp -d "$temp_parent/pi-test.XXXXXX")" git_askpass="$(type -P false)" readonly temp_parent test_root git_askpass mkdir -p "$test_root/home/.config" "$test_root/tmp" "$test_root/cache/npm" # Mark the generated root so cleanup can verify ownership before deleting it. touch "$test_root/.pi-test-owned" "$test_root/npm-userconfig" "$test_root/npm-globalconfig" # Only remove the marked directory created above, never an unverified path. cleanup() { local status=$? trap - EXIT case "$test_root" in "$temp_parent"/pi-test.*) if [[ -d "$test_root" && ! -L "$test_root" && -f "$test_root/.pi-test-owned" ]]; then rm -rf -- "$test_root" else printf "Refusing to remove unverified test directory: %s\n" "$test_root" >&2 [[ $status -ne 0 ]] || status=1 fi ;; *) printf "Refusing to remove unexpected test directory: %s\n" "$test_root" >&2 [[ $status -ne 0 ]] || status=1 ;; esac exit "$status" } trap cleanup EXIT # Start from an empty environment and allow only required platform and test settings. test_env=( "PATH=$PATH" "PWD=$PWD" "HOME=$test_root/home" "USERPROFILE=$test_root/home" "TMPDIR=$test_root/tmp" "TMP=$test_root/tmp" "TEMP=$test_root/tmp" "XDG_CONFIG_HOME=$test_root/home/.config" "XDG_CACHE_HOME=$test_root/cache" "LANG=C" "LC_ALL=C" "TZ=UTC" "GIT_CONFIG_NOSYSTEM=1" "GIT_CONFIG_GLOBAL=/dev/null" "GIT_TERMINAL_PROMPT=0" "GIT_ASKPASS=$git_askpass" "GIT_EDITOR=true" "GIT_SEQUENCE_EDITOR=true" "NPM_CONFIG_USERCONFIG=$test_root/npm-userconfig" "NPM_CONFIG_GLOBALCONFIG=$test_root/npm-globalconfig" "NPM_CONFIG_CACHE=$test_root/cache/npm" "PI_NO_LOCAL_LLM=1" "AWS_EC2_METADATA_DISABLED=true" ) # Native Windows needs these inherited values to launch child processes. for name in SystemRoot SYSTEMROOT WINDIR COMSPEC PATHEXT; do value="${!name-}" [[ -z "$value" ]] || test_env+=("$name=$value") done # Trust extra certificates of proxies that inspect HTTPS (corporate proxies, sandboxes). # Network tests fail with self-signed certificate errors without it. value="${NODE_EXTRA_CA_CERTS-}" [[ -z "$value" ]] || test_env+=("NODE_EXTRA_CA_CERTS=$value") # The pi-env tests build their daemon with cargo, whose toolchains live under the real home. test_env+=("RUSTUP_HOME=${RUSTUP_HOME:-$HOME/.rustup}" "CARGO_HOME=${CARGO_HOME:-$HOME/.cargo}") # Preserve CI detection only for runner behavior and test reporting. for name in CI GITHUB_ACTIONS; do value="${!name-}" [[ -z "$value" ]] || test_env+=("$name=$value") done echo "Running tests without API keys in isolated home: $test_root/home" env -i "${test_env[@]}" npm test - tsconfig.base.json (554 bytes)
{ "compilerOptions": { "target": "ES2024", "module": "Node16", "lib": ["ES2024"], "strict": true, "erasableSyntaxOnly": true, "verbatimModuleSyntax": true, "esModuleInterop": true, "skipLibCheck": true, "forceConsistentCasingInFileNames": true, "declaration": true, "declarationMap": true, "sourceMap": true, "inlineSources": true, "inlineSourceMap": false, "moduleResolution": "Node16", "resolveJsonModule": true, "allowImportingTsExtensions": true, "rewriteRelativeImportExtensions": true, "types": ["node"] } } - tsconfig.json (5520 bytes)
{ "extends": "./tsconfig.base.json", "compilerOptions": { "noEmit": true, "module": "NodeNext", "moduleResolution": "NodeNext", "paths": { "@earendil-works/chord": ["./packages/chord/src/index.ts"], "@earendil-works/chord/context": ["./packages/chord/src/context/index.ts"], "@earendil-works/chord/delta": ["./packages/chord/src/delta/index.ts"], "@earendil-works/chord/json": ["./packages/chord/src/json.ts"], "@earendil-works/chord/watch": ["./packages/chord/src/watch.ts"], "@earendil-works/chord/output": ["./packages/chord/src/output.ts"], "@earendil-works/chord/sqlite": ["./packages/chord/src/sqlite/index.ts"], "@earendil-works/chord/sqlite/node": ["./packages/chord/src/sqlite/node.ts"], "@earendil-works/chord/sqlite/cloudflare": ["./packages/chord/src/sqlite/cloudflare.ts"], "@earendil-works/chord/telemetry-sqlite": ["./packages/chord/src/telemetry-sqlite/index.ts"], "@earendil-works/chord/telemetry-sqlite/node": ["./packages/chord/src/telemetry-sqlite/node.ts"], "@earendil-works/chord/telemetry-ingest": ["./packages/chord/src/telemetry-ingest/index.ts"], "@earendil-works/chord/http": ["./packages/chord/src/node/http/index.ts"], "@earendil-works/chord/workers": ["./packages/chord/src/node/workers/index.ts"], "@earendil-works/chord/compression": ["./packages/chord/src/node/compression.ts"], "@earendil-works/chord/workers/child": ["./packages/chord/src/node/workers/child.ts"], "@earendil-works/chord/plugins": ["./packages/chord/src/node/plugins/index.ts"], "@earendil-works/chord/plugins/runtime": ["./packages/chord/src/node/plugins/runtime.ts"], "@earendil-works/chord/app": ["./packages/chord/src/node/app/index.ts"], "@earendil-works/chord/app/files": ["./packages/chord/src/node/app/files-entry.ts"], "@earendil-works/chord/web": ["./packages/chord/src/web/index.ts"], "@earendil-works/chord/surfaces": ["./packages/chord/src/node/surfaces/index.ts"], "@earendil-works/chord/react": ["./packages/chord/src/react/index.ts"], "@earendil-works/chord/tui": ["./packages/chord/src/tui/index.ts"], "@earendil-works/chord/tui/launcher": ["./packages/chord/src/node/tui/index.ts"], "@earendil-works/chord/sync": ["./packages/chord/src/sync/index.ts"], "@acme/app/*": ["./packages/chord/test/examples/49-plugin-api/app/*.ts"], "@notes/app/*": ["./packages/chord/test/examples/53-app/app/*.ts"], "@board/app/*": ["./packages/chord/test/examples/56-web/app/*.ts"], "@chat/app/*": ["./packages/chord/test/examples/59-chat/app/*.ts"], "@tally/app/*": ["./packages/chord/test/examples/62-telemetry/app/*.ts"], "@earendil-works/pi-telemetry": ["./packages/telemetry/src/index.ts"], "@earendil-works/pi-telemetry/testing": ["./packages/telemetry/src/testing/index.ts"], "@earendil-works/pi-telemetry/render": ["./packages/telemetry/src/render.ts"], "@earendil-works/pi-codemode": ["./packages/codemode/src/index.ts"], "@earendil-works/pi-codemode/cloudflare": ["./packages/codemode/src/cloudflare.ts"], "@earendil-works/pi-codemode/declarations": ["./packages/codemode/src/declarations.ts"], "@earendil-works/pi-codemode/portable": ["./packages/codemode/src/portable.ts"], "@earendil-works/pi-codemode/source": ["./packages/codemode/src/source.ts"], "@earendil-works/pi-codemode/worker": ["./packages/codemode/src/runtime/worker.ts"], "@earendil-works/pi-schema": ["./packages/schema/src/index.ts"], "@earendil-works/pi-schema/formats": ["./packages/schema/src/formats/index.ts"], "@earendil-works/pi-mcp": ["./packages/mcp/src/index.ts"], "@earendil-works/pi-mcp/oauth": ["./packages/mcp/src/oauth/index.ts"], "@earendil-works/pi-mcp/testing": ["./packages/mcp/src/testing/index.ts"], "@earendil-works/pi-ai": ["./packages/ai/src/index.ts"], "@earendil-works/pi-ai/oauth": ["./packages/ai/src/oauth.ts"], "@earendil-works/pi-ai/*": ["./packages/ai/src/*.ts", "./packages/ai/src/providers/*.ts"], "@earendil-works/pi-ai/dist/*": ["./packages/ai/src/*"], "@earendil-works/pi-durable": ["./packages/durable/src/index.ts"], "@earendil-works/pi-durable/testing": ["./packages/durable/src/testing/index.ts"], "@earendil-works/pi-durable/*": [ "./packages/durable/src/*.ts", "./packages/durable/src/*/index.ts", "./packages/durable/src/*" ], "@earendil-works/pi-env": ["./packages/env/src/index.ts"], "@earendil-works/pi-agent-core": ["./packages/agent/src/index.ts"], "@earendil-works/pi-coding-agent": ["./packages/coding-agent/src/index.ts"], "@earendil-works/pi-coding-agent/hooks": ["./packages/coding-agent/src/core/hooks/index.ts"], "@earendil-works/pi-coding-agent/*": ["./packages/coding-agent/src/*"], "typebox": ["./node_modules/typebox"], "@earendil-works/pi-tui": ["./packages/tui/src/index.ts"], "@earendil-works/pi-tui/*": ["./packages/tui/src/*"], "@earendil-works/pi-agent-old": ["./packages/agent-old/src/index.ts"], "@earendil-works/pi-agent-old/*": ["./packages/agent-old/src/*"] } }, "include": ["scripts/model-catalog-protocol.ts", "scripts/model-catalog-protocol.test.ts", "packages/*/src/**/*", "packages/*/test/**/*", "packages/ai/catalog/**/*", "packages/durable/test-workers/**/*", "packages/coding-agent/examples/**/*", "packages/agent/examples/**/*"], "exclude": ["**/dist/**", "packages/coding-agent/examples/extensions/**", "packages/coding-agent/test/suite/regressions/8261-subagent-project-trust.test.ts", "packages/coding-agent/test/subagent-extension.test.ts"] } - vitest.base.ts (7415 bytes)
import { fileURLToPath } from "node:url"; import { defineConfig } from "vitest/config"; export const workspaceSourcePaths = { chordIndex: fileURLToPath(new URL("./packages/chord/src/index.ts", import.meta.url)), chordContext: fileURLToPath(new URL("./packages/chord/src/context/index.ts", import.meta.url)), chordDelta: fileURLToPath(new URL("./packages/chord/src/delta/index.ts", import.meta.url)), chordJson: fileURLToPath(new URL("./packages/chord/src/json.ts", import.meta.url)), chordWatch: fileURLToPath(new URL("./packages/chord/src/watch.ts", import.meta.url)), chordOutput: fileURLToPath(new URL("./packages/chord/src/output.ts", import.meta.url)), chordSqlite: fileURLToPath(new URL("./packages/chord/src/sqlite/index.ts", import.meta.url)), chordSqliteNode: fileURLToPath(new URL("./packages/chord/src/sqlite/node.ts", import.meta.url)), chordSqliteCloudflare: fileURLToPath(new URL("./packages/chord/src/sqlite/cloudflare.ts", import.meta.url)), chordTelemetrySqlite: fileURLToPath(new URL("./packages/chord/src/telemetry-sqlite/index.ts", import.meta.url)), chordHttp: fileURLToPath(new URL("./packages/chord/src/node/http/index.ts", import.meta.url)), chordWorkers: fileURLToPath(new URL("./packages/chord/src/node/workers/index.ts", import.meta.url)), chordPlugins: fileURLToPath(new URL("./packages/chord/src/node/plugins/index.ts", import.meta.url)), chordPluginsRuntime: fileURLToPath(new URL("./packages/chord/src/node/plugins/runtime.ts", import.meta.url)), chordApp: fileURLToPath(new URL("./packages/chord/src/node/app/index.ts", import.meta.url)), chordWeb: fileURLToPath(new URL("./packages/chord/src/web/index.ts", import.meta.url)), chordSurfaces: fileURLToPath(new URL("./packages/chord/src/node/surfaces/index.ts", import.meta.url)), chordReact: fileURLToPath(new URL("./packages/chord/src/react/index.ts", import.meta.url)), chordTui: fileURLToPath(new URL("./packages/chord/src/tui/index.ts", import.meta.url)), chordTuiLauncher: fileURLToPath(new URL("./packages/chord/src/node/tui/index.ts", import.meta.url)), telemetryIndex: fileURLToPath(new URL("./packages/telemetry/src/index.ts", import.meta.url)), telemetryTesting: fileURLToPath(new URL("./packages/telemetry/src/testing/index.ts", import.meta.url)), mcpIndex: fileURLToPath(new URL("./packages/mcp/src/index.ts", import.meta.url)), mcpOAuth: fileURLToPath(new URL("./packages/mcp/src/oauth/index.ts", import.meta.url)), mcpTesting: fileURLToPath(new URL("./packages/mcp/src/testing/index.ts", import.meta.url)), aiIndex: fileURLToPath(new URL("./packages/ai/src/index.ts", import.meta.url)), aiCompat: fileURLToPath(new URL("./packages/ai/src/compat.ts", import.meta.url)), aiModels: fileURLToPath(new URL("./packages/ai/src/models.ts", import.meta.url)), aiOAuth: fileURLToPath(new URL("./packages/ai/src/oauth.ts", import.meta.url)), aiApi: fileURLToPath(new URL("./packages/ai/src/api", import.meta.url)), aiProviders: fileURLToPath(new URL("./packages/ai/src/providers", import.meta.url)), aiUtils: fileURLToPath(new URL("./packages/ai/src/utils", import.meta.url)), agentIndex: fileURLToPath(new URL("./packages/agent/src/index.ts", import.meta.url)), codemodeIndex: fileURLToPath(new URL("./packages/codemode/src/index.ts", import.meta.url)), codemodeDeclarations: fileURLToPath(new URL("./packages/codemode/src/declarations.ts", import.meta.url)), codemodeSource: fileURLToPath(new URL("./packages/codemode/src/source.ts", import.meta.url)), codingAgentIndex: fileURLToPath(new URL("./packages/coding-agent/src/index.ts", import.meta.url)), tuiIndex: fileURLToPath(new URL("./packages/tui/src/index.ts", import.meta.url)), } as const; export default defineConfig({ resolve: { alias: [ { find: /^@earendil-works\/chord$/, replacement: workspaceSourcePaths.chordIndex }, { find: /^@earendil-works\/chord\/context$/, replacement: workspaceSourcePaths.chordContext }, { find: /^@earendil-works\/chord\/delta$/, replacement: workspaceSourcePaths.chordDelta }, { find: /^@earendil-works\/chord\/json$/, replacement: workspaceSourcePaths.chordJson }, { find: /^@earendil-works\/chord\/watch$/, replacement: workspaceSourcePaths.chordWatch }, { find: /^@earendil-works\/chord\/output$/, replacement: workspaceSourcePaths.chordOutput }, { find: /^@earendil-works\/chord\/sqlite$/, replacement: workspaceSourcePaths.chordSqlite }, { find: /^@earendil-works\/chord\/sqlite\/node$/, replacement: workspaceSourcePaths.chordSqliteNode }, { find: /^@earendil-works\/chord\/sqlite\/cloudflare$/, replacement: workspaceSourcePaths.chordSqliteCloudflare }, { find: /^@earendil-works\/chord\/telemetry-sqlite$/, replacement: workspaceSourcePaths.chordTelemetrySqlite }, { find: /^@earendil-works\/chord\/http$/, replacement: workspaceSourcePaths.chordHttp }, { find: /^@earendil-works\/chord\/workers$/, replacement: workspaceSourcePaths.chordWorkers }, { find: /^@earendil-works\/chord\/plugins$/, replacement: workspaceSourcePaths.chordPlugins }, { find: /^@earendil-works\/chord\/plugins\/runtime$/, replacement: workspaceSourcePaths.chordPluginsRuntime }, { find: /^@earendil-works\/chord\/app$/, replacement: workspaceSourcePaths.chordApp }, { find: /^@earendil-works\/chord\/web$/, replacement: workspaceSourcePaths.chordWeb }, { find: /^@earendil-works\/chord\/surfaces$/, replacement: workspaceSourcePaths.chordSurfaces }, { find: /^@earendil-works\/chord\/react$/, replacement: workspaceSourcePaths.chordReact }, { find: /^@earendil-works\/chord\/tui$/, replacement: workspaceSourcePaths.chordTui }, { find: /^@earendil-works\/chord\/tui\/launcher$/, replacement: workspaceSourcePaths.chordTuiLauncher }, { find: /^@earendil-works\/pi-telemetry$/, replacement: workspaceSourcePaths.telemetryIndex }, { find: /^@earendil-works\/pi-telemetry\/testing$/, replacement: workspaceSourcePaths.telemetryTesting }, { find: /^@earendil-works\/pi-mcp$/, replacement: workspaceSourcePaths.mcpIndex }, { find: /^@earendil-works\/pi-mcp\/oauth$/, replacement: workspaceSourcePaths.mcpOAuth }, { find: /^@earendil-works\/pi-mcp\/testing$/, replacement: workspaceSourcePaths.mcpTesting }, { find: /^@earendil-works\/pi-ai$/, replacement: workspaceSourcePaths.aiIndex }, { find: /^@earendil-works\/pi-ai\/compat$/, replacement: workspaceSourcePaths.aiCompat }, { find: /^@earendil-works\/pi-ai\/models$/, replacement: workspaceSourcePaths.aiModels }, { find: /^@earendil-works\/pi-ai\/oauth$/, replacement: workspaceSourcePaths.aiOAuth }, { find: /^@earendil-works\/pi-ai\/utils\/(.+)$/, replacement: `${workspaceSourcePaths.aiUtils}/$1.ts`, }, { find: /^@earendil-works\/pi-ai\/api\/(.+)$/, replacement: `${workspaceSourcePaths.aiApi}/$1.ts`, }, { find: /^@earendil-works\/pi-ai\/providers\/(.+)$/, replacement: `${workspaceSourcePaths.aiProviders}/$1.ts`, }, { find: /^@earendil-works\/pi-agent-core$/, replacement: workspaceSourcePaths.agentIndex }, { find: /^@earendil-works\/pi-codemode$/, replacement: workspaceSourcePaths.codemodeIndex }, { find: /^@earendil-works\/pi-codemode\/declarations$/, replacement: workspaceSourcePaths.codemodeDeclarations }, { find: /^@earendil-works\/pi-codemode\/source$/, replacement: workspaceSourcePaths.codemodeSource }, { find: /^@earendil-works\/pi-tui$/, replacement: workspaceSourcePaths.tuiIndex }, ], }, });