AIStoreHub Package Vault: BrowserKitten: In-Browser Autonomous AI Agent
Package Slug: browser-kitten
Archetype: agent
Upstream Repository: https://github.com/Player-YN/BrowserKitten
Workspace Files & Specifications (66 files)
- AGENTS.md (19492 bytes)
# Browser Kitten Chrome MV3 扩展,当前 manifest 版本为 `1.2.0`。本文记录源码结构、进程职责、数据合同与本地验证方式;功能现状以代码为准,不限定未来功能范围。 加载根就是 **本文件夹**(根上有 `manifest.json`)——开发和陌生人都加载它。`python scripts/pack_extension.py --zip <路径>` 只在发版时跑:它从当前工作区的 `manifest.json` + `icons/` + `src/`(不依赖 Git,包含新增模块) 生成一个不含 `.md` / `tests` 的 `extension/`(**已 gitignore,不入库**)与 Release zip。当前 `manifest.name` = `Browser Kitten`,`action.default_title` = `Open Browser Kitten`。改 `src/` 后点 **重新加载**。无 `package.json`,不跑 npm。 加载步骤与本地验证:[README.md](README.md)。 侧栏 UI 的视觉与交互规范:[Design.md](Design.md)。新增界面复用其 token、按钮层级与响应式规则。 ## 文档怎么读 | 文件 | 写什么 | 什么时候打开 | |------|--------|----------------| | 本文件 | 进程分层、数据对象、目录地图 | 任何任务的第一站 | | [src/AGENTS.md](src/AGENTS.md) | Chrome 宿主:SW / offscreen / 侧栏 / content script / 消息总线 / `action` 运输 | 改扩展壳、RPC、标签、选区 | | [src/agent/AGENTS.md](src/agent/AGENTS.md) | Session Workspace:store、工具循环、工具契约、skills、guest FS、`sys` ABI | 改模型循环、工具、prompt、持久化领域 | | [src/preview/AGENTS.md](src/preview/AGENTS.md) | 画布标签页(docs / site) | 改 docs / site 预览 | | [src/sidepanel/README.md](src/sidepanel/README.md) | 侧栏 UI 模块与滚动契约 | 改对话面板布局 / i18n | | 本文件「实现现状与恢复约束」 | 持久化 / `sys` / 长任务的实现与恢复约束 | 改这三块方向前 | 事实以仓库代码为准。设置在 `chrome.storage.local` 的 `browserbot_*` 键;进程间消息 `target` 用 `browserbot-*`。 ## 技术分层 扩展不是单页应用。Chrome 把代码拆进不同进程;agent 只住在 **offscreen** 里,侧栏只发 RPC。 ```text ┌─ 用户网页(任意 http(s))─────────────────────────────────┐ │ content_script.js all_frames │ │ 点选选区 · 每 frame 的 action snapshot/mutate │ └──────────────────────────────▲─────────────────────────────┘ │ tabs.sendMessage {frameId} ┌─ Service Worker src/background.js ────────────────────────┐ │ 路由:workspace_rpc / workspace_page_action / workspace_sys │ │ sheet_host / canvas_host / 截图 / 下载 / 预览 / tabGroups │ │ tab 租约(storage.session)· 政策/ticket 最后一道门 │ │ alarms 唤醒(store 权威)· SW 不持有 journal / 模型循环 │ └──┬───────────────────────────┬─────────────────────────────┘ │ workspace_rpc │ 打开 chrome-extension:// 标签 ▼ ▼ ┌─ Offscreen src/offscreen/ ─┐ ┌─ Preview src/preview/ ──┐ │ SessionWorkspaceService │ │ docs / site / │ │ IDB + OPFS 会话仓库 │ │ artifactPreview │ │ AI SDK ToolLoopAgent │ │ docs / site 预览 │ │ iframe → sandbox QuickJS │ └──────────────────────────┘ └──────────────▲──────────────┘ │ workspaceRpc(method, params) ┌─ Side panel src/sidepanel.html + sidepanel.js ────────────┐ │ 对话 UI、选区、交付物。不跑 agent 循环。 │ └────────────────────────────────────────────────────────────┘ ``` | 层 | 职责 | 不该做 | |----|------|--------| | Side panel | 渲染会话、收集用户输入、调 `workspaceRpc` | 持有 Durable store、调模型 | | Service worker | 权限面:标签、下载、跨 frame 扇出、打开预览页、`workspace_sys` | 长期状态、ToolLoopAgent | | Offscreen | 会话 runtime:store + `sendMessage` + 工具 | 直接碰 DOM / `chrome.tabs`(经 SW 消息) | | Content script | 页面选区与 `action` 本 frame 执行 | 会话语义、模型、`sys.eval`(那是 SW `userScripts`) | | Sandbox | `run` 的访客 JS(无 `chrome.*`;经 `sys` RPC 调 SW) | 读扩展存储、改 SelectionGroup | | Preview 标签 | 把 artifact 画成可编辑画布 | 模型循环 | ## 一次用户消息 ```text 侧栏 send → workspaceRpc('sendMessage') → SW forwardWorkspaceRpc → offscreen SessionWorkspaceService.sendMessage → sessionWorkspace/sendMessage.js beginExecution createSessionTools + inventoryFromSession(瞄准,不藏工具) AI SDK 7 ToolLoopAgent(sessionAgent.js,toolChoice=auto) 工具:inspect / acquire / run / clarify / action / task / doc / web (sys 不是工具;在 run 代码里调,目录:inspect view=sys) (普通 send 不建 durable task;task 工具始终在 schema 里) settleExecution → session_workspace_event 广播回侧栏 ``` 工具实现与契约:[src/agent/AGENTS.md](src/agent/AGENTS.md)。`action` 出 offscreen 后如何打到页面:[src/AGENTS.md](src/AGENTS.md)「`action` 运输」。 ## 数据对象 都在 offscreen 的 `DurableSessionWorkspaceStore` 里(IndexedDB `browserbot-session-workspace-v1` + OPFS blobs)。纯内存 `SessionWorkspaceStore` 只给测试。 | 对象 | 是什么 | 谁改 | |------|--------|------| | **Session** | 一轮任务:messages、title、绑定的 group ids | UI RPC + `sendMessage` | | **Group + WebItem** | 用户拥有的环境上下文(点选选区、剪贴板钉、页面条目) | **仅 UI RPC**。工具禁止 mutate SelectionGroup | | **Artifact** | 会话交付物(文档 HTML / site HTML / 文件) | 工具 `run` / office 工具 / UI 创建。UI 按 `docs`/`data`/`web`/`media`/`files` 投影;打开面见 `artifactCapability.js` | | **Execution** | 单次用户 turn 的租约与 `/scratch`;崩溃后作废 | `sendMessage` | | **Task** | 可选 durable 进度对象(`meta` 的 `task:*`:合同、plan / evidence / dueAt、来源、观察基线、报告策略与停止条件)。**不是** call journal | 确认后的 `start` / 勾选长任务的 `/plan` 或未绑定任务的 `schedule` 新建;已绑定任务自主更新调度;UI `updateTask`;alarms 续跑 | | **Guest FS** | 访客可见 `/context`(只读)· `/artifacts`(持久)· `/scratch`(本轮) | `run` 沙箱 | | **sys** | 浏览器机器 ABI(tabs / eval / fetch / **cdp** / download / screenshot)。guest 无 `chrome.*` | `run` → SW `workspace_sys` | 画布种类(inventory 瞄准,工具始终在):`doc` · `web`(`data-browserbot-kind=site`)。文档预览是标记 HTML。csv / xlsx 按文件打开。 ## 加载根树 ```text manifest.json # MV3:SW / side_panel / content_scripts / sandbox / CSP icons/ # 16|32|48|128 scripts/pack_extension.py # 发版:生成 gitignore 的 extension/ 与 zip tests/ # `*.test.mjs` 逻辑回归(纯 Node)· browser-smoke(Playwright,CI 不跑) src/ background.js # Service worker(type: module) content_script.js # <all_urls> all_frames;点选 + action sidepanel.html|js|css # 侧栏壳;编排在 sidepanel.js sidepanel/ # 侧栏子模块(见该目录 README) offscreen/ # 创建 SessionWorkspaceService;嵌 sandbox iframe sandbox/ # QuickJS guest(manifest.sandbox) preview/ # 画布 / 通用预览标签页 agent/ # 模型循环与领域(offscreen 主进口) llm.js provider.js # BYOK OpenAI-compatible vnext/service/ # SessionWorkspaceService RPC 门面 vnext/sessionWorkspace/# store、sendMessage、tools、office vnext/primitives/ # acquire / run 宿主原语 vnext/adapters/ # QuickJS、AI SDK vendor loader vnext/host/ # workspaceRpc、browserSysHost、tabLease、taskScheduler、 # accessPolicy、riskClassify、dispatchTicket、callJournal、 # operationGate、postcondition vnext/skills/ # playbook(不是工具) ``` `src/preview/vendor/fflate.js` 已跟踪,打开 docx 时用。 ## 实现现状与恢复约束 以下记录当前代码的行为和验证范围。修改持久化 / `sys` / 长任务时先核对相关实现。 - **Access Policy 是宿主门,不是 prompt,也没有模式开关**(`accessPolicy.js`,旧的 Guarded/Full 存储键启动时删除)。已知付款不派发;已知删除要 one-shot 宿主审批(不是 `clarify`);其余 known/unknown/raw(含 eval、CDP)**best-effort 自动放行**,文案不得承诺绝不付款/删除。分类器是启发式,raw 可以绕过。journal 行保留 `accessMode` 字段,恒为 `full`。 - **有 call journal,但不是 exactly-once**:独立 IDB `browserbot-call-journal-v1`。实际运行路径 IDB 打不开或事务失败 → `JOURNAL_UNAVAILABLE`,外部写不 dispatch;内存 journal **只**能 `memoryJournal: true` 测试参数启用,不自动降级。write-ahead:`prepared → awaiting_approval|authorized → dispatched → succeeded|failed|unknown → verified|needs_human`。崩溃卡在 authorized/dispatched 标 failed/unknown 并丢掉残留 ticket,**不盲重放**。SW `callId` 取消表仍是内存。task 是进度,audit 是观察日志,journal 是恢复真相。 - **最小 postcondition 是可扩展合同,不是业务完成证明**:action/delete/download/artifact 有宿主核对;不可证明的 send 只能 `needs_human`,模型自然语言 evidence 不能当 verified。 - **可选 durable task(进度 + alarms,不是 journal)**:记录在 IDB `meta` 的 `task:*`(goal / plan / evidence / dueAt / status)。SW `chrome.alarms` 按 store 的 `dueAt` 与 `stopAt` 唤醒,**store 是权威**。不是 exactly-once。普通 `sendMessage` 不自动新建 task;存在本会话有效的非终态 `activeTaskId` 时,后续消息会继承该任务上下文。`taskRun+taskId`(alarm / UI resume)显式绑定已有 task;确认后的 `start` / 勾选长任务的 `/plan` 也会绑定当前执行。`completed` 只来自成功的 `task complete`。 - **本地定时检查**:Agent 可在用户目标范围内自主创建、修改和移除调度,无需用户给出计时指令。已绑定任务的 `schedule` 更新同一 Task 并 yield;未绑定才新建。新 `schedule` 默认 `targetMode: discover`、`reportPolicy: changes`;`page` 显式绑定 URL,旧任务从 `targetPage` 推断来源模式并保留每轮报告。`observe` 的稳定 key/fingerprint 账本记录基线与未处理候选,最多 512 条,不自动淘汰;`handledKeys` 只表示 Agent 声明已核实处理,不替代 call journal。安静 `wait` 更新任务而不追加聊天报告;`unschedule` 移除唤醒、间隔、截止时间并保留任务进度;`complete` / cancel 永久停止并清掉调度。`stopAt` 宿主硬截止取消,`stopWhen` 模型判断;定时轮次默认 5 分钟预算(1–30 分钟),超时暂停。没有页面即时订阅,也没有浏览器关闭时的云端续跑。 - **live 页 tab 租约由 SW 管理、写入 `chrome.storage.session`**:先成功登记,后派发页面操作;跨 SW 重启恢复,不跨浏览器重启/扩展重载。首次执行前与 offscreen 的活动执行对账,状态读写失败则拒绝派发,不以超时自动抢锁。结束时先撤销该 execution 的后续派发资格、取消可取消调用、释放所属 CDP,再放租约。它不是 journal,不保证已发出的副作用停止。锁粒度是 tab,不是跨标签的同账号业务资源;预览 workLock 仍独立。 - **持久化仍导出内存快照**:大工作区需要改成记录/文件的增量提交。新文件发布前崩溃可能留下未被引用的 OPFS 文件;orphan GC 未实现,不自动清理无法证明归属的文件。`task:*` 跟着 `meta` 快照走,仍然不是增量 journal。 - **OPFS 不可用时**新字节写入 IDB;已有 OPFS 引用不会因暂时不可用被清空。 - **交付物版本不是完整协同编辑**:旧调用方可省略 `expectedRevision`,raw guest 写入推进版本但不携带读版本;没有自动合并,也没有冲突解决界面。 - **页面目标绑定 documentId**:`action` 快照记录 frame + documentId + URL;所有 mutate 要求不透明 rev。导航/SPA 路由变化使旧快照失效;SW 重启后重新观察。`sys.eval/waitFor/page-fetch` 用 `userScripts.target.documentIds`,可带 `documentId/expectedUrl` 前置条件;不是对同一文档中所有 DOM 变化的原子保护。CDP 按 session + execution 归属并在结束时 detach;网络事件仍不是持久日志。 - **截图不是原子快照**:捕获前后校验活动标签,但仍是视口合成,不适合需要严格文档身份的视觉执行。 - **同一 session 的第二次执行返回 busy**,不支持并发 turn。普通回合与 durable-task 回合都**无 hop 上限**;task 停点是 `wait` / `pause` / `complete` / host yield / abort,不是步数。定时任务另有单轮时间预算。 - **当前宿主接口**:尚未接入 native messaging / 本地命令 / 原生 OS 控制;Office 高保真往返尚未验证。 ## 日常约定 - 改代码 → `chrome://extensions` 点本扩展 **重新加载**。offscreen / SW 会重建;`action` 的 `rev` 在 SW 内存,重载后需重新 `snapshot`。 - 权限见 `manifest.json`:`sidePanel` `activeTab` `tabs` `scripting` `storage` `identity` `alarms` `downloads` `offscreen` `tabGroups` `webNavigation` `debugger`;**可选** `userScripts` `tabCapture`;`host_permissions: <all_urls>`。`userScripts` 服务 guest 脚本能力;`debugger` 同时用于 guest `sys.cdp`、宿主坐标输入和显式 CDP 上传([src/AGENTS.md](src/AGENTS.md))。可选权限只能由用户点击授予:`tabCapture` 在侧栏听音/录屏授权条的点击里申请,`userScripts` 在设置「高级能力」里申请(Chrome 138+ 还要用户在扩展详情页打开「允许用户脚本」);未授予时宿主返回 `PERMISSION_REQUIRED` + `permission`。`debugger` 按 Chrome 规定不能设为可选。`identity` 只用于 Browser Kitten 账户登录(`launchWebAuthFlow`)。`alarms` 唤醒 due task 与已发布 Workflow 的定时新运行,不持有 journal。 - 命令:`toggle-picker` = Alt+Shift+S;`capture-screenshot` = Alt+Shift+C。 - Git:本地 `main`;`origin` = `https://github.com/Player-YN/BrowserKitten.git`。不要改 `git config`。不要 force-push `main`。 - 逻辑回归:`node --test tests/*.test.mjs`(与 CI 同一 glob。覆盖原有磁盘/租约/task/`bot-*`,以及 Access Policy / approval / call journal / postcondition / host last-door)。 - 实机烟测(手动,CI 不跑):`node tests/browser-smoke.cjs <playwright路径>`。默认加载根是仓库根;要验**真正发出去的那棵树**,先 `python scripts/pack_extension.py` 再 `BROWSERBOT_LOAD_ROOT=extension node tests/browser-smoke.cjs <playwright路径>`。独立 Chromium、一次性 profile,不碰你的浏览器状态。覆盖:IDB/OPFS 关闭重开、真实 QuickJS 错误往返、`run`→文件→artifact 登记、非 offscreen `sys` 请求拒绝、真实 localhost fetch 与正文超时、offscreen RPC 版本冲突拒绝。证据在 `output/playwright/`,不入库。 - CI(`.github/workflows/ci.yml`)跑 `node --test tests/*.test.mjs` + pack 形状断言(含「pack 里必须带 TSV 修复」这条防漂移守卫),不装浏览器。 - BYOK:侧栏填 Key → `browserbot_providers`。无 Key 时 offscreen 仍可启动,`sendMessage` 时再解析模型。 - **BYOK 需要付费套餐**(`src/agent/byokGate.js`):自己的 Key(非 `bk_cloud_*` 记录、非 base 指向 browserkitten.com 的 Tavily/Firecrawl 槽)只在已登录且套餐为 developer / standard / pro / max 时可用;free/试用只能用积分。BYOK 调用不经我们服务器、不扣积分。缓存 `browserbot_cloud_entitlement = { plan, checkedAt }`:读 `/api/me` 时写,退出登录与 401 时清;72 h 内有效,超过 6 h 先刷新(网络失败按 72 h)。拒绝码 `BYOK_REQUIRES_PLAN`,侧栏通知条给「查看套餐」/「登录」。**unpacked 构建豁免**(manifest 无 `update_url`,Web Store 才会加),商店构建才受限;想在 unpacked 上试门,设 `browserbot_byok_enforce_unpacked = true`。这是客户端门,不是服务器强制。不调模型的路径(个人库工具、结构化 Workflow 回放、用户自己的页面操作)不受限。 - 测试后端:未打包的开发版可以在 `chrome.storage.local` 设 `browserbot_cloud_origin = 'https://staging.browserkitten.com'`,之后的登录走 staging;商店版(manifest 有 `update_url`)永远只连正式环境。登录时选定的地址存进账户记录,之后的请求都用它(`cloudModels.js` `resolveCloudOrigin`)。 - 积分(Browser Kitten 账户):`src/agent/cloudAccount.js`。登录按服务端 `hosted` 能力写入对应的固定 id 服务商记录(`bk_cloud_chat|image|stt|jev`,baseURL 指向 `browserkitten.com/api/v1`,apiKey = 设备令牌),Tavily / Firecrawl 为空或已指向本服务时更新;不覆盖用户已配的 Key,不抢已可用的活动服务商。登出只撤回这些。套餐与积分每次请求由服务端判定,本地记录不授予任何权益。账户接口接线见 [docs/COMMERCIAL_LAUNCH.md](docs/COMMERCIAL_LAUNCH.md)。 ## Chrome BOT 1.2.0 候选版接线 实现与验收记录见 [docs/CHROME_BOT_OPTIMIZATION.md](docs/CHROME_BOT_OPTIMIZATION.md)。此版本尚未通过真实 Chrome 全链路验收。 - RPC:`workspaceRpcContract.js` 明确允许的方法、调用角色与读取重试属性;不是枚举 service 的公开方法。只读可重试,写入回执丢失为 `RPC_OUTCOME_UNKNOWN`,不自动重投。侧栏与预览共用 `workspaceClient.js`。调用方从 Chrome 原生 sender 判定,不接受消息自称的角色。 - 页面:`pageActionHost.js` 负责文档目标、快照、同 tab action 排队;`documentTarget.js` 负责文档前置校验。不要把这条队列描述为所有任意 sys 代码的串行执行器。 - 生命周期:`browserExecution.js` 统一取消、CDP 清理、租约释放与 SW 启动对账;`tabLease.js` 只由 SW 配置 storage.session 持久适配器。 - 打包:`python scripts/pack_extension.py` 后执行 `python scripts/verify_extension.py extension --source .`,检查字面量本地依赖与源码字节一致性;不代替浏览器测试。 - aistorehub.json (550 bytes)
{ "name": "BrowserKitten: In-Browser Autonomous AI Agent", "slug": "browser-kitten", "version": "1.0.0", "type": "agent", "description": "Open-source Chrome Manifest V3 sidepanel extension and autonomous AI agent. Brings your own LLM to active browser sessions to read live web pages, execute guest scripts, and automate workflows without cloud infrastructure.", "owner": "community", "repository": "catalog", "github_repo_url": "https://github.com/Player-YN/BrowserKitten", "tags": [], "imported_at": "2026-11-21T23:56:28.674Z" } - CHANGELOG.md (1363 bytes)
# Changelog Notable user-facing changes are recorded here. `main` may be ahead of tagged GitHub releases; the current manifest version is **1.2.0**. An unreleased entry does not announce a published release. ## [Unreleased] ### Added - Local ongoing-task observations, change-aware reporting, configurable stop conditions, and per-round budgets. - Autonomous scheduling and removal of follow-ups within the authorized goal. - English and Chinese project homepages, setup guides, documentation navigation, and community contribution/reporting files. ### Changed - Recoverable tasks live in a collapsible drawer above the composer instead of the top of the chat. - The fixed Agent personality is English and separate from runtime execution contracts. - Distribution instructions distinguish Inference and System1 setup; packages include third-party notices. ### Fixed - System1's settings probe uses the same chat endpoint and decision format as the page executor. ## [1.1.0] — 2026-09-16 See the original [release notes](https://github.com/Player-YN/BrowserKitten/releases/tag/v1.1.0). Earlier tagged builds remain available in [GitHub Releases](https://github.com/Player-YN/BrowserKitten/releases). [Unreleased]: https://github.com/Player-YN/BrowserKitten/compare/v1.1.0...HEAD [1.1.0]: https://github.com/Player-YN/BrowserKitten/releases/tag/v1.1.0
- CODE_OF_CONDUCT.md (1506 bytes)
# Code of Conduct We want Browser Kitten to be a useful, welcoming place to collaborate. These expectations apply to issues, pull requests, reviews, and other project spaces, regardless of a person's background, identity, or level of experience. - Be respectful, patient, and specific. Critique ideas and code rather than people. - Welcome questions, different perspectives, and good-faith corrections. - Respect privacy. Do not publish another person's credentials, private communications, or identifying information without permission. - Do not harass, threaten, discriminate, use sexualized language or imagery, or repeatedly disrupt discussions. Spam and personal attacks are not welcome. Maintainers may edit or remove inappropriate content, close disruptive discussions, or restrict participation. They should explain moderation decisions when doing so does not expose private information or cause further harm. For conduct concerns, contact [@Player-YN](https://github.com/Player-YN) through a private contact method listed on their profile. If none is available, ask for a private contact channel without posting the incident or personal details publicly. This project does not promise a fixed response time. **中文:**请友善、具体地讨论问题,尊重他人隐私,不进行骚扰、歧视、人身攻击或刷屏。行为问题可通过维护者个人主页提供的私密联系方式反馈;没有私密入口时,可先请求联系方式,不要公开当事人的隐私。
- CONTRIBUTING.md (3495 bytes)
# Contributing to Browser Kitten [Home](README.md) · [Documentation](docs/README.md) · [Code of Conduct](CODE_OF_CONDUCT.md) Thanks for helping improve Browser Kitten. Documentation fixes, clear bug reports, and focused code changes are useful contributions. **English and Chinese are both welcome / 欢迎使用中文或英文。** ## Choose the right starting point - Search existing [issues](https://github.com/Player-YN/BrowserKitten/issues) before opening another one. - For bugs, include a small reproduction and the exact error code using the bug template. - For a substantial change, open a feature request describing the problem and intended behavior before investing in a large PR. - Send security findings through [private reporting](SECURITY.md), not public issues. ## Work locally ```sh git clone https://github.com/Player-YN/BrowserKitten.git cd BrowserKitten git switch -c your-change ``` Load this repository root in Chrome as an unpacked extension. There is no `package.json`, npm install, or production build step. See [Getting started](docs/GETTING_STARTED.md) for browser setup and provider configuration. Use your own credentials; never commit them. The side panel renders UI, the offscreen document runs the agent and store, and the Service Worker dispatches browser operations. Read the relevant [architecture documentation](docs/README.md#develop-and-understand--开发与架构) before changing those boundaries. Reuse existing UI tokens and browser ownership / permission contracts. ## Validate the change proportionately Runtime regressions require **Node.js 22+**. Packaging requires **Python 3.10+**: ```sh node --test tests/*.test.mjs python3 scripts/pack_extension.py python3 scripts/verify_extension.py extension --source . git diff --check ``` - For behavior changes, run relevant tests first, then the existing full suite. Add regression coverage when it demonstrates a meaningful failure; avoid tests that merely repeat implementation details. - For UI changes, reload the extension and inspect the affected flow at a narrow side-panel width, including light/dark themes and keyboard interaction when relevant. - For documentation-only changes, check rendered Markdown, navigation, relative links, and both README languages. Runtime tests are unnecessary unless the change affects behavior or packaging. The optional browser smoke test uses Playwright and an independent temporary Chromium profile: ```sh node tests/browser-smoke.cjs <path-to-playwright-package> ``` `BROWSERBOT_LOAD_ROOT=extension` tests the packaged tree; `BROWSERBOT_CHROMIUM_EXECUTABLE` selects an installed Chromium executable. Model and website outcomes require separate live verification. Do not report them as tested from a Node-only run. ## Submit a focused pull request Explain the problem, resulting behavior, and checks performed. Include screenshots for visible UI changes, and identify material limits or unverified behavior. Update affected documentation; if the README structure or capabilities change, keep [English](README.md) and [中文](README.zh-CN.md) aligned. Keep generated `extension/`, `dist/`, browser profiles, logs, account tokens, and task exports out of commits. Preserve required third-party notices. Changes should respect the existing [MIT license](LICENSE); this project does not require a separate CLA. Maintainers may request revisions or decline changes outside the project's current scope. There is no guaranteed review turnaround.
- Design.md (9056 bytes)
# Browser Kitten · Design 侧栏 UI 的设计基准。新增或修改界面时先读本文,再复用现有组件与 token。本文记录现有 UI 规范;权限、执行、存储合同仍以 `AGENTS.md` 和领域文档为准。 ## 1. 视觉方向 **Quiet Black / Quiet Light:安静的黑白工作台,少量橙色提示交互与状态。** 内容和操作优先。通过表面明度、细边框、间距和字重建立层级,不靠渐变、发光、大面积品牌色或装饰动画。可点击的常驻入口必须像控件,不能看起来像禁用的灰色说明。 保留猫形品牌标识。现有点选、截图图标沿用;新增图标保持简单、同尺寸、同笔画重量,不为了填满按钮增加装饰。 ## 2. 颜色与表面 唯一颜色来源是 [tokens.css](src/sidepanel/css/tokens.css)。组件引用语义变量,不复制颜色值或另建主题。 - 页面底色 `--paper`:深色 `#050506`,浅色 `#f4f3ef`。 - 正文 `--ink` / `--text`:深色 `#f4f3ef`,浅色 `#111113`。 - 卡片 `--surface` / `--bg-elevated`:深色 `#111113`,浅色 `#ffffff`。 - 次级表面 `--surface-2`:深色 `#1c1c1e`,浅色 `#eceae4`。 - 分隔线 `--border`,可点击边界 `--border-strong`。普通边框 1px。 - 次要元数据用 `--text-muted`,不能用它表示默认可操作按钮的主标签。 - 橙色 `--signal` / `--accent` 为 `#ff6a00`,用于键盘焦点、选中边界和需要关注的运行状态。 - 成功、警告、错误沿用 `--green`、`--amber`、`--danger`,并提供文字或图标,不能只靠颜色说明结果。 普通表面不投影,使用 `--shadow` / `--shadow-card`(当前为 `none`)。弹窗通过遮罩和边框分层。主按钮使用 `--ink` 背景及 `--paper` 文字,浅色主题自动反转。 ## 3. 尺寸与排版 - 字体使用 `--font`;代码、结构化定义和原始输出使用 `--font-code`。 - 正常控件文字 12–13px;区域标题约 15px。11px 只用于紧凑字段标签和元数据,不作为主操作字号。 - 主操作字重 650–700;正文保持普通字重,不整片加粗。 - 常用间距沿用 `--space-1/2/3/4`,对应 4 / 8 / 12 / 16px。紧凑按钮组可用 6px。 - 容器圆角 `--radius`(14px),入口与卡片 `--radius-sm`(10px),小控件 `--radius-xs`(6px)。 - 顶部入口高度统一为 `--density-pick-h`,正常 34px、compact 30px;字级和内边距跟随相应 `--density-pick-*`。 长名称保留单行省略和完整 tooltip / accessible name。结果、错误及必要说明允许换行,不能将关键失败原因截断。 ## 4. 按钮与交互 **主操作**:实心黑白反色按钮,例如点选、发送、弹窗最终的添加/更新。同一区域避免多个相互竞争的主按钮。 **次级操作**:表面底色、细描边、正常文字颜色。截图、工具、Workflow 使用同一高度、圆角和交互规则。「⋯」是同高度方形管理按钮,保留可访问名称。 **辅助操作**:删除条目、关闭、展开等可使用低强调图标按钮;需要明确命中区域与说明。权限盾牌属于顶栏状态入口,不挤进聊天输入工具栏。 - Hover:轻微改变背景或边框;已有快捷按钮允许上移 1px。 - Pressed:短促反馈,不移动整块布局。 - Focus-visible:清晰橙色轮廓,不能去掉 outline 后不提供替代。 - Disabled:降低强调并真实禁用,不能只更改颜色。 - 系统要求 reduced-motion 时关闭非必要动画与过渡。 - 确认类操作使用明确动词;试运行说明真实页面副作用,发布必须由用户点击添加/更新。 ## 5. 页面职责与布局 ### 顶栏与快捷入口 顶栏保留品牌、当前页面、权限盾牌、齿轮设置入口;「⋯」用于快捷区的个人库管理。快捷区顺序为点选、截图、工具、Workflow、个人库管理。工具和 Workflow 可被已固定的用户条目替换;替换后维持原有按钮样式。 五个快捷入口始终保持同一行,不通过换行增加顶部高度。窄栏优先缩减横向留白与间距,按钮允许收缩,长名称省略并保留完整 tooltip / accessible name;不能隐藏常驻入口或产生横向溢出。使用 `.panel` 的 container query,不按显示器宽度判断。选区摘要、固定展开与清空另行排列,不能挤压常驻入口。入口网格只在 `sidepanel.css` 定义,避免旧布局规则覆盖新入口。 ### 点选与资料组 点选得到对象;快捷操作直接处理对象;资料组保存和组织;参考让 Bot 理解对象。资料组与对话引用都不授予浏览器执行权限。 无内容时不展示资料组或空抽屉。有内容后按对象、适用操作、文字剪贴板、收集位置排列。不要恢复用户已要求删除的灰色解释段落;必要说明放 tooltip、可访问描述或相关帮助中。对象名称、计数和操作标签仍可见。 点选后短暂展开、随后收起;悬停、固定与键盘焦点遵守现有抽屉规则。菜单操作期间不得误收起。 ### 对话与附件 输入任务不要求先点选或创建资料组。参考计数只来自实际绑定。上传和粘贴预览必须位于 `.composer-inner` 内,随输入框移动,不能掉到聊天框外。删除预览只改变待发送附件。 打开对话后,消息直接落在 `--paper` 上:不画 `.session-thread` / `.task-card` 的大边框井、也不再给卡片套呼吸流光。用户气泡保持橙色胶囊;助手回复是裸散文。运行中的进度说明也是裸散文(与回答同字号、正文字色),和灰色小图标活动行在同一条时间线里按顺序穿插;回合结束后与活动行一起折成一行,展开时带 HH:MM 时间。不进思考块,不画成卡片或气泡。首页(无会话线程)不要再长出第二张卡;输入坞仍停在 composer。 滚动以 [layout.css](src/sidepanel/css/layout.css) 为准:首页由 `#panelScroll` 滚动,打开对话后由 `.task-body` 滚动,保留会话标题与输入区域。不要为每张普通内容卡新建滚动容器。代码预览、附件列表、弹窗编辑器可有明确的高度限制。 可恢复的持续任务放在输入区上方的独立抽屉(`footer.composer > #durableTaskDock`),不插入聊天顶部或消息时间线。抽屉与单张任务都默认折叠,展开状态按会话保留;折叠行用短标题、计数与状态,长目标单行省略并保留完整 tooltip。展开后才能看步骤、来源、进度和操作。抽屉内容整体限高 `min(38vh, 320px)` 并独立滚动,单张任务不另建滚动容器,滚轮不转送聊天。继续使用中性表面、细边框和橙色焦点,不加大面积橙色卡框。 对话滚动条是 overlay,不占布局:静止时隐藏;滚轮 / 触控 / 触摸板滚动时出现,最后一次滚动后约 700ms 再隐去。只点亮**滚动条本身**(对话 6px / 思考块 4px 的 thumb),不是整列轨道。`.task-body` 与思考块 `.think-body-wrap` 各有一条星光条,互不连动——在思考块里滑记录只点亮块内小条,不会点亮外层对话条。星尘裁在 thumb 里(`overflow: hidden`),不洒到正文。`prefers-reduced-motion` 保留渐变与光晕,冻结粒子。不是橙色霓虹管,也不再绕卡片跑彗星。 ### 个人库与 Workflow 个人库、参数输入、流程编辑沿用相同表面、圆角和按钮层级。窄栏表单允许折行或堆叠,名称与错误信息不能撑出侧栏。草稿编辑区不因运行事件重新绘制并丢失输入。 「工具」是独立的确定性脚本;「自动化」是可录制、可循环的结构化流程;「Agent 任务」通过模型对话执行。界面必须区分这三者。运行状态展示当前步骤、已通过的检查、失败原因及恢复位置;用户确认不能显示成系统已验证。运行时不能让编辑弹窗遮住宿主审批。 ## 6. 实现位置与验收边界 - [tokens.css](src/sidepanel/css/tokens.css):颜色、字体、圆角、间距、密度。 - [layout.css](src/sidepanel/css/layout.css):高度链、主要滚动与响应式布局。 - [motion.css](src/sidepanel/css/motion.css):动效及 reduced-motion。 - [sidepanel.css](src/sidepanel.css):组件外观、快捷入口与个人库布局。 - [sidepanel/README.md](src/sidepanel/README.md):模块、事件与交互合同。 CSS 加载顺序保持 tokens → layout → motion → components。优先合并同一组件的已有规则,避免不断追加高优先级覆盖。旧界面尚未完全收敛的样式随实际改动修正,不以统一设计为理由重写无关模块。 提交前运行相关 Node 回归和差异检查。实际外观由 Chrome 验收:深浅主题、中英文、常规与窄侧栏、空选区与混合选区、长固定名称、键盘焦点、弹窗和粘贴附件。纯文案或样式不新增快照测试;Node 通过不代表真实浏览器视觉验收通过。
- INSTRUCTIONS.md (1953 bytes)
--- name: Site Tool Reuse description: User needs an outcome a live website or already-open logged-in tab already provides — edit, export, generate, convert, upload, send — including how to discover and compare those capabilities with other channels. --- # When this applies The result may already exist as an in-page control, an open tab, or a service the user is logged into. This playbook is a discovery method, not a required path. # Discover 1. Read this turn's world: focusPage / activeTab, and openTabs (tabCount + domains only). 2. Snapshot or screenshot the current page if you need controls or pixels. 3. If a matching domain is already open, you can focus that tab. Opening a parallel editor is optional. 4. Note logged-in services the user named. # Compare then choose Write one comparison line before acting: - Ready-made site tool: what it already does, and the visible success check. - action (ref / name / pointer), run + sys, or a self-built canvas: what each can do here. Pick the channel that best fits quality, reliability, and cost for this task. Host codes (NO_TARGET, CDP_BUSY, TAB_NOT_VISIBLE) are facts for switching, not a prescribed order. # Operate Use whichever live-tab or guest-sys channel you chose. After a mutate, check the page (url / controls / screenshot if attached). # Check afterward Confirm a host-visible postcondition (new filename, canvas object, download, URL change). Tool `ok` is not task success. Upload `siteAccepted` stays unknown — look at the page. # If the preferred route fails Re-enter discovery. Try another control, another open domain, another channel, or another logged-in service the user already has. Then compare again. A failed preferred route is not a lock; discovery stays allowed. # Parameterize Keep task class, intents, and locate strategies (role, accessible name, region, or viewport-relative pointer). Do not persist `ref`, `rev`, coordinates, or fragile CSS.
- LICENSE (1075 bytes)
MIT License Copyright (c) 2026 Yiteng (Player-YN) Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions: The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software. THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
- manifest.json (2242 bytes)
{ "manifest_version": 3, "name": "Browser Kitten", "version": "1.2.0", "description": "An AI assistant in your side panel. It reads, clicks, and fills the pages you are signed in to, and hands back finished work.", "icons": { "16": "icons/icon-16.png", "32": "icons/icon-32.png", "48": "icons/icon-48.png", "128": "icons/icon-128.png" }, "permissions": [ "sidePanel", "activeTab", "tabs", "scripting", "storage", "identity", "alarms", "downloads", "offscreen", "tabGroups", "webNavigation", "debugger" ], "optional_permissions": [ "userScripts", "tabCapture" ], "host_permissions": [ "<all_urls>" ], "commands": { "toggle-picker": { "suggested_key": { "default": "Alt+Shift+S", "mac": "Alt+Shift+S" }, "description": "Pick content on the page" }, "capture-screenshot": { "suggested_key": { "default": "Alt+Shift+C", "mac": "Alt+Shift+C" }, "description": "Capture a screen region to the clipboard and the chat" } }, "side_panel": { "default_path": "src/sidepanel.html" }, "background": { "service_worker": "src/background.js", "type": "module" }, "sandbox": { "pages": [ "src/sandbox/runtime.html", "src/sandbox/siteFrame.html" ] }, "content_scripts": [ { "matches": [ "<all_urls>" ], "js": [ "src/content_script.js" ], "run_at": "document_idle", "all_frames": true } ], "web_accessible_resources": [ { "resources": [ "src/agent/vnext/sessionWorkspace/pickContext.js" ], "matches": [ "<all_urls>" ] } ], "action": { "default_title": "Open Browser Kitten", "default_icon": { "16": "icons/icon-16.png", "32": "icons/icon-32.png", "48": "icons/icon-48.png", "128": "icons/icon-128.png" } }, "content_security_policy": { "extension_pages": "script-src 'self' 'wasm-unsafe-eval'; object-src 'self';", "sandbox": "sandbox allow-scripts; script-src 'self' 'unsafe-eval' 'wasm-unsafe-eval'; object-src 'self';" }, "minimum_chrome_version": "135" } - README.md (7870 bytes)
<a id="top"></a> <p align="center"> <img src="icons/icon-128.png" width="80" height="80" alt="Browser Kitten logo"> </p> <h1 align="center">Browser Kitten</h1> <p align="center"><strong>An agent for the browser you already use.</strong><br>Read signed-in pages, take action, create files, and follow up locally.</p> <p align="center"> <a href="https://github.com/Player-YN/BrowserKitten/actions/workflows/ci.yml"><img src="https://github.com/Player-YN/BrowserKitten/actions/workflows/ci.yml/badge.svg?branch=main" alt="Runtime regression and package integrity CI"></a> <a href="manifest.json"><img src="https://img.shields.io/badge/Chrome-135%2B-ff6a00?style=flat-square" alt="Desktop Chrome 135 or newer"></a> <a href="LICENSE"><img src="https://img.shields.io/badge/license-MIT-626873?style=flat-square" alt="MIT License"></a> <a href="CONTRIBUTING.md"><img src="https://img.shields.io/badge/contributions-welcome-626873?style=flat-square" alt="Contributions welcome"></a> </p> <p align="center"><strong>English</strong> · <a href="README.zh-CN.md">简体中文</a></p> <p align="center"><a href="#quick-start">Quick start</a> · <a href="#configuration">Configuration</a> · <a href="docs/README.md">Documentation</a> · <a href="#current-limits">Current limits</a> · <a href="CONTRIBUTING.md">Contribute</a> · <a href="https://github.com/Player-YN/BrowserKitten/issues">Issues</a></p> --- Browser Kitten is an MIT-licensed Chrome MV3 extension with an agent in the side panel. It works with your existing browser session and keeps conversations, workspace files, and follow-up tasks on this computer. Model and other provider requests use the services you configure. **Current source version: 1.2.0 · Load unpacked · No build step for installation.** ## What you can do | Capability | In practice | | :--- | :--- | | **Read & research** | Understand pages you are signed in to, select relevant content, and collect findings. | | **Act on pages** | Delegate clicks, form entry, and navigation to a separately configured execution model. | | **Create deliverables** | Save documents, HTML pages, data, and files in a session workspace. | | **Follow up** | Arrange local checks, preserve progress, and manage ongoing tasks from a collapsible drawer. | <details> <summary><strong>See the actual side panel — light & dark</strong></summary> <br> <p align="center"> <img src="docs/assets/sidepanel-light.png" width="300" alt="Actual Browser Kitten side panel in light mode, with an unsent example task"> <img src="docs/assets/sidepanel-dark.png" width="300" alt="Actual Browser Kitten side panel in dark mode, with an unsent example task"> </p> <p align="center"><sub>Captured from the extension in a fresh browser profile. The example is an unsent draft, not a completed run.</sub></p> </details> ## Quick start 1. **[Download the current source ZIP](https://github.com/Player-YN/BrowserKitten/archive/refs/heads/main.zip)** and extract it into a folder you will keep. 2. Open `chrome://extensions` in desktop Chrome, enable **Developer mode**, then choose **Load unpacked**. Select `BrowserKitten-main`, the folder that directly contains `manifest.json`. 3. Pin **Browser Kitten**, refresh an ordinary website tab, open the side panel, and configure your providers below. You do **not** need Node.js, Python, npm, or a build command to install it. The source folder itself is the extension load root. > **Trying the latest work?** The download above follows `main`. The separate GitHub Releases page may contain older tagged versions. For the full walkthrough, permissions, update instructions, and troubleshooting, see **[Getting started](docs/GETTING_STARTED.md)** · **[中文安装指南](docs/GETTING_STARTED.zh-CN.md)**. ## Configuration Open **Settings** from the side-panel menu. For a bring-your-own-key setup: | Setting | Needed for | Enter | | :--- | :--- | :--- | | **Inference** | Planning, analysis, and tool calls | Base URL, API key, and a tool-capable model ID | | **System1** | Page clicks and form interactions | A separate execution-model configuration using an OpenAI-compatible `/chat/completions` endpoint | | Image / Transcription / Search | Additional media and web capabilities | Configure only the services you need | Inference and System1 can use the same provider, but must be configured separately. The unpacked build accepts your own keys without a Browser Kitten subscription; the external providers still charge for usage. Account-hosted services are optional and depend on the separate backend. Try a small first task: > Summarize the current page and save its key findings as a document. **Enter** inserts a newline. **Ctrl+Enter / Cmd+Enter** sends. Enable page scripts and capture permissions only when the task needs them; see the [permission guide](docs/GETTING_STARTED.md#permissions). ## Local follow-ups Describe the goal, permitted actions, reporting conditions, and when to stop. The agent can arrange or remove wakeups within that goal. Manage them in the **collapsible task drawer above the input**. - Chrome must remain running and the computer awake. Closing the side panel is fine. - Checks run on a minute-scale schedule; immediate webpage-event subscriptions are not implemented. - New scheduled tasks can stay quiet when nothing meaningful changes, and report actions, changes, or requests for help. See [follow-up behavior and controls](docs/GETTING_STARTED.md#local-follow-ups). ## Current limits - This is a local browser extension; there is no cloud execution environment or native OS / terminal control. - The host currently enforces **per-tab leases** and **per-session execution slots**, rather than a global browser lock. Automatic grouping of all agent-controlled website tabs is not implemented. Run one browser-operating task at a time. - Restricted Chrome pages and the Web Store are not ordinary automation targets. Browser smoke tests do not establish success on every website or with every model. - Settings and workspaces are local to this Chrome profile. They do not automatically move to another computer. Relevant task content may be sent to your configured providers. Read [updates and data](docs/GETTING_STARTED.md#updates-and-data) before replacing an installation. ## Documentation | Start here | Go deeper | | :--- | :--- | | [Getting started](docs/GETTING_STARTED.md) · [安装指南](docs/GETTING_STARTED.zh-CN.md) | [Architecture](AGENTS.md) · [Agent runtime](src/agent/AGENTS.md) *(中文)* | | [Documentation index](docs/README.md) | [Tools & Workflow](docs/PERSONAL_LIBRARY.md) · [Workflow contract](docs/WORKFLOW_V1.md) *(中文)* | | [Troubleshooting & support](SUPPORT.md) | [UI design](Design.md) · [Media](docs/CHAT_MEDIA_INPUT.md) *(中文)* | | [Changelog](CHANGELOG.md) | [Security policy](SECURITY.md) · [Third-party notices](THIRD_PARTY_NOTICES.md) | ## Contributing Bug reports, documentation fixes, and focused pull requests are welcome. Start with **[CONTRIBUTING.md](CONTRIBUTING.md)** for setup and checks. Use a [bug report](https://github.com/Player-YN/BrowserKitten/issues/new?template=bug_report.yml) or [feature request](https://github.com/Player-YN/BrowserKitten/issues/new?template=feature_request.yml) to give maintainers useful context. Please follow the [Code of Conduct](CODE_OF_CONDUCT.md). Report vulnerabilities through [private security reporting](https://github.com/Player-YN/BrowserKitten/security/advisories/new), rather than a public issue. No response-time commitment is implied. ## License & acknowledgments [MIT](LICENSE) © 2026 Yiteng (Player-YN). Built with open-source projects including Vercel AI SDK, QuickJS, esbuild, Zod, Lucide, and fflate. See [third-party notices](THIRD_PARTY_NOTICES.md) for their respective licenses. <p align="right"><a href="#top">Back to top ↑</a></p>
- README.zh-CN.md (7430 bytes)
<a id="top"></a> <p align="center"> <img src="icons/icon-128.png" width="80" height="80" alt="Browser Kitten logo"> </p> <h1 align="center">Browser Kitten</h1> <p align="center"><strong>在你正在使用的浏览器里,把任务推进到完成。</strong><br>读取已登录网页 · 操作页面 · 生成文件 · 本地持续跟进</p> <p align="center"> <a href="https://github.com/Player-YN/BrowserKitten/actions/workflows/ci.yml"><img src="https://github.com/Player-YN/BrowserKitten/actions/workflows/ci.yml/badge.svg?branch=main" alt="Runtime regression and package integrity CI"></a> <a href="manifest.json"><img src="https://img.shields.io/badge/Chrome-135%2B-ff6a00?style=flat-square" alt="Desktop Chrome 135 or newer"></a> <a href="LICENSE"><img src="https://img.shields.io/badge/license-MIT-626873?style=flat-square" alt="MIT License"></a> <a href="CONTRIBUTING.md"><img src="https://img.shields.io/badge/contributions-welcome-626873?style=flat-square" alt="Contributions welcome"></a> </p> <p align="center"><a href="README.md">English</a> · <strong>简体中文</strong></p> <p align="center"><a href="#快速开始">快速开始</a> · <a href="#模型配置">模型配置</a> · <a href="docs/README.md">文档导航</a> · <a href="#当前边界">当前边界</a> · <a href="CONTRIBUTING.md">参与贡献</a> · <a href="https://github.com/Player-YN/BrowserKitten/issues">问题反馈</a></p> --- Browser Kitten 是 MIT 开源的 Chrome MV3 扩展,在侧栏中运行通用 Agent。它使用你已有的浏览器登录状态,将会话、工作区文件和持续任务保存在本机;模型及其他服务请求使用你配置的服务商。 **当前源码版本:1.2.0 · 加载已解压的扩展程序 · 安装无需构建。** ## 可以做什么 | 能力 | 具体用途 | | :--- | :--- | | **读取** | 理解已登录网页,点选相关内容,收集并整理发现。 | | **操作** | 通过单独配置的执行模型完成点击、填写和页面导航。 | | **交付** | 在会话工作区保存文档、HTML 网页、数据和文件。 | | **跟进** | 安排本地检查、保存进度,在可收起的任务抽屉中管理长任务。 | <details> <summary><strong>查看真实侧栏界面:浅色与深色</strong></summary> <br> <p align="center"> <img src="docs/assets/sidepanel-light.png" width="300" alt="Browser Kitten 浅色侧栏真实截图,输入框内为尚未发送的示例任务"> <img src="docs/assets/sidepanel-dark.png" width="300" alt="Browser Kitten 深色侧栏真实截图,输入框内为尚未发送的示例任务"> </p> <p align="center"><sub>在独立浏览器配置中截取。截图为英文界面,任务仅为未发送的草稿,并非执行成功演示。</sub></p> </details> ## 快速开始 1. **[下载当前源码 ZIP](https://github.com/Player-YN/BrowserKitten/archive/refs/heads/main.zip)**,解压到一个准备长期保留的位置。 2. 在桌面 Chrome 打开 `chrome://extensions`,开启 **开发者模式**,点击 **加载已解压的扩展程序**,选择直接包含 `manifest.json` 的 `BrowserKitten-main` 文件夹。 3. 固定 **Browser Kitten**,刷新普通网站标签,点击扩展图标打开侧栏,再按下方说明配置模型。 安装**不需要 Node.js、Python、npm 或构建命令**,源码根目录本身就是可加载的扩展。 > **体验最新代码:**上面的下载链接跟随 `main`;GitHub Releases 中的历史标签版本可能更旧。 完整步骤、授权、更新和故障处理见 **[中文安装指南](docs/GETTING_STARTED.zh-CN.md)** · **[English guide](docs/GETTING_STARTED.md)**。 ## 模型配置 从侧栏菜单进入 **设置**,使用自己的 API Key 时: | 设置 | 用途 | 需要填写 | | :--- | :--- | :--- | | **推理** | 理解任务、分析和调用工具 | Base URL、API Key、支持工具调用的模型 ID | | **System1** | 网页点击、填写与控件选择 | 单独配置执行模型,使用 OpenAI-compatible `/chat/completions` 接口 | | 生图 / 转写 / 搜索 | 额外的媒体与网络能力 | 按任务需要配置 | 推理和 System1 可以使用同一服务商,但要分别配置。unpacked 版本默认允许自带 Key,无需 Browser Kitten 订阅;外部 API 调用仍由对应服务商计费。账户积分服务可选,依赖独立后端。 先试一个小任务: > 总结当前页面的主要内容,并把关键结论整理成一份文档。 **Enter 换行,Ctrl+Enter / Cmd+Enter 发送。** 页面脚本、录音或录屏按需授权,详见[权限说明](docs/GETTING_STARTED.zh-CN.md#页面脚本与录音--录屏)。 ## 本地持续任务 说明目标、允许的操作、报告条件和结束条件,Agent 可在目标内自行安排或移除唤醒。任务位于**输入框上方可收起的任务抽屉**中。 - Chrome 需保持运行,电脑需处于唤醒状态;关闭侧栏后仍可续跑。 - 当前按分钟级间隔检查,尚未实现网页事件即时订阅。 - 新定时任务在没有有意义变化时可以静默更新状态,在有变化、采取行动或需要帮助时报告。 详见[持续任务与控制](docs/GETTING_STARTED.zh-CN.md#持续任务与定时检查)。 ## 当前边界 - 当前为本机浏览器扩展,没有云端执行环境,也没有原生 OS 或终端控制。 - 宿主实现的是**按标签页租约**及**按会话执行槽**,尚未实现全浏览器独占及所有 Agent 网页自动入组。体验时请一次运行一个需要操作浏览器的任务。 - Chrome 内部页、Chrome Web Store 等受限页面不能按普通网页操控;浏览器烟测不代表所有网站和模型均已验收。 - 设置与工作区属于当前 Chrome 配置,不会自动跨电脑迁移;执行任务所需内容可能发送给你配置的服务商。 更新前请阅读[更新、数据与能力边界](docs/GETTING_STARTED.zh-CN.md#更新数据与能力边界)。 ## 文档导航 | 从这里开始 | 深入了解 | | :--- | :--- | | [安装指南](docs/GETTING_STARTED.zh-CN.md) · [English guide](docs/GETTING_STARTED.md) | [架构](AGENTS.md) · [Agent runtime](src/agent/AGENTS.md) | | [完整文档索引](docs/README.md) | [个人工具](docs/PERSONAL_LIBRARY.md) · [Workflow](docs/WORKFLOW_V1.md) | | [故障与支持](SUPPORT.md) | [UI 规范](Design.md) · [媒体输入](docs/CHAT_MEDIA_INPUT.md) | | [变更记录](CHANGELOG.md) | [安全政策](SECURITY.md) · [第三方许可](THIRD_PARTY_NOTICES.md) | ## 参与贡献 欢迎问题报告、文档修正和聚焦的 PR。开发环境、检查方式见 **[CONTRIBUTING.md](CONTRIBUTING.md)**。可通过[缺陷报告](https://github.com/Player-YN/BrowserKitten/issues/new?template=bug_report.yml)或[功能建议](https://github.com/Player-YN/BrowserKitten/issues/new?template=feature_request.yml)提供上下文,使用中文或英文均可。 请遵守[社区行为规范](CODE_OF_CONDUCT.md)。安全漏洞请走[私密报告入口](https://github.com/Player-YN/BrowserKitten/security/advisories/new),不要公开提交利用细节;项目未承诺固定响应时间。 ## 许可证与致谢 [MIT](LICENSE) © 2026 Yiteng (Player-YN)。使用了 Vercel AI SDK、QuickJS、esbuild、Zod、Lucide 和 fflate 等开源项目;各自许可证见[第三方许可说明](THIRD_PARTY_NOTICES.md)。 <p align="right"><a href="#top">返回顶部 ↑</a></p>
- SECURITY.md (1839 bytes)
# Security policy ## Report privately Use **[GitHub private vulnerability reporting](https://github.com/Player-YN/BrowserKitten/security/advisories/new)** for suspected vulnerabilities. Do not publish exploit details, API keys, tokens, cookies, or private browser content in issues or pull requests. Include the affected commit or extension version, Chrome version, a minimal reproduction, expected impact, and a proposed mitigation if you have one. Synthetic test pages and redacted examples are preferable to real account data. Give maintainers an opportunity to investigate before public disclosure. No response deadline or bounty program is promised. ## Versions and scope Please check whether the issue reproduces on current `main`; older tagged builds may not include subsequent fixes. There is no published long-term support window. This policy covers code in this repository. Hosted account services and model providers are separate systems; identify which component is affected. Browser Kitten runs inside a signed-in browser and uses broad webpage permissions. Prompts and tab groups do not establish a security boundary. Page ownership, sandboxing, host gates, and dispatch checks are relevant controls, but they do not justify blanket claims about preventing every harmful action. Local storage is not a credential vault, and necessary task content may be sent to configured providers. Setup, data handling, and current implementation limits are documented in [Getting started](docs/GETTING_STARTED.md) and the [runtime architecture](AGENTS.md). **中文:**安全漏洞请通过上方 GitHub 私密漏洞报告入口反馈。请附受影响版本、Chrome 版本和最小复现,不要在公开 Issue 中上传密钥、Cookie、私人页面内容或利用细节。项目未承诺固定响应时限或漏洞奖金。
- SKILL.md (357 bytes)
--- name: Page Restyle description: User wants the current http(s) tab restyled in place — hide named clutter, a calmer reading layout, contrast, or a floating TOC. --- The live document is the outcome. One tagged `<style>`; undo = remove that node. Idempotent id (`#paw-restyle`). After navigate or SPA remount, inject again. This is not a saved clone.
- src/agent/AGENTS.md (33971 bytes)
# src/agent — Session Workspace Runtime 进程分层见根 [AGENTS.md](../../AGENTS.md)。Chrome 运输见 [../AGENTS.md](../AGENTS.md)。 本目录是 **offscreen 里跑的会话 runtime**。侧栏只 `import` 设置、标签、workspace 客户端;**不**在侧栏跑 `ToolLoopAgent`。 入口:`vnext/service/sessionWorkspaceService.js`。每条用户消息:`vnext/sessionWorkspace/sendMessage.js` → AI SDK 7 `ToolLoopAgent`(`sessionAgent.js`,`toolChoice=auto`)。没有手写多步 tool loop,没有 chat/run 模式分裂。 ```text sidepanel workspaceRpc('sendMessage') → SessionWorkspaceService.sendMessage → sendMessage(store, input) → createSessionTools + runSessionToolLoopAgent ``` 没有 barrel / `index.js` re-export 层。每个 import 都写具体文件路径:`offscreen/runtime.js` → `vnext/service/sessionWorkspaceService.js` → `vnext/runSession.product.js` → `vnext/sessionWorkspace/index.js`。新增模块不要再建 barrel。 ## 目录 | 路径 | 职责 | |------|------| | `vnext/service/` | `SessionWorkspaceService`:RPC 门面、abort 注册、把 host 回调接到 SW | | `vnext/sessionWorkspace/` | 领域:store、group、artifact、FS、prompt、tools、office、sendMessage | | `vnext/primitives/` | `acquire` / `run` 宿主原语(工具层调用,不是模型直接 API) | | `vnext/adapters/` | QuickJS `codeRuntime`、sandboxClient、`vendor/ai-sdk-loader.mjs` | | `vnext/host/` | `workspaceRpc`、`browserSysHost`、`tabLease`、`taskScheduler`、`accessPolicy`、`riskClassify`、`dispatchTicket`、`callJournal`、`operationGate`、`postcondition` | | `vnext/skills/` | 打包 playbook(`SKILL.md` + `skillSource.js`)。**不是工具** | | `llm.js` `provider.js` `modelCatalog.js` | BYOK OpenAI-compatible HTTPS;每轮 `resolveLanguageModel` | | `contextWindowProbe.js` `contextWindowStore.js` | 目录无窗口时的独立 chat 探测与 `browserbot_context_window_evidence_v1`;不进 ToolLoopAgent,发送不 await | | `webAcquireSettings.js` | `browserbot_web_acquire` | | `byokGate.js` | 自己的 Key 需要付费套餐:`browserbot_cloud_entitlement` 缓存、`byokAllowed`、`BYOK_REQUIRES_PLAN`;unpacked 豁免 | | `skills.js` | 用户固化 skill(`browserbot_user_skills`),侧栏用;与 `vnext/skills` 不同 | | `artifacts.js` `draftStore.js` `state.js` `trajectory.js` `documentRender.js` `render/` | 旧表面。当前交付物走 `sessionWorkspace/artifacts.js`。改功能前先确认调用方 | ## 领域与 store `DurableSessionWorkspaceStore`(`durableStore.js`)继承内存 `SessionWorkspaceStore`,实际运行使用 durable store。集合: `sessions` · `groups` · `groupMembers` · `items` · `sessionBindings` · `artifacts` · `fsNodes` · `meta`;blobs 在 OPFS。`executions` / `leases` 是回合书,崩溃后作废。可选 durable task 存在 `meta` 键 `task:{taskId}`(进 IDB 快照);`ensureTaskForMessage` 只给测试,实际 `sendMessage` 不调用它。 工具访问合同(`tools.js` 文件头):**SelectionGroup / WebItem / sessionBindings 不能经工具改**。只有 UI RPC(`createGroup`、`bindGroups`、`syncTabSelection`、clipboard pin…)可以。Guest `run(code)` 没有 store、没有 `chrome.*`。浏览器机器经 **`sys`**(`browserSys.js` → SW `workspace_sys`): | 调用 | 作用 | |------|------| | `sys.help()` / `inspect view=sys` | ABI 目录(`browserbot-sys-v1`) | | `sys.capabilities()` | 实时探测用户脚本、debugger、截图、下载可用性与大小限制 | | `sys.tabs.list` / `current` / `frames` | 标签与 frame(进程表) | | `sys.tabs.open` / `navigate` / `reload` / `close` / `focus` | 进程控制(已有 `tabs` 权限) | | `sys.eval({ world, code, tabId, frameId })` | `code` 是 async 函数体。`MAIN` = 页面 JS 堆;`USER` = 自有世界 + DOM | | `sys.fetch({ as:'page'\|'extension', url, tabId, init })` | 两块网卡:页面身份(MAIN fetch,cookies+Referer;用户要的登录态/验证码/本机 IP 链接默认这条)vs 扩展身份(`credentials:'omit'`,无 cookie)。省略 `as` 时宿主仍当 extension | | `sys.cdp` | CDP 管道:`{ method, params }` 自动 attach;`action: attach\|detach\|events\|targets`。能在页面跑脚本的命令(`Runtime.evaluate` 等)与 `sys.eval` 同一开关:`userScripts` 未授予或不可用 → `PERMISSION_REQUIRED` + `permission:'userScripts'`(`cdpScriptGate.js`) | | `sys.download` / `sys.screenshot` | `chrome.downloads.download`:本 profile cookie jar + 本机 IP,无标签 Referer(不是 extension fetch)。登录态/Referer/验证码优先 `as:'page'`。截图为视口合成 | | `sys.upload` | 把 guest FS(`/scratch` 最常用,也接受 `/artifacts` `/context`)或 `itemId`/`artifactId` 挂到页面 file input / dropzone。默认 MAIN-world `DataTransfer` + `input.files` + 手派 `input`/`change`;找不到 input 再脚本 drop。`auto` 绝不走 CDP。回执 `siteAccepted:'unknown'`,`trusted:false`(脚本事件)。不是「站点已接受」 | `eval` / page `fetch` 走 `chrome.userScripts.execute`。`sys.upload` 走 `chrome.scripting.executeScript({ world:'MAIN', func, args })` 分块注入字节,不走 `sys.eval` 的 100k 源码顶。`sys.cdp` 走 `chrome.debugger`(浏览器调试协议管道)。DevTools 已挂上时会 `CDP_BUSY`。guest `sys.cdp` 的脚本类命令有宿主门(`host/cdpScriptGate.js`,在 `browserSysHost.sysCdp` 里、占租约/消费 ticket/attach 之前判定):**白名单**——只有观察与可信输入类 domain(`Input` `Page` `DOM` `Network` `Accessibility` `CSS` `Emulation` `Target` 等)且不在其中的脚本例外名单(`Page.addScriptToEvaluateOnNewDocument` / `createIsolatedWorld` / `setDocumentContent` / `setBypassCSP`、`DOM.setOuterHTML`、事件处理器或 `javascript:` 的 `DOM.setAttribute*`、非 http(s)/`about:blank` 的 `Page.navigate` / `Target.createTarget`、`Target.attachToTarget` 等子会话)不需要开关;`Runtime`(除 enable/getProperties 等只读)、`Debugger`、`Fetch` 和未知 domain 一律当脚本。这些命令要求 `userScripts` 已授予且 `chrome.userScripts` 可用(复用 `userScriptsDenied`),否则 `PERMISSION_REQUIRED` + `permission:'userScripts'`,不 attach、不出横幅。宿主自己的 CDP(坐标点击、`Page.getLayoutMetrics`、上传 `method:'cdp'`、attach 回收探针 `Runtime.evaluate 'void 0'`)不经过这道门。返回值必须能 JSON 序列化。`eval` / page `fetch` / `cdp` / `upload` 只允许 http(s) 可注入页;扩展预览页返回 `NEED_PAGE`(MAIN world 不能碰到 `chrome.*`)。包括 `tabs.current` 在内,需要目标标签的操作未带 `tabId`/`defaultTabId` 都是 `NEED_PAGE`,不回退焦点标签。`eval` / `waitFor` / page `fetch` / `tabs.navigate|reload|focus|close` / `cdp` / `upload` 会先占 storage.session 中的 tab 租约,他 session 占用同一 tab → `TAB_LEASED`。可变 `sys` 还要过 Access Policy + journal ticket:已知付款 `PAYMENT_DENIED`,已知删除等审批,其余(含 raw eval、可变 CDP、`upload method:'cdp'`)best-effort 放行,**不能**当成绝不付款/删除。`SYS_ABORTED` / `SYS_TIMEOUT` 表示等待结束,不表示副作用已撤销。可见标签截图失败为 `TAB_NOT_VISIBLE` / `TARGET_CHANGED`。`targetId` 经 `getTargets` 校验 URL 并归一到 page tabId,按 execution 归属与释放 CDP。`eval/waitFor/page-fetch/upload` 用 documentIds 定向,可传 documentId/expectedUrl 校验先前观察。宿主在 `sys.waitFor` 进行时把本次 `run` deadline 顶到覆盖 waitFor(默认 30s / 最大 120s),不把所有 run 默认改成 120s。 宿主政策与 journal(不是 task,不是 audit): - 只有一种模式,没有开关。`decideAccess(classified)`:已知付款 deny,已知删除 approve,其余 auto。 - 分类在 `riskClassify.js`:忽略模型 `risk`/`intent`。已知付款 deny;已知删除要 `approvalGate` one-shot(UI 不是 clarify)。 - `operationGate.gatedDispatch` 在 offscreen 写 journal、等人、发 ticket;SW `consumeDispatchTicket` 是最后一道门。 - journal 库:`browserbot-call-journal-v1`。启动把未收束的 `dispatched` 标 `unknown`。verified 不重放。 - `postcondition.js` 只认宿主事实。不可证明的 POST/send → `needs_human`。 `sys.fetch` / `sys.screenshot` 可传 `saveTo: '/scratch/…' | '/artifacts/…'`:宿主写入 guest FS,返回文件回执;`run` 登记交付物。调用携带身份与截止时间,错误保留 `code`。停止可中止扩展 fetch,但已派发的页面/CDP 副作用可能已发生,超时后需查证状态。 Artifact 具有 `revision`;`updateArtifact` 可携带 `expectedRevision`,不同内容的过期写入返回 `ARTIFACT_CONFLICT`。文档(`doc`)与站点(`web`)的常用写入路径已接入。csv / xlsx 按文件打开,没有表格画布工具。raw guest 写主文件推进 revision,但仍不是带读版本的条件写入。 模型可见面:没有单独的 `sys` 工具。ISA 写在 `run` 的 `code` 字段(`SYS_MODEL_HINT`);`run.description` 只指向该字段与 `inspect view=sys`。`inspect.view` enum 含 `sys`、每轮 world 有 `browserSys=browserbot-sys-v1`。完整目录仍是 `inspect view=sys` 或 guest `sys.help()`。 Guest FS(`fs.js`): | 访客路径 | 宿主映射 | 权限 | |----------|----------|------| | `/context` | `/session/{id}/context` | 只读 | | `/artifacts` | `/session/{id}/artifacts` | 持久读写 | | `/scratch` | `/tmp/{id}/{executionId}` | 本轮;无 execution 则拒绝 | ## 一回合 `sendMessage.js`:写入 user message → `beginExecution` → 组世界索引(bound groups/items、artifact 概览、focusPage)→ `buildSessionAgentInstructions` + 本轮 world block → `createSessionTools` → `inventoryFromSession`(**瞄准目标,不隐藏工具**)→ `runSessionToolLoopAgent`。 普通 `sendMessage` **不创建** durable task。`task start` 弹出确认卡,**只有 UI 回执**才创建任务并写入 `session.activeTaskId`、绑定当前 execution。模型传入 `approved:true` 无效。普通 `/plan` 卡默认勾选「同时开启长任务」:批准且勾选才用同一 UI 回执建 Task 并绑当前 execution;不勾选、拒绝、改计划、或已有非终态 `activeTaskId` 则只钉本回合 `frozenPlan`。`task schedule` 在已绑定任务时更新其调度并 yield,未绑定才创建未来/周期任务。绑定后的新消息作为该任务的补充上下文;`updateTask(detach)` 退出后继续普通聊天。续跑由 SW alarms 或 UI `updateTask(resume)` 再 `sendMessage({ taskRun, taskId, taskContinuation: true })`,不写假用户消息。 `prepareStep`(`makeOfficePrepareStep`)每 hop **读** store(`getTaskContext` → `readTask`)把**用户确认的任务合同**和进度重新注入 instructions。有绑定 Task 时不再注入 `sessionGoal` / `intentState` / 平行的 `frozenPlan`。task 写入只走 `hostTask` → `hostTaskMutation`。`completed` **只**来自成功的 `task complete`;空 plan settle 是 `ready`,未 complete 的回合 settle 是 `ready`/`paused`,abort/deadline 是 `paused`(结果未知),不会写成 `completed`。完成、取消或退出后停止注入该合同。压缩后每 hop 仍从 `session.messages` 注入有界用户原文。 ## 回滚用户消息 侧栏确认后 `workspaceRpc('rewindSession')`(`rewindSession.js`)。先 `abortCurrentExecution` 并等到该会话写完,再按 `messageId`(或第几个 user 的 `userIndex`)把 durable `session.messages` 从这条用户消息起截掉,含这条。`session.compact.throughMessageId` 不在留下的前缀里时删掉 `session.compact`。回执 `restored.content` 由侧栏放回输入框。文件、artifact、已经发生的页面副作用、未完成的 durable task 都不回滚。`/context/conversation.json` 是留下的 `messages` 的只读视图,跟着这条切。`/context/user-requests.json` 还拼上 `session.userClarifications`,被截掉的回合里答过的澄清仍可能在里面。找不到这条消息时 `REWIND_NOT_FOUND`。 ## Compact(模型线,不是 UI 全文) `contextCompact.js`:占用 **本模型窗口 ≥ 80%** 才压。不改写 `session.messages` / artifacts / 组 / guest FS,不新建 durable task。system prefix 与 tool schema 不进摘要。窗口解析:覆盖 → 探测 → 目录 → 家族;压缩数学可用 256k 默认,侧栏 `allowDefault: false`(芯片仍显示未知,不把默认画成已测)。 1. **时机**:`sendMessage` 进 ToolLoop **之前**跑一次 `runSessionCompact`;`prepareStep` 每 hop 估 SDK `messages`,再超 80% 且可切则再压。同一 `throughMessageId` / 同一 hop 不连压。SDK 可返回缩短的 `messages`。 2. **Compact 不定义授权**:摘要 `Goal` 只是历史/发现/进度,可写 `session.sessionGoal` 兼容字段,**不再作为独立目标注入**。用户授权只存在于确认过的 Durable Task 合同。`sessionIntent.js` 仍从持久化原文每 hop 选取有界用户原文;clarify 答案在工具投影前原样持久化。完整原文:`/context/user-requests.json` 与 `/context/conversation.json`。 3. **Tail**:`findCompactCutByBudget` 从末尾在 **user 边界**切开,预算 `min(0.2 * window, 20000)`,至少保留 2 个 user turn。 4. **摘要作者**:主路径 `doGenerate` 写 Goal / Constraints / Decisions / Files/artifacts / Errors / Remaining / Next(1500 output token / 4000 字符 cap,省略显式标注)。重压缩使用前摘要 + 新覆盖的历史,不重复塞最早全部历史。输入按窗口有界,用户要求优先、最近工具证据其次;不再固定截每条 user 的前 400 字或整体前 8000 字。`callModel` 桩与生成失败走有界抽取回退;取消摘要请求沿本轮 signal,未生成替代摘要不推进覆盖标记。 5. **Mid-loop**:已落盘历史更新 `session.compact`;wire replay 保留 source messageId,展开的同一 assistant turn 从最后一条 wire part 后切,避免孤立 tool-result。估算/缩短识别真实 SDK `tool-result.output`,旧结果保留 callId/toolName、格式合法的摘要、路径/结果线索,最近工具结果也受共享正文预算限制;不可省略的协议占位开销单独计入总窗口;预算 `min(20000, 0.2 * window)`。首个 mid-loop compact 当跳即刷新原文提示。UI transcript 不动。 审查证据、边界和验收见 [COMPACT_INTENT.md](../../docs/COMPACT_INTENT.md)。 进度说明(`progressNotes.js`):只取**以工具调用结束的那一步**里模型写的正文(不带工具调用的一步会结束 ToolLoopAgent,成为最终回答,按 finishReason 区分)。不加工具、不加步。`sessionAgent.consumeAgentStream` 在该步第一个 tool-call 之前发 `progress-note { text, stepIndex, toolIndex, at }`(`sendMessage` 补 `sessionId`/`executionId`);运行中写 `session.turnProgress = { executionId, startedAt, notes }`,结束(completed / failed / aborted)写进本回合唯一的 assistant 记录 `progress: [{ at, text, stepIndex, toolIndex }]` 并删掉 `turnProgress`,不额外追加 assistant 消息。`getSession().liveProgress` 只返回当前活动 execution 的槽位。task yield 时 SDK 把最后一步说明当最终文本,这条从 `progress` 去掉(回答拥有它)。说明是模型原话,不是宿主事实,不进 audit。催促:距上条说明(或回合开始)≥ 6 步或 ≥ 60 s,`prepareStep` 在 instructions 末尾加一次 `PROGRESS_NUDGE_TEXT`,同一段空白只催一次;System1 / 标题 / 压缩等轻量调用不经过 prepareStep。 普通回合与 durable-task 回合都无 hop 上限。task 停点:`task wait` / `task pause` / `task complete` / 更新当前任务的 `task schedule` / host yield / abort(`createTaskStopState`)。成功 yield 之后 gate 抑制后续(含同步兄弟)工具。定时任务另有单轮时间预算,默认 5 分钟(`maxRoundMinutes` 1–30)。非法 tool call 会 `repairSessionToolCall` 一次。 清单:`canvasInventory.js` 的 `SESSION_TOOL_NAMES`(含 `task`)。`toolSchedule.js` 原样返回该列表。 The fixed English Agent personality lives in `prompt.js` (`SYSTEM_PROMPT_VERSION = 'v27-agent-personality'`): autonomous execution, resource discovery, evidence, continuity, follow-up and truthful communication. `runtimeInstructions.js` separately preserves the delegate, media, progress-note and workspace-file contracts; `sendMessage.js` composes both with the existing skill catalog, library instructions and Task contract. Media routes remain appended by prepareStep from current connection evidence. The personality asks for a browser-level Agent tab group and exclusive browser control; those are behavioral instructions, not implemented global ownership guarantees (the host currently enforces per-tab leases and per-session execution slots). `createSession` scaffolds `/artifacts/AGENTS.md`, `Memory.md` and `STATUS.md`. Bounded AGENTS text is injected each hop; settle rewrites its pointers, and these files never grant user authorization. Memory holds cross-turn facts; STATUS holds everyday progress, with Task checkpoints taking precedence when bound. Ordinary sends do not create Tasks; the Agent may arrange follow-ups through task tools within the authorized goal. Recipes remain in skills, loaded on demand through `inspect view=skill`. Side-panel current/next text is not injected into the prompt. 交付物打开:`openClassify.js` 负责 kind;`artifactCapability.js` 是家族 × 打开面 × 徽标的单一映射。主 chip 是 `docs` / `data` / `web` / `media` / `files`;`design`/`slides` 只兼容旧数据。未知 HTML/JS 不得在 extension origin 执行。仅文件名的 `.html` 保持中性,不假装 proven site/doc;首次打开分类后可持久化 `capability` hint。site 本波 = 可编辑/可渲染的静态 HTML+CSS + 宿主 postMessage bridge(`src/sandbox/siteFrame.*`),**不是**任意 guest JS。 ## 当前工具清单 | id | 定义 | 现状 | |----|------|------| | `inspect` | `sessionWorkspace/tools.js` | 只读查找:`view` = groups / group / item / artifacts / files / skill / workbook / range / html / sys | | `acquire` | 同上 | 把未知公开网带进会话:search / fetch / map / crawl / image / note / transcribe。已打开或需登录/验证码的 URL 走 `run` + `sys.fetch as:"page"`。`transcribe` 读 guest path / artifactId / 公开音频 URL,走设置里的转写 API(`purpose:'stt'`) | | `run` | 同上 | 访客机:sandbox JS/TS + fs + `sys`。`op` 登记文档、站点、文件。日常改文档走 `doc`,改站点走 `web` | | `clarify` | 同上 | 模型主动提问或 plan 卡。删除/模糊提交的门是宿主 `answerApproval`,不依赖模型调用 clarify | | `action` | 同上 | 本轮 `activeTab` 的显式 `tabId` live-page(snapshot 后同代 mutate)。`op=listen` 听当前标签声音(需侧栏授权),画面仍用 snapshot。无 tabId → `NEED_EXPLICIT_TAB`。运输见 [../AGENTS.md](../AGENTS.md) | | `task` | `sessionWorkspace/taskTool.js` | 合同、进度与观察:`start` / `amend` / `inspect` / `plan` / `checkpoint` / `observe` / `wait` / `pause` / `complete` / `schedule` / `unschedule`。`start`/`amend` 走确认卡。`wait`/`pause`/`complete` 成功即 yield。`schedule` 更新当前任务并 yield,未绑定才新建;`unschedule` 去掉后续唤醒,继续本轮 | | `doc` | 同上 | 标记 HTML 文档:`act` = read / write。无文件 → `NO_DOCUMENT` | | `web` | 同上 | `data-browserbot-kind=site`:`act` = read / write / undo / clone / capture。画布是静态 HTML+CSS;宿主 bridge 在 sandbox frame,用户脚本仍被剥 | 网站没有对应页时返回 `NO_CANVAS`。文档未登记返回 `NO_DOCUMENT`。网站复刻走 `web act=clone`。 BYOK 门(`byokGate.js`):非 `bk_cloud_*` 的供应商与非 browserkitten.com 的 Tavily/Firecrawl 槽,只在已登录且套餐为 developer/standard/pro/max 时可用。卡点:`SessionWorkspaceService.resolveLanguageModel`(回合、压缩、标题、/learn、选区 chips 都从这里拿模型,抛 `BYOK_REQUIRES_PLAN`)· `imageGen.generateSessionImage` · `transcribe.transcribeSessionAudio` · `sendMessage` 里 `gateWebAcquire`(本轮把自有 web 槽置空并记 `byokBlocked`;search/map/crawl 返回 `BYOK_REQUIRES_PLAN`,fetch 回退匿名 GET)· SW `pageActionHost.runChooseAction`(System1)· SW 截图 OCR · 后台探测(`MediaCapabilityChecks` / `ContextWindowChecks` 无套餐时静默跳过)。设置里的「测试 / 拉模型列表」不受限(配置行为)。缓存 `browserbot_cloud_entitlement = { plan, checkedAt }`(72 h 有效,>6 h 先刷新 `/api/me`,网络失败按 72 h;退出登录/4 - src/agent/anthropicProvider.js (16269 bytes)
/** * Fetch-based Anthropic Messages API LanguageModel (AI SDK v2 shape). * Speaks POST {base}/messages with x-api-key + anthropic-version. * ToolLoopAgent-compatible: maps tools, tool_use, and tool_result. */ import { anthropicMessagesUrl, CHAT_PROTOCOL_ANTHROPIC } from './llm.js'; import { isAbortLike, toAbortError } from './vnext/host/userStop.js'; export const ANTHROPIC_VERSION = '2023-06-01'; export const ANTHROPIC_DEFAULT_MAX_TOKENS = 4096; /** * @param {string} apiKey * @returns {Record<string, string>} */ export function anthropicRequestHeaders(apiKey) { return { 'content-type': 'application/json', 'x-api-key': String(apiKey || ''), 'anthropic-version': ANTHROPIC_VERSION }; } /** * Extract a JSON Schema object from an AI SDK tool definition. * @param {any} tool */ export function anthropicToolInputSchema(tool) { const raw = tool?.inputSchema?.jsonSchema || tool?.inputSchema || tool?.parameters?.jsonSchema || tool?.parameters || null; if (raw && typeof raw === 'object' && !raw._def && (raw.type || raw.properties)) { return { type: raw.type || 'object', properties: raw.properties && typeof raw.properties === 'object' ? raw.properties : {}, ...(Array.isArray(raw.required) ? { required: raw.required } : {}), ...(raw.additionalProperties !== undefined ? { additionalProperties: raw.additionalProperties } : {}) }; } return { type: 'object', properties: {} }; } /** * Map AI SDK tools → Anthropic tools[]. * @param {any[]|null|undefined} tools */ export function convertToolsToAnthropic(tools) { if (!Array.isArray(tools) || !tools.length) return undefined; const out = []; for (const t of tools) { if (!t || typeof t !== 'object') continue; if (t.type === 'provider-defined') continue; const name = String(t.name || t.toolName || '').trim(); if (!name) continue; out.push({ name, description: String(t.description || ''), input_schema: anthropicToolInputSchema(t) }); } return out.length ? out : undefined; } /** * @param {any} toolChoice * @param {any[]|undefined} anthropicTools */ export function convertToolChoiceToAnthropic(toolChoice, anthropicTools) { if (!anthropicTools?.length) return undefined; if (toolChoice == null || toolChoice === 'auto') return { type: 'auto' }; if (toolChoice === 'none' || toolChoice === 'required') { return toolChoice === 'none' ? { type: 'auto' } : { type: 'any' }; } if (typeof toolChoice === 'object') { if (toolChoice.type === 'none') return { type: 'auto' }; if (toolChoice.type === 'required' || toolChoice.type === 'any') return { type: 'any' }; if (toolChoice.type === 'tool' && toolChoice.toolName) { return { type: 'tool', name: String(toolChoice.toolName) }; } if (toolChoice.type === 'auto') return { type: 'auto' }; } return { type: 'auto' }; } function extractText(content) { if (content == null) return ''; if (typeof content === 'string') return content; if (!Array.isArray(content)) return String(content); return content .map((p) => { if (typeof p === 'string') return p; if (p?.type === 'text') return p.text || ''; return ''; }) .join(''); } function parseToolInput(input) { if (input == null) return {}; if (typeof input === 'object') return input; if (typeof input === 'string') { try { return JSON.parse(input || '{}'); } catch { return { _raw: input }; } } return {}; } function toolResultContent(output) { if (output == null) return ''; if (typeof output === 'string') return output; if (typeof output === 'object') { // AI SDK often wraps as { type: 'json'|'text'|'error-…', value|text|… } if (output.type === 'text' && typeof output.text === 'string') return output.text; if (output.type === 'json') { try { return JSON.stringify(output.value ?? output); } catch { return String(output.value ?? ''); } } if (typeof output.value === 'string') return output.value; try { return JSON.stringify(output.value ?? output); } catch { return String(output); } } return String(output); } function userContentParts(content) { if (typeof content === 'string') return content; if (!Array.isArray(content)) return extractText(content); /** @type {any[]} */ const parts = []; for (const part of content) { if (!part || typeof part !== 'object') continue; if (part.type === 'text' && part.text) { parts.push({ type: 'text', text: String(part.text) }); continue; } if (part.type === 'image' || part.type === 'file') { const mediaType = String(part.mediaType || part.mimeType || 'image/jpeg') .split(';')[0] .trim(); if (!mediaType.startsWith('image/')) continue; let data = part.data ?? part.image; if (typeof data === 'object' && data?.data != null) data = data.data; if (typeof data === 'string' && data.startsWith('data:')) { const m = /^data:([^;]+);base64,(.+)$/i.exec(data); if (m) { parts.push({ type: 'image', source: { type: 'base64', media_type: m[1] || mediaType, data: m[2] } }); continue; } } if (typeof data === 'string' && data.length > 32) { parts.push({ type: 'image', source: { type: 'base64', media_type: mediaType, data: data.replace(/^data:[^,]+,/, '') } }); } } } if (!parts.length) return extractText(content) || ''; if (parts.length === 1 && parts[0].type === 'text') return parts[0].text; return parts; } /** * Convert AI SDK LanguageModel prompt → Anthropic Messages body fields. * @param {any} prompt * @returns {{ system?: string, messages: object[] }} */ export function convertPromptToAnthropicMessages(prompt) { /** @type {object[]} */ const messages = []; /** @type {string[]} */ const systemParts = []; const pushUser = (content) => { if (content == null || content === '') return; const last = messages[messages.length - 1]; if (last?.role === 'user' && Array.isArray(last.content) && Array.isArray(content)) { last.content.push(...content); return; } messages.push({ role: 'user', content }); }; const list = Array.isArray(prompt) ? prompt : [{ role: 'user', content: String(prompt ?? '') }]; for (const part of list) { if (!part || typeof part !== 'object') continue; if (part.role === 'system') { const text = typeof part.content === 'string' ? part.content : extractText(part.content); if (text) systemParts.push(text); continue; } if (part.role === 'user') { pushUser(userContentParts(part.content)); continue; } if (part.role === 'assistant') { /** @type {any[]} */ const blocks = []; const content = part.content; if (typeof content === 'string') { if (content) blocks.push({ type: 'text', text: content }); } else if (Array.isArray(content)) { for (const c of content) { if (!c || typeof c !== 'object') continue; if (c.type === 'text' && c.text) { blocks.push({ type: 'text', text: String(c.text) }); } else if (c.type === 'tool-call' || c.type === 'tool_use') { blocks.push({ type: 'tool_use', id: String(c.toolCallId || c.id || ''), name: String(c.toolName || c.name || ''), input: parseToolInput(c.input ?? c.args) }); } else if (c.type === 'reasoning' && c.text) { // Anthropic extended thinking is separate; skip plaintext reasoning in replay. } } } if (blocks.length) messages.push({ role: 'assistant', content: blocks }); continue; } if (part.role === 'tool') { /** @type {any[]} */ const results = []; const content = part.content; if (Array.isArray(content)) { for (const c of content) { if (!c || typeof c !== 'object') continue; if (c.type === 'tool-result' || c.type === 'tool_result') { results.push({ type: 'tool_result', tool_use_id: String(c.toolCallId || c.tool_use_id || ''), content: toolResultContent(c.output ?? c.content ?? c.result) }); } } } if (results.length) pushUser(results); } } // Anthropic requires the first message to be user. if (messages.length && messages[0].role !== 'user') { messages.unshift({ role: 'user', content: '(continue)' }); } /** @type {{ system?: string, messages: object[] }} */ const out = { messages }; if (systemParts.length) out.system = systemParts.join('\n\n'); return out; } /** * Build a full Anthropic Messages request body from LanguageModel doGenerate options. * @param {{ * modelId: string, * prompt: any, * tools?: any[], * toolChoice?: any, * maxOutputTokens?: number, * temperature?: number, * topP?: number, * stopSequences?: string[] * }} opts */ export function buildAnthropicMessagesRequest(opts) { const { system, messages } = convertPromptToAnthropicMessages(opts.prompt); const tools = convertToolsToAnthropic(opts.tools); const maxTokens = typeof opts.maxOutputTokens === 'number' && opts.maxOutputTokens > 0 ? Math.round(opts.maxOutputTokens) : ANTHROPIC_DEFAULT_MAX_TOKENS; /** @type {Record<string, unknown>} */ const body = { model: opts.modelId, max_tokens: maxTokens, messages }; if (system) body.system = system; if (tools) { body.tools = tools; const choice = convertToolChoiceToAnthropic(opts.toolChoice, tools); if (choice) body.tool_choice = choice; } if (typeof opts.temperature === 'number') body.temperature = opts.temperature; if (typeof opts.topP === 'number') body.top_p = opts.topP; if (Array.isArray(opts.stopSequences) && opts.stopSequences.length) { body.stop_sequences = opts.stopSequences; } return body; } /** * Map Anthropic stop_reason → AI SDK finishReason. * @param {string|null|undefined} reason */ export function mapAnthropicStopReason(reason) { const r = String(reason || ''); if (r === 'tool_use') return { unified: 'tool-calls', raw: r }; if (r === 'max_tokens') return { unified: 'length', raw: r }; if (r === 'stop_sequence') return { unified: 'stop', raw: r }; if (r === 'end_turn' || r === 'stop') return { unified: 'stop', raw: r || 'end_turn' }; return { unified: r ? 'other' : 'stop', raw: r || undefined }; } /** * Convert Anthropic message response → LanguageModelV2 doGenerate content. * @param {any} body */ export function convertAnthropicResponseToDoGenerate(body) { /** @type {any[]} */ const content = []; const blocks = Array.isArray(body?.content) ? body.content : []; for (const block of blocks) { if (!block || typeof block !== 'object') continue; if (block.type === 'text' && block.text) { content.push({ type: 'text', text: String(block.text) }); } else if (block.type === 'tool_use') { content.push({ type: 'tool-call', toolCallId: String(block.id || ''), toolName: String(block.name || ''), input: JSON.stringify(block.input ?? {}) }); } else if (block.type === 'thinking' && block.thinking) { content.push({ type: 'reasoning', text: String(block.thinking) }); } } if (!content.length) content.push({ type: 'text', text: '' }); const usageIn = Number(body?.usage?.input_tokens); const usageOut = Number(body?.usage?.output_tokens); const inputTokens = Number.isFinite(usageIn) ? usageIn : 0; const outputTokens = Number.isFinite(usageOut) ? usageOut : 0; return { content, finishReason: mapAnthropicStopReason(body?.stop_reason), usage: { inputTokens, outputTokens, totalTokens: inputTokens + outputTokens }, warnings: [] }; } function synthesizeStreamFromGenerate(generated) { const content = Array.isArray(generated.content) ? generated.content : []; const warnings = generated.warnings || []; return { stream: new ReadableStream({ start(controller) { controller.enqueue({ type: 'stream-start', warnings }); let textId = 0; for (const part of content) { if (part?.type === 'text') { const id = String(textId++); controller.enqueue({ type: 'text-start', id }); if (part.text) { controller.enqueue({ type: 'text-delta', id, delta: String(part.text) }); } controller.enqueue({ type: 'text-end', id }); } else if (part?.type === 'tool-call') { controller.enqueue({ type: 'tool-call', toolCallId: part.toolCallId, toolName: part.toolName, input: part.input }); } else if (part?.type === 'reasoning' && part.text) { const id = `reasoning-${textId++}`; controller.enqueue({ type: 'reasoning-start', id }); controller.enqueue({ type: 'reasoning-delta', id, delta: String(part.text) }); controller.enqueue({ type: 'reasoning-end', id }); } } controller.enqueue({ type: 'finish', finishReason: generated.finishReason || { unified: 'stop' }, usage: generated.usage || { inputTokens: 0, outputTokens: 0, totalTokens: 0 } }); controller.close(); } }) }; } /** * Create an Anthropic Messages LanguageModel for ToolLoopAgent / generateText. * @param {{ * modelId: string, * apiKey: string, * baseURL: string, * fetch?: typeof fetch, * providerName?: string * }} config */ export function createAnthropicLanguageModel(config) { const modelId = String(config.modelId || '').trim(); const apiKey = String(config.apiKey || ''); const baseURL = String(config.baseURL || '').replace(/\/$/, ''); const fetchImpl = typeof config.fetch === 'function' ? config.fetch : fetch; const providerName = (config.providerName && String(config.providerName).trim()) || 'anthropic'; const url = anthropicMessagesUrl(baseURL); return { specificationVersion: 'v2', provider: providerName, modelId, protocol: CHAT_PROTOCOL_ANTHROPIC, supportedUrls: Promise.resolve({}), async doGenerate(options) { if (options?.abortSignal?.aborted) { throw toAbortError(options.abortSignal.reason); } const body = buildAnthropicMessagesRequest({ modelId, prompt: options.prompt, tools: options.tools, toolChoice: options.toolChoice, maxOutputTokens: options.maxOutputTokens, temperature: options.temperature, topP: options.topP, stopSequences: options.stopSequences }); let response; try { respon - src/agent/artifacts.js (23183 bytes)
/** * BrowserBot — run-scoped in-memory artifacts (meta materialize API). * * Atomic helpers only: text, table (CSV), zip pack. * No scenario-named tools (report/taobao/etc). Download wiring is C2. */ import { rowsToCsv, csvEscape } from './dataTools/structuredData.js'; /** * @typedef {'text'|'table'|'zip'|'binary'} ArtifactKind * * @typedef {Object} ArtifactRecord * @property {string} artifactId * @property {ArtifactKind} kind * @property {string} name * @property {string} mime * @property {string|null} content UTF-8 text when applicable; null for pure binary * @property {Uint8Array} bytes * @property {number} size * @property {number} createdAt * @property {object} [meta] * * @typedef {Object} ArtifactRef * @property {string} artifactId * @property {string} name * @property {string} mime * @property {ArtifactKind} kind * @property {number} size */ /** @type {Map<string, Map<string, ArtifactRecord>>} */ const runStores = new Map(); /** * @param {string} [runId] * @returns {string} */ function resolveRunId(runId) { if (runId && typeof runId === 'string') return runId; return `run_${Date.now().toString(36)}_${Math.random().toString(36).slice(2, 8)}`; } /** * @param {string} [name] * @param {string} fallback * @returns {string} */ function sanitizeName(name, fallback) { const raw = (name == null || name === '' ? fallback : String(name)).trim() || fallback; // Strip path separators / control chars; keep unicode letters & common name chars return raw.replace(/[\\/:*?"<>|\x00-\x1f]/g, '_').slice(0, 200) || fallback; } /** * @param {string} [runId] * @returns {string} */ function nextArtifactId(runId) { const r = (runId || 'local').replace(/[^a-zA-Z0-9_-]/g, '').slice(0, 24) || 'local'; return `art_${r}_${Date.now().toString(36)}_${Math.random().toString(36).slice(2, 8)}`; } /** * Get or create the artifact Map for a run. * @param {string} runId * @returns {Map<string, ArtifactRecord>} */ export function getRunArtifactMap(runId) { const id = resolveRunId(runId); let map = runStores.get(id); if (!map) { map = new Map(); runStores.set(id, map); } return map; } /** * Drop all artifacts for a run (or entire registry when runId omitted and wipeAll). * @param {string} [runId] * @param {{ wipeAll?: boolean }} [opts] */ export function clearRunArtifacts(runId, opts = {}) { if (opts.wipeAll) { runStores.clear(); return; } if (runId) runStores.delete(runId); } /** * @param {ArtifactRecord} rec * @returns {ArtifactRef} */ function toRef(rec) { return { artifactId: rec.artifactId, name: rec.name, mime: rec.mime, kind: rec.kind, size: rec.size }; } /** * Store a record in the run map. * @param {string} runId * @param {ArtifactRecord} rec * @returns {ArtifactRef} */ function putArtifact(runId, rec) { const map = getRunArtifactMap(runId); map.set(rec.artifactId, rec); return toRef(rec); } /** * @param {string} artifactId * @param {string} [runId] When omitted, search all run maps * @returns {ArtifactRecord|null} */ export function getArtifact(artifactId, runId) { if (!artifactId) return null; if (runId) { const map = runStores.get(runId); return map?.get(artifactId) || null; } for (const map of runStores.values()) { if (map.has(artifactId)) return map.get(artifactId) || null; } return null; } /** * @param {string} runId * @returns {ArtifactRef[]} */ export function listArtifacts(runId) { const map = runStores.get(runId); if (!map) return []; return [...map.values()].map(toRef); } /** * True when a table cell is null/undefined/blank string. * @param {any} v * @returns {boolean} */ function isEmptyCell(v) { if (v == null) return true; if (typeof v === 'string' && v.trim() === '') return true; return false; } /** * Validate table-shaped data (pure; no chrome). * @param {{ * columns?: string[], * rows?: Array<Record<string, any>|any[]>, * minRows?: number, * requiredColumns?: string[] * }} opts * @returns {{ * ok: boolean, * rowCount: number, * columnCount: number, * emptyRequired: string[], * issues: string[], * nonEmptyRate: number * }} */ export function validateTableArtifact(opts = {}) { const { columns, objects } = normalizeTable(opts.columns, opts.rows || []); const rowCount = objects.length; const columnCount = columns.length; /** @type {string[]} */ const issues = []; /** @type {string[]} */ const emptyRequired = []; // Default minRows=1: empty tables cannot pass validation (finish gate false-success fix) const minRows = opts.minRows != null && Number.isFinite(Number(opts.minRows)) ? Number(opts.minRows) : 1; if (rowCount < minRows) { issues.push( rowCount === 0 ? 'empty table (0 rows)' : `rowCount ${rowCount} < minRows ${minRows}` ); } const required = Array.isArray(opts.requiredColumns) ? opts.requiredColumns.map((c) => String(c)) : []; for (const col of required) { if (!columns.includes(col)) { emptyRequired.push(col); issues.push(`required column missing: ${col}`); continue; } let hasEmpty = rowCount === 0; if (!hasEmpty) { for (const row of objects) { if (isEmptyCell(row[col])) { hasEmpty = true; break; } } } if (hasEmpty) { if (!emptyRequired.includes(col)) emptyRequired.push(col); issues.push(`required column has empty values: ${col}`); } } let total = 0; let nonEmpty = 0; if (columnCount > 0 && rowCount > 0) { for (const row of objects) { for (let i = 0; i < columns.length; i++) { total++; if (!isEmptyCell(row[columns[i]])) nonEmpty++; } } } const nonEmptyRate = total === 0 ? 0 : nonEmpty / total; return { ok: issues.length === 0, rowCount, columnCount, emptyRequired, issues, nonEmptyRate }; } /** * Validate text artifact payload (pure; no chrome). * @param {{ content?: string, minChars?: number }} opts * @returns {{ ok: boolean, charCount: number, issues: string[] }} */ export function validateTextArtifact(opts = {}) { const content = opts.content == null ? '' : String(opts.content); const charCount = content.length; /** @type {string[]} */ const issues = []; // Default minChars=1: empty text cannot pass validation (finish gate false-success fix) const minChars = opts.minChars != null && Number.isFinite(Number(opts.minChars)) ? Number(opts.minChars) : 1; if (charCount < minChars) { issues.push( charCount === 0 ? 'empty text content' : `charCount ${charCount} < minChars ${minChars}` ); } return { ok: issues.length === 0, charCount, issues }; } /** * Short human preview of an artifact record. Never dumps multi-MB payloads. * @param {ArtifactRecord|null|undefined} rec * @param {{ maxChars?: number, maxRows?: number }} [opts] * @returns {string} */ export function previewArtifactContent(rec, opts = {}) { const maxCharsRaw = opts.maxChars != null ? Number(opts.maxChars) : 800; const maxRowsRaw = opts.maxRows != null ? Number(opts.maxRows) : 5; const maxChars = Number.isFinite(maxCharsRaw) ? Math.max(0, Math.floor(maxCharsRaw)) : 800; const maxRows = Number.isFinite(maxRowsRaw) ? Math.max(0, Math.floor(maxRowsRaw)) : 5; if (!rec || typeof rec !== 'object') return ''; /** * @param {string} s * @returns {string} */ const cap = (s) => { const str = s == null ? '' : String(s); if (str.length <= maxChars) return str; return maxChars === 0 ? '' : str.slice(0, maxChars) + '…'; }; if (rec.kind === 'text') { return cap(rec.content == null ? '' : String(rec.content)); } if (rec.kind === 'table') { const columns = Array.isArray(rec.meta?.columns) ? rec.meta.columns.map(String) : []; const header = columns.length ? columns.join('\t') : '(no columns)'; const content = rec.content == null ? '' : String(rec.content); // Scan only a bounded prefix so huge CSVs do not force full-line splits. const bomOffset = content.charCodeAt(0) === 0xfeff ? 1 : 0; const scanLimit = Math.min(content.length - bomOffset, Math.max(maxChars * 8, 4096)); const slice = content.slice(bomOffset, bomOffset + Math.max(0, scanLimit)); const rawLines = slice.split(/\r?\n/).filter((l) => l.length > 0); let dataStart = 0; if (columns.length && rawLines.length) { const first = rawLines[0]; // Header row when first cell matches first column name (CSV-escaped or plain). if (first === columns.join(',') || first.startsWith(columns[0] + ',') || first === columns[0]) { dataStart = 1; } } const picked = rawLines.slice(dataStart, dataStart + maxRows); let out = header; for (const line of picked) { out += '\n' + line; if (out.length >= maxChars) break; } const totalRows = typeof rec.meta?.rowCount === 'number' ? rec.meta.rowCount : null; if (totalRows != null && totalRows > maxRows && out.length < maxChars) { const more = `\n… +${totalRows - maxRows} more rows`; if (out.length + more.length <= maxChars) out += more; else out = cap(out + more); } return cap(out); } if (rec.kind === 'zip') { const n = rec.meta?.fileCount != null ? rec.meta.fileCount : '?'; const names = Array.isArray(rec.meta?.fileNames) ? rec.meta.fileNames.slice(0, maxRows).map(String).join(', ') : ''; const base = `[zip] ${rec.name || 'pack.zip'} (${n} files)`; return cap(names ? `${base}: ${names}` : base); } if (rec.content != null) { return cap(String(rec.content)); } return cap(`[${rec.kind || 'artifact'}] ${rec.name || ''} size=${rec.size ?? 0}`); } /** * Attach a validation result onto rec.meta.validation (mutates rec). * @param {ArtifactRecord} rec * @param {object} validation * @returns {ArtifactRecord} */ export function attachValidation(rec, validation) { if (!rec || typeof rec !== 'object') return rec; if (!rec.meta || typeof rec.meta !== 'object') { rec.meta = {}; } rec.meta.validation = validation; return rec; } /** * Create a plain text (or other text/*) artifact. * @param {{ name: string, mime?: string, content?: string, runId?: string }} opts * @returns {ArtifactRef} */ export function createTextArtifact(opts = {}) { const runId = resolveRunId(opts.runId); const name = sanitizeName(opts.name, 'artifact.txt'); const mime = (opts.mime && String(opts.mime)) || 'text/plain'; const content = opts.content == null ? '' : String(opts.content); const bytes = new TextEncoder().encode(content); const artifactId = nextArtifactId(runId); /** @type {ArtifactRecord} */ const rec = { artifactId, kind: 'text', name, mime, content, bytes, size: bytes.length, createdAt: Date.now(), meta: { runId } }; attachValidation(rec, validateTextArtifact({ content })); return putArtifact(runId, rec); } /** * Store a binary artifact (PDF, etc.). * @param {{ runId?: string, name?: string, mime?: string, bytes: Uint8Array }} opts * @returns {ArtifactRef} */ export function createBinaryArtifact(opts = {}) { const runId = resolveRunId(opts.runId); const name = sanitizeName(opts.name, 'artifact.bin'); const mime = (opts.mime && String(opts.mime)) || 'application/octet-stream'; const bytes = opts.bytes instanceof Uint8Array ? opts.bytes : new Uint8Array(0); const artifactId = nextArtifactId(runId); /** @type {ArtifactRecord} */ const rec = { artifactId, kind: 'binary', name, mime, content: null, bytes, size: bytes.length, createdAt: Date.now(), meta: { runId } }; const ok = bytes.length > 0; attachValidation(rec, { ok, issues: ok ? [] : ['empty binary'], byteCount: bytes.length }); return putArtifact(runId, rec); } /** * Normalize table rows to objects keyed by columns. * Accepts rows as objects or arrays; columns optional for objects. * @param {string[]|undefined} columns * @param {Array<Record<string, any>|any[]>} rows * @returns {{ columns: string[], objects: Array<Record<string, any>> }} */ function normalizeTable(columns, rows) { const list = Array.isArray(rows) ? rows : []; if (list.length === 0) { return { columns: Array.isArray(columns) ? columns.map(String) : [], objects: [] }; } const first = list[0]; if (Array.isArray(first)) { const cols = Array.isArray(columns) && columns.length ? columns.map(String) : first.map((_, i) => `col_${i + 1}`); const objects = list.map((row) => { const arr = Array.isArray(row) ? row : []; /** @type {Record<string, any>} */ const obj = {}; for (let i = 0; i < cols.length; i++) obj[cols[i]] = arr[i] ?? ''; return obj; }); return { columns: cols, objects }; } // Object rows const objects = list.map((row) => row && typeof row === 'object' && !Array.isArray(row) ? /** @type {Record<string, any>} */ (row) : {} ); let cols = Array.isArray(columns) && columns.length ? columns.map(String) : null; if (!cols) { const seen = new Set(); cols = []; for (const row of objects) { for (const k of Object.keys(row)) { if (!seen.has(k)) { seen.add(k); cols.push(k); } } } } return { columns: cols, objects }; } /** * Build CSV string (Excel-friendly BOM). Empty table still emits header if columns given. * @param {string[]} columns * @param {Array<Record<string, any>>} objects * @returns {string} */ function tableToCsv(columns, objects) { if (objects.length === 0) { if (!columns.length) return '\uFEFF'; return `\uFEFF${columns.map(csvEscape).join(',')}`; } return rowsToCsv(objects, { fieldNames: columns, bom: true }); } /** * Create a table artifact as CSV (xlsx-compatible via Excel open). * @param {{ name?: string, columns?: string[], rows?: Array<Record<string, any>|any[]>, runId?: string }} opts * @returns {ArtifactRef} */ export function createTableArtifact(opts = {}) { const runId = resolveRunId(opts.runId); let name = sanitizeName(opts.name, 'table.csv'); if (!/\.csv$/i.test(name)) name = `${name}.csv`; const { columns, objects } = normalizeTable(opts.columns, opts.rows || []); const content = tableToCsv(columns, objects); const bytes = new TextEncoder().encode(content); const artifactId = nextArtifactId(runId); /** @type {ArtifactRecord} */ const rec = { artifactId, kind: 'table', name, mime: 'text/csv', content, bytes, size: bytes.length, createdAt: Date.now(), meta: { runId, columns, rowCount: objects.length } }; attachValidation(rec, validateTableArtifact({ columns, rows: objects })); return putArtifact(runId, rec); } // ── Minimal ZIP (store method, no compression) ───────────────────────── - src/agent/byokGate.js (11575 bytes)
/** * BYOK needs a paid Browser Kitten plan. * * Using your own API keys (any provider the user configured: not a fixed-id `bk_cloud_*` * record, and not a Tavily / Firecrawl slot whose base URL is browserkitten.com) is * allowed only while the user is signed in to Browser Kitten on the developer, * standard, pro or max plan. Free / trial accounts use credits only. BYOK calls never * reach our server and never cost credits; this is a client-side gate, checked at the * places the extension calls a user-configured model / search / fetch service. * * chrome.storage.local browserbot_cloud_entitlement = { plan, checkedAt } * written whenever /api/me is read (sign-in, Settings, the refresh below); * cleared on sign-out and on a 401 from /api/me. * * Freshness: a paid plan checked within 72 h allows BYOK. Older than 6 h (or not * allowed at all) → refresh /api/me first; a network failure falls back to the 72 h rule. * * Unpacked builds are exempt: the Chrome Web Store adds `update_url` to the manifest, * so a manifest without it is a development (unpacked) load. Store builds are gated. * No manifest at all (Node tests) also counts as unpacked. To try the gate on an * unpacked load, set chrome.storage.local `browserbot_byok_enforce_unpacked = true`. * * Leaf module: imports only cloudModels.js, so llm.js / cloudAccount.js / SW / offscreen * can all use it without cycles. Storage goes direct when chrome.storage exists * (SW, side panel) and through the SW bridge otherwise (offscreen). */ import { BK_CLIENT_HEADER, CLOUD_ORIGIN, bkClientHeaderValue, cloudOriginOf, isCloudProviderId } from './cloudModels.js'; export const BYOK_ENTITLEMENT_KEY = 'browserbot_cloud_entitlement'; export const BYOK_ENFORCE_UNPACKED_KEY = 'browserbot_byok_enforce_unpacked'; /** Same key cloudAccount.js writes the device token to. */ export const BYOK_ACCOUNT_KEY = 'browserbot_cloud_account'; export const BYOK_REQUIRES_PLAN = 'BYOK_REQUIRES_PLAN'; export const BYOK_PLANS = Object.freeze(['developer', 'standard', 'pro', 'max']); export const BYOK_REFRESH_AFTER_MS = 6 * 60 * 60 * 1000; export const BYOK_GRACE_MS = 72 * 60 * 60 * 1000; /** A cache timestamp this far in the future is treated as bogus. */ const FUTURE_SKEW_MS = 5 * 60 * 1000; /** A refused call retries /api/me at most this often (an upgrade shows up without waiting 6 h). */ const DENIED_REFRESH_MIN_MS = 30 * 1000; export const BYOK_PLAN_MESSAGE = 'Using your own API keys needs a Browser Kitten plan (Developer or above). Sign in or upgrade, or switch this service to Browser Kitten credits.'; /** The cache entry to store after reading /api/me. */ export function entitlementFromMe(me, now = Date.now()) { const plan = String(me?.plan || 'free').trim().toLowerCase() || 'free'; return { plan, checkedAt: now }; } /** * Pure decision. * @param {{ plan?: string, checkedAt?: number } | null | undefined} entitlement * @param {{ now?: number, unpacked?: boolean }} [opts] * @returns {{ allowed: boolean, refresh: boolean, reason: 'unpacked'|'paid'|'signed-out'|'plan'|'expired' }} * `refresh`: the caller should read /api/me before deciding (non-fatal if that fails). */ export function byokAllowed(entitlement, { now = Date.now(), unpacked = false } = {}) { if (unpacked) return { allowed: true, refresh: false, reason: 'unpacked' }; const plan = String(entitlement?.plan || '').toLowerCase(); const checkedAt = Number(entitlement?.checkedAt); if (!plan || !Number.isFinite(checkedAt) || checkedAt <= 0) { return { allowed: false, refresh: true, reason: 'signed-out' }; } const age = now - checkedAt; const sane = age >= -FUTURE_SKEW_MS; const paid = BYOK_PLANS.includes(plan); const withinGrace = sane && age <= BYOK_GRACE_MS; const stale = !sane || age > BYOK_REFRESH_AFTER_MS; if (!paid) return { allowed: false, refresh: true, reason: 'plan' }; if (!withinGrace) return { allowed: false, refresh: true, reason: 'expired' }; return { allowed: true, refresh: stale, reason: 'paid' }; } /** Web Store builds carry `update_url`; unpacked loads (and Node, with no manifest) do not. */ export function isUnpackedManifest(manifest) { return !manifest || typeof manifest !== 'object' || !manifest.update_url; } /** A provider record / settings providerId the user configured themselves (legacy flat keys have none). */ export function isByokProviderId(id) { return !isCloudProviderId(id); } /** A web slot base URL that is ours (credits), not the user's own Tavily / Firecrawl account. */ export function isCloudWebBase(baseURL) { try { return !!cloudOriginOf(new URL(String(baseURL || '')).origin); } catch { return false; } } export function byokPlanError(purpose = '') { return Object.assign(new Error(BYOK_PLAN_MESSAGE), { code: BYOK_REQUIRES_PLAN, purpose: String(purpose || '') }); } /** The failed-tool shape the web / image / stt / System1 paths return to the model. */ export function byokPlanFailure(purpose = '', extra = {}) { return { ok: false, code: BYOK_REQUIRES_PLAN, error: BYOK_PLAN_MESSAGE, purpose: String(purpose || ''), ...extra }; } /** * Web settings for a turn: BYOK slots are blanked when not allowed; `byokBlocked` lists * which, so search / map / crawl can say why instead of "not configured". The saved * settings are never changed. Pure. */ export function webAcquireForPlan(settings, allowed) { const s = settings && typeof settings === 'object' ? { ...settings } : {}; if (allowed) return s; const blocked = []; if (s.tavilyKey && !isCloudWebBase(s.tavilyBaseURL)) { s.tavilyKey = ''; blocked.push('tavily'); } if (s.firecrawlKey && !isCloudWebBase(s.firecrawlBaseURL)) { s.firecrawlKey = ''; blocked.push('firecrawl'); } if (s.braveKey) { s.braveKey = ''; blocked.push('brave'); } if (blocked.length) s.byokBlocked = blocked; return s; } export function webAcquireHasByok(settings) { const s = settings || {}; return !!( (s.tavilyKey && !isCloudWebBase(s.tavilyBaseURL)) || (s.firecrawlKey && !isCloudWebBase(s.firecrawlBaseURL)) || s.braveKey ); } // ---------- chrome-facing ---------- async function localGet(keys) { const c = globalThis.chrome; if (c?.storage?.local?.get) return (await c.storage.local.get(keys)) || {}; if (c?.runtime?.sendMessage) { const res = await c.runtime.sendMessage({ target: 'browserbot-background', action: 'storage_local_get', keys }); if (!res?.ok) throw new Error(res?.error || 'storage_local_get failed'); return res.result || {}; } return {}; } async function localSet(values) { const c = globalThis.chrome; if (c?.storage?.local?.set) return c.storage.local.set(values); if (c?.runtime?.sendMessage) { const res = await c.runtime.sendMessage({ target: 'browserbot-background', action: 'storage_local_set', values }); if (!res?.ok) throw new Error(res?.error || 'storage_local_set failed'); } } function currentManifest() { try { return globalThis.chrome?.runtime?.getManifest?.() || null; } catch { return null; } } /** Best-effort: a storage failure must not fail sign-in or Settings. */ export async function writeByokEntitlement(me, now = Date.now()) { try { await localSet({ [BYOK_ENTITLEMENT_KEY]: entitlementFromMe(me, now) }); } catch { /* next gated call refreshes */ } } export async function clearByokEntitlement() { try { await localSet({ [BYOK_ENTITLEMENT_KEY]: null }); } catch { /* ignore */ } } let refreshing = null; let lastRefreshAt = 0; /** Tests only. */ export function resetByokGateState() { refreshing = null; lastRefreshAt = 0; } async function refreshEntitlement(token, { fetchImpl, now, origin = CLOUD_ORIGIN }) { const client = bkClientHeaderValue(); const res = await fetchImpl(`${origin}/api/me`, { headers: { ...(client ? { [BK_CLIENT_HEADER]: client } : {}), authorization: `Bearer ${token}` } }); if (res.status === 401) { await clearByokEntitlement(); return null; } if (!res.ok) throw Object.assign(new Error(`HTTP ${res.status}`), { status: res.status }); const me = await res.json(); const ent = entitlementFromMe(me, now); await localSet({ [BYOK_ENTITLEMENT_KEY]: ent }).catch(() => {}); return ent; } /** * Current verdict, refreshing /api/me when the cache says so. * @param {{ now?: number, fetchImpl?: typeof fetch, refresh?: boolean }} [opts] * @returns {Promise<{ allowed: boolean, reason: string, plan: string, signedIn: boolean }>} */ export async function byokStatus({ now = Date.now(), fetchImpl = globalThis.fetch, refresh = true } = {}) { const manifest = currentManifest(); let bag = {}; try { bag = await localGet([BYOK_ENTITLEMENT_KEY, BYOK_ACCOUNT_KEY, BYOK_ENFORCE_UNPACKED_KEY]); } catch { bag = {}; } const unpacked = isUnpackedManifest(manifest) && bag[BYOK_ENFORCE_UNPACKED_KEY] !== true; const token = typeof bag[BYOK_ACCOUNT_KEY]?.token === 'string' ? bag[BYOK_ACCOUNT_KEY].token : ''; if (unpacked) return { allowed: true, reason: 'unpacked', plan: String(bag[BYOK_ENTITLEMENT_KEY]?.plan || ''), signedIn: !!token }; // Signed out means no entitlement, whatever the cache still says. if (!token) return { allowed: false, reason: 'signed-out', plan: '', signedIn: false }; let ent = bag[BYOK_ENTITLEMENT_KEY] || null; let verdict = byokAllowed(ent, { now }); if (verdict.refresh && refresh && typeof fetchImpl === 'function') { // Stale or missing cache: always ask. A fresh "free" answer: at most every 30 s, // so an upgrade is picked up quickly without a request per tool call. const stale = verdict.allowed || !ent || !(now - Number(ent.checkedAt) <= BYOK_REFRESH_AFTER_MS); if (refreshing || stale || now - lastRefreshAt >= DENIED_REFRESH_MIN_MS) { if (!refreshing) { lastRefreshAt = now; const origin = cloudOriginOf(bag[BYOK_ACCOUNT_KEY]?.origin) || CLOUD_ORIGIN; refreshing = refreshEntitlement(token, { fetchImpl, now, origin }).finally(() => { refreshing = null; }); } try { ent = await refreshing; verdict = byokAllowed(ent, { now }); } catch { /* offline / server error: the cached entitlement and its 72 h grace decide */ } } } return { allowed: verdict.allowed, reason: ent ? verdict.reason : 'signed-out', plan: String(ent?.plan || ''), signedIn: !!ent }; } /** Throws BYOK_REQUIRES_PLAN unless BYOK is allowed now. */ export async function assertByokAllowed(purpose, opts) { const status = await byokStatus(opts); if (!status.allowed) throw byokPlanError(purpose); return status; } /** Cloud records pass; anything the user configured goes through the gate. */ export async function assertByokForProvider(purpose, providerId, opts) { if (!isByokProviderId(providerId)) return null; return assertByokAllowed(purpose, opts); } /** Boolean form for background probes: skip silently when not allowed. */ export async function byokAllowedForProvider(purpose, providerId, opts) { if (!isByokProviderId(providerId)) return true; try { return (await byokStatus(opts)).allowed; } catch { return false; } } /** Web settings for a turn (see webAcquireForPlan). Reads nothing when no BYOK slot is set. */ export async function gateWebAcquire(settings, opts) { if (!webAcquireHasByok(settings)) return settings; let allowed = false; try { allowed = (await byokStatus(opts)).allowed; } catch { allowed = false; } return webAcquireForPlan(settings, allowed); } - src/agent/chatMedia.js (5576 bytes)
/** Input declarations and cached checks are evidence. Unknown audio/video fail closed. * A model declaration and a supported wire transport are both required. */ import { normalizeModelsBaseURL, classifyChatVisionCapability } from './modelCatalog.js'; import { mergeMediaEvidence } from './mediaCapabilityStore.js'; export const MAX_CHAT_MEDIA_BYTES = 16 * 1024 * 1024; export const INPUT_MODALITIES = ['text', 'image', 'audio', 'video']; export const MEDIA_TRANSPORTS = ['auto', 'openai-audio', 'openrouter']; export function mediaModelKey(base, model) { return `${normalizeModelsBaseURL(base)}\n${String(model || '').trim()}`; } export function normalizeMediaOverrides(raw) { const out = {}; if (!raw || typeof raw !== 'object') return out; for (const [key, value] of Object.entries(raw)) { if (!key.includes('\n') || !value || typeof value !== 'object') continue; const entry = {}; for (const kind of INPUT_MODALITIES) { if (typeof value[kind] === 'boolean') entry[kind] = value[kind]; } if (MEDIA_TRANSPORTS.includes(value.transport)) entry.transport = value.transport; if (Object.keys(entry).length) out[key] = entry; } return out; } // Deliberately small, versioned defaults. Catalog/user declarations supersede these. const GEMINI_AV = /^(?:google\/)?gemini-(?:2\.0-flash(?:-001)?|2\.5-(?:flash|pro)(?:-preview-\d\d-\d\d)?)$/; const OPENAI_AUDIO = /^(?:openai\/)?(?:gpt-4o(?:-mini)?-audio-preview(?:-\d{4}-\d{2}-\d{2})?|gpt-audio(?:-mini)?(?:-\d{4}-\d{2}-\d{2})?)$/; export function resolveChatMediaCapabilities(settings = {}, models = [], evidence = null, now = Date.now()) { const model = String(settings.model || '').trim(); const base = normalizeModelsBaseURL(settings.apiBase || ''); // No suffix matching: vendor-a/foo must not inherit vendor-b/foo's capabilities. const entry = models.find((m) => m?.id === model); const override = settings.chatMediaOverrides?.[mediaModelKey(base, model)] || {}; const declared = Array.isArray(entry?.inputModalities) ? entry.inputModalities : null; let hostname = ''; try { hostname = new URL(base).hostname; } catch { /* unknown endpoint */ } const official = ['api.openai.com', 'api.deepseek.com', 'generativelanguage.googleapis.com', 'openrouter.ai'].includes(hostname); const knownAV = ['generativelanguage.googleapis.com', 'openrouter.ai'].includes(hostname) && GEMINI_AV.test(model); const vision = official ? classifyChatVisionCapability(model) : 'unknown'; const inputs = {}; for (const kind of INPUT_MODALITIES) { let state = 'unknown', source = 'unknown'; if (kind === 'text') { state = 'supported'; source = 'chat'; } if (kind === 'image' && vision !== 'unknown') { state = vision === 'vision' ? 'supported' : 'unsupported'; source = 'builtin'; } if (kind === 'audio' || kind === 'video') { if (knownAV || (kind === 'audio' && ['api.openai.com', 'openrouter.ai'].includes(hostname) && OPENAI_AUDIO.test(model))) { state = 'supported'; source = 'builtin'; } else if ( vision === 'text-only' || hostname === 'api.deepseek.com' || (official && /^(?:openai\/)?gpt-4o(?:-mini)?$/.test(model)) ) { // Official DeepSeek documents image input on Flash; it has no audio/video SKU. state = 'unsupported'; source = 'builtin'; } } if (declared) { state = declared.includes(kind) ? 'supported' : 'unsupported'; source = 'catalog'; } if (typeof override[kind] === 'boolean') { state = override[kind] ? 'supported' : 'unsupported'; source = 'user'; } inputs[kind] = { state, source }; } let transport = MEDIA_TRANSPORTS.includes(override.transport) ? override.transport : 'auto'; if (transport === 'auto') { transport = 'none'; try { const u = new URL(base); if (u.hostname === 'openrouter.ai' && u.pathname.replace(/\/$/, '') === '/api/v1') transport = 'openrouter'; if ((u.hostname === 'api.openai.com' && u.pathname === '/v1') || (u.hostname === 'generativelanguage.googleapis.com' && /^\/v1(?:beta)?\/openai$/.test(u.pathname))) transport = 'openai-audio'; } catch { /* custom gateways require an explicit transport declaration */ } } const caps = mergeMediaEvidence({ model, base, inputs, transport, maxBytes: MAX_CHAT_MEDIA_BYTES }, evidence, now); caps.routes = { nativeAudio: canSendChatMedia(caps, 'audio/wav'), nativeVideo: canSendChatMedia(caps, 'video/webm'), videoAudio: caps.videoAudio, file: 'acquire action=fetch url or path', liveVideo: 'action op=record seconds=8 (tab picture + audio; max 30s; video grant)', liveAudio: 'action op=listen (audio only)', still: 'action op=snapshot', sttConfigured: Boolean(settings.stt?.apiKey) }; return caps; } export function canSendChatMedia(caps, mimeType) { const mime = String(mimeType || '').split(';')[0].trim().toLowerCase(); const kind = mime.split('/')[0]; if (caps?.inputs?.[kind]?.rejectedFormats?.includes(mime)) return false; if (caps?.inputs?.[kind]?.source === 'verified' && !caps.inputs[kind].formats?.includes(mime)) return false; if (kind === 'image') return caps?.inputs?.image?.state !== 'unsupported'; if (caps?.inputs?.[kind]?.state !== 'supported') return false; if (kind === 'audio') return ['openai-audio', 'openrouter'].includes(caps.transport) && ['audio/wav', 'audio/mpeg'].includes(mime); return kind === 'video' && caps.transport === 'openrouter' && ['video/mp4', 'video/mpeg', 'video/mov', 'video/quicktime', 'video/webm'].includes(mime); } - src/agent/chatWireRoute.js (6407 bytes)
/** * Chat wire routing for arbitrary BYOK gateways. * * Stored provider.protocol (explicit always wins): * openai-compatible — POST {base}/chat/completions (default) * anthropic — POST {base}/messages * openai-responses — unsupported adapter (clear error; never remapped) * auto — optional model-id heuristic on ANY base (multiplex gateways) * * When protocol is omitted: openai-compatible, except host api.anthropic.com → anthropic. * No product special-case for opencode.ai / openrouter.ai. * Jev stays purpose:'jev' + /systemone; jev-* models are rejected on the chat path. */ import { CHAT_PROTOCOL_ANTHROPIC, CHAT_PROTOCOL_OPENAI, CHAT_PROTOCOL_RESPONSES, CHAT_PROTOCOL_AUTO, DEFAULT_CHAT_PROTOCOL, anthropicMessagesUrl, openaiChatCompletionsUrl, openaiResponsesUrl, systemOneUrl, normalizeChatProtocol } from './llm.js'; /** @typedef {'openai-compatible'|'anthropic'|'openai-responses'|'auto'|'systemone'|'google'} ChatWireKind */ /** * Strip provider prefixes: `opencode/claude-…`, `anthropic/claude-…`. * @param {string} modelId */ export function bareChatModelId(modelId) { const raw = String(modelId || '').trim(); if (!raw) return ''; const slash = raw.lastIndexOf('/'); return (slash >= 0 ? raw.slice(slash + 1) : raw).trim(); } /** * Only the well-known direct Anthropic host gets an omitted-protocol default. * @param {string} baseURL */ export function isAnthropicApiBase(baseURL) { try { const host = new URL(String(baseURL || '')).hostname.toLowerCase(); return host === 'api.anthropic.com'; } catch { return /(?:^|\/\/)api\.anthropic\.com(?:[:/]|$)/i.test(String(baseURL || '')); } } /** * Model-id heuristic used only when protocol === 'auto'. * @param {string} modelId * @returns {ChatWireKind} */ export function inferChatWireFromModelId(modelId) { const id = bareChatModelId(modelId).toLowerCase(); if (!id) return CHAT_PROTOCOL_OPENAI; if (/^jev([-_.]|$)/i.test(id)) return 'systemone'; if ( /^gpt-5/.test(id) || /^gpt-6/.test(id) || /^o[1-9]/.test(id) || /^grok-/.test(id) || /^muse-spark/.test(id) ) { return CHAT_PROTOCOL_RESPONSES; } if (/^gemini-/.test(id)) return 'google'; if (/^claude[-_]/.test(id) || id.startsWith('claude')) return CHAT_PROTOCOL_ANTHROPIC; if (/^qwen3\.(5|6|7|8)([-_.]|$)/.test(id)) return CHAT_PROTOCOL_ANTHROPIC; return CHAT_PROTOCOL_OPENAI; } /** * @param {ChatWireKind} wire * @param {string} baseURL * @param {string} source */ function supportedRoute(wire, baseURL, source) { if (wire === CHAT_PROTOCOL_ANTHROPIC) { return { wire, protocol: CHAT_PROTOCOL_ANTHROPIC, url: anthropicMessagesUrl(baseURL), supported: true, source }; } return { wire: CHAT_PROTOCOL_OPENAI, protocol: CHAT_PROTOCOL_OPENAI, url: openaiChatCompletionsUrl(baseURL), supported: true, source }; } /** * Resolve chat wire for a BYOK provider. * @param {{ * baseURL?: string, * model?: string, * protocol?: string|null, * purpose?: string * }} input */ export function resolveChatWire(input = {}) { const baseURL = String(input.baseURL || '').trim().replace(/\/$/, ''); const model = String(input.model || '').trim(); const purpose = input.purpose || 'chat'; if (purpose === 'jev') { return { wire: 'systemone', protocol: null, url: systemOneUrl(baseURL), supported: true, source: 'purpose' }; } const stored = normalizeChatProtocol(input.protocol); if (stored === CHAT_PROTOCOL_RESPONSES) { return unsupportedWire(CHAT_PROTOCOL_RESPONSES, baseURL, model, 'explicit'); } if (stored === CHAT_PROTOCOL_ANTHROPIC) { return supportedRoute(CHAT_PROTOCOL_ANTHROPIC, baseURL, 'explicit'); } if (stored === CHAT_PROTOCOL_OPENAI) { return supportedRoute(CHAT_PROTOCOL_OPENAI, baseURL, 'explicit'); } if (stored === CHAT_PROTOCOL_AUTO) { const wire = inferChatWireFromModelId(model); if (wire === CHAT_PROTOCOL_ANTHROPIC || wire === CHAT_PROTOCOL_OPENAI) { return supportedRoute(wire, baseURL, 'auto'); } return unsupportedWire(wire, baseURL, model, 'auto'); } // Omitted protocol: only api.anthropic.com defaults to Messages. if (isAnthropicApiBase(baseURL)) { return supportedRoute(CHAT_PROTOCOL_ANTHROPIC, baseURL, 'host'); } return supportedRoute(DEFAULT_CHAT_PROTOCOL, baseURL, 'default'); } /** * @param {ChatWireKind} wire * @param {string} baseURL * @param {string} model * @param {string} source */ function unsupportedWire(wire, baseURL, model, source) { const id = bareChatModelId(model) || model || '(no model)'; if (wire === 'systemone') { return { wire, protocol: null, url: systemOneUrl(baseURL), supported: false, source, code: 'CHAT_WIRE_JEV', error: `Model "${id}" uses System One (/systemone). Add it under System1, not chat inference.` }; } if (wire === CHAT_PROTOCOL_RESPONSES || wire === 'responses') { return { wire: CHAT_PROTOCOL_RESPONSES, protocol: CHAT_PROTOCOL_RESPONSES, url: openaiResponsesUrl(baseURL), supported: false, source, code: 'CHAT_WIRE_RESPONSES_UNSUPPORTED', error: `Protocol openai-responses (POST …/responses) is not implemented yet. Choose OpenAI Compatible or Anthropic Messages, or pick a model that those wires support.` }; } if (wire === 'google') { return { wire, protocol: null, url: '', supported: false, source, code: 'CHAT_WIRE_GOOGLE_UNSUPPORTED', error: `Model "${id}" requires Google Generative AI paths. Browser Kitten does not implement that wire yet.` }; } return { wire, protocol: null, url: '', supported: false, source, code: 'CHAT_WIRE_UNSUPPORTED', error: `Unsupported chat wire "${wire}" for model "${id}".` }; } /** * @param {string} baseURL * @param {{ model?: string, protocol?: string|null, purpose?: string }} [opts] */ export function buildChatEndpointUrl(baseURL, opts = {}) { return resolveChatWire({ baseURL, ...opts }).url; } /** * @param {{ baseURL?: string, model?: string, protocol?: string|null }} settings */ export function resolveChatProtocol(settings = {}) { return resolveChatWire(settings); } - src/agent/clipboardStore.js (3987 bytes)
/** * Session clipboard pin store — pure (no chrome). * Pins survive page selection deselect; clear only via removeIds/clear. */ /** * @typedef {{ * id: string, * text: string, * kind?: string, * sourceIndex?: number|null, * pinned: true, * createdAt: number * }} ClipboardPinItem */ let _seq = 0; /** * @returns {string} */ function nextId() { _seq += 1; return `clip_${Date.now().toString(36)}_${_seq.toString(36)}`; } /** * Create an isolated clipboard store (per run / test). * @returns {{ * pinItems: (items: Array<object|string>) => ClipboardPinItem[], * listItems: () => ClipboardPinItem[], * removeIds: (ids: string[]) => number, * clear: () => void, * size: () => number * }} */ export function createClipboardStore() { /** @type {ClipboardPinItem[]} */ const items = []; /** * @param {Array<object|string>} raw * @returns {ClipboardPinItem[]} */ function pinItems(raw) { const list = Array.isArray(raw) ? raw : []; /** @type {ClipboardPinItem[]} */ const pinned = []; for (const entry of list) { if (entry == null) continue; let text = ''; let kind; let sourceIndex = null; let id; if (typeof entry === 'string') { text = entry; } else if (typeof entry === 'object') { text = entry.text != null ? String(entry.text) : entry.content != null ? String(entry.content) : entry.body != null ? String(entry.body) : ''; if (entry.kind != null) kind = String(entry.kind); if (typeof entry.sourceIndex === 'number') sourceIndex = entry.sourceIndex; else if (typeof entry.index === 'number') sourceIndex = entry.index; if (entry.id != null && String(entry.id).trim()) id = String(entry.id).trim(); } if (!text) continue; /** @type {ClipboardPinItem} */ const item = { id: id || nextId(), text, pinned: true, createdAt: Date.now() }; if (kind) item.kind = kind; if (sourceIndex != null) item.sourceIndex = sourceIndex; items.push(item); pinned.push(item); } return pinned; } function listItems() { return items.map((it) => ({ ...it })); } /** * @param {string[]} ids * @returns {number} removed count */ function removeIds(ids) { const set = new Set((Array.isArray(ids) ? ids : []).map((x) => String(x))); if (!set.size) return 0; let removed = 0; for (let i = items.length - 1; i >= 0; i--) { if (set.has(items[i].id)) { items.splice(i, 1); removed++; } } return removed; } function clear() { items.length = 0; } function size() { return items.length; } return { pinItems, listItems, removeIds, clear, size }; } /** Module-level session store (shared across tools when ctx does not inject one). */ const sessionStore = createClipboardStore(); /** * @returns {ReturnType<typeof createClipboardStore>} */ export function getSessionClipboardStore() { return sessionStore; } /** * Pin into the session store. * @param {Array<object|string>} items */ export function pinItems(items) { return sessionStore.pinItems(items); } /** @returns {import('./clipboardStore.js').ClipboardPinItem[]} */ export function listItems() { return sessionStore.listItems(); } /** * @param {string[]} ids */ export function removeIds(ids) { return sessionStore.removeIds(ids); } export function clear() { sessionStore.clear(); } /** * Resolve store from ToolContext hooks or session default. * @param {{ clipboardStore?: ReturnType<typeof createClipboardStore>, pinClipboardItems?: Function, getClipboardItems?: Function }} [ctx] */ export function resolveClipboardStore(ctx) { if (ctx?.clipboardStore && typeof ctx.clipboardStore.pinItems === 'function') { return ctx.clipboardStore; } return sessionStore; } - src/agent/cloudAccount.js (18251 bytes)
/** * Browser Kitten account: credits instead of the user's own keys. * * Sign-in writes provider records with fixed ids (chat always; image / stt / System1 * only when /api/me says the server hosts them) that point at the Browser Kitten API * with the device token as apiKey, and fills * the Tavily / Firecrawl slots only when they are empty. Every model, image, STT, * System1, search and fetch call then works unchanged. Nothing the user configured * is overwritten, and an active BYOK provider stays active. * * Sign-out removes exactly what sign-in added. The server decides plan and credits * on every request; nothing stored here grants anything (see site server/router.js). * * chrome.storage.local browserbot_cloud_account = { token, deviceId, filled: { tavily, firecrawl }, connectedAt } * chrome.storage.local browserbot_cloud_entitlement = { plan, checkedAt } (byokGate.js: own keys need a paid plan) */ import { BK_CLIENT_HEADER, CLOUD_IDS, CLOUD_MODELS, CLOUD_ORIGIN, bkClientHeaderValue, cloudOriginOf, isCloudProvider, resolveCloudOrigin } from './cloudModels.js'; import { defaultImageConfig, loadProvidersState, saveProvidersState } from './llm.js'; import { loadWebAcquireSettings, saveWebAcquireSettings } from './webAcquireSettings.js'; import { BYOK_REQUIRES_PLAN, clearByokEntitlement, writeByokEntitlement } from './byokGate.js'; export const CLOUD_API_BASE = `${CLOUD_ORIGIN}/api/v1`; export const CLOUD_TAVILY_BASE = `${CLOUD_ORIGIN}/api/tavily`; export const CLOUD_FIRECRAWL_BASE = `${CLOUD_ORIGIN}/api/firecrawl`; export const CLOUD_ACCOUNT_URL = `${CLOUD_ORIGIN}/account.html`; /** API bases for one backend origin (production or, for unpacked builds, staging). */ export function cloudBases(origin = CLOUD_ORIGIN) { return { api: `${origin}/api/v1`, tavily: `${origin}/api/tavily`, firecrawl: `${origin}/api/firecrawl`, account: `${origin}/account.html` }; } /** A Tavily / Firecrawl slot filled by sign-in (points at one of our origins), not the user's own. */ const isCloudSlot = (baseURL, kind) => { const origin = cloudOriginOf(baseURL); return !!origin && String(baseURL).replace(/\/$/, '') === cloudBases(origin)[kind]; }; export const CLOUD_ACCOUNT_KEY = 'browserbot_cloud_account'; export const CLOUD_NAME = 'Browser Kitten'; export { CLOUD_IDS, CLOUD_MODELS, CLOUD_ORIGIN, isCloudProvider }; /** * Each record always sends its route (bk-auto, …); the server picks the real model. * @returns {Record<'chat'|'image'|'stt'|'jev', object>} */ export function cloudProviderRecords(token, origin = CLOUD_ORIGIN) { const CLOUD_API_BASE = cloudBases(origin).api; const createdAt = Date.now(); return { chat: { id: CLOUD_IDS.chat, name: CLOUD_NAME, baseURL: CLOUD_API_BASE, apiKey: token, model: CLOUD_MODELS.chat, purpose: 'chat', protocol: 'openai-compatible', createdAt }, image: { id: CLOUD_IDS.image, name: CLOUD_NAME, baseURL: '', apiKey: '', model: '', purpose: 'image', createdAt, image: defaultImageConfig({ enabled: true, protocol: 'openai-image', path: '/images/generations', baseURL: CLOUD_API_BASE, apiKey: token, model: CLOUD_MODELS.image }) }, stt: { id: CLOUD_IDS.stt, name: CLOUD_NAME, baseURL: CLOUD_API_BASE, apiKey: token, model: CLOUD_MODELS.stt, purpose: 'stt', createdAt }, jev: { id: CLOUD_IDS.jev, name: CLOUD_NAME, baseURL: CLOUD_API_BASE, apiKey: token, model: CLOUD_MODELS.jev, purpose: 'jev', createdAt } }; } const usable = (p) => !!p && (!!String(p.apiKey || '').trim() || !!String(p.image?.apiKey || '').trim()); /** * Which extra services the server hosts, from /api/me `hosted`. Chat is always written * (it is the point of signing in). An absent or malformed `hosted` means an older * server that hosts everything, as before. * @param {{ image?: boolean, stt?: boolean, systemone?: boolean } | null | undefined} hosted */ export function cloudHostedPurposes(hosted) { const all = !hosted || typeof hosted !== 'object'; return { chat: true, image: all || hosted.image === true, stt: all || hosted.stt === true, jev: all || hosted.systemone === true }; } /** * What sign-in writes. Cloud becomes active for a purpose only when nothing usable is active, * and image / STT / System1 records only exist when the server hosts them. * @param {Awaited<ReturnType<typeof loadProvidersState>>} state * @param {ReturnType<typeof import('./webAcquireSettings.js').normalizeWebAcquireSettings>} web * @param {string} token * @param {{ chat?: boolean, light?: boolean, systemone?: boolean, image?: boolean, stt?: boolean } | null} [hosted] /api/me `hosted` */ export function planCloudSignIn(state, web, token, hosted = null, origin = CLOUD_ORIGIN) { const records = cloudProviderRecords(token, origin); const bases = cloudBases(origin); const on = cloudHostedPurposes(hosted); const added = ['chat', 'image', 'stt', 'jev'].filter((k) => on[k]).map((k) => records[k]); const providers = [...(state.providers || []).filter((p) => !isCloudProvider(p)), ...added]; // A purpose the server does not host keeps the user's choice, minus any stale cloud id. const pick = (activeId, active, cloudId, purpose) => { if (usable(active) && !isCloudProvider(active)) return activeId; if (on[purpose]) return cloudId; return activeId && !String(activeId).startsWith('bk_cloud_') ? activeId : null; }; const webPatch = {}; const filled = { tavily: false, firecrawl: false }; if (!web.tavilyKey || isCloudSlot(web.tavilyBaseURL, 'tavily')) { Object.assign(webPatch, { tavilyKey: token, tavilyBaseURL: bases.tavily, tavilyNickname: CLOUD_NAME }); if (!web.braveKey) webPatch.searchProvider = 'tavily'; filled.tavily = true; } if (!web.firecrawlKey || isCloudSlot(web.firecrawlBaseURL, 'firecrawl')) { Object.assign(webPatch, { firecrawlKey: token, firecrawlBaseURL: bases.firecrawl, firecrawlNickname: CLOUD_NAME }); filled.firecrawl = true; } return { providers, activeProviderId: pick(state.activeProviderId, state.active, CLOUD_IDS.chat, 'chat'), activeImageProviderId: pick(state.activeImageProviderId, state.activeImage, CLOUD_IDS.image, 'image'), activeSttProviderId: pick(state.activeSttProviderId, state.activeStt, CLOUD_IDS.stt, 'stt'), activeJevProviderId: pick(state.activeJevProviderId, state.activeJev, CLOUD_IDS.jev, 'jev'), webPatch, filled }; } /** What sign-out undoes: the cloud records, and web slots only while they still point at us. */ export function planCloudSignOut(state, web) { const providers = (state.providers || []).filter((p) => !isCloudProvider(p)); const keep = (id) => (id && !String(id).startsWith('bk_cloud_') ? id : null); const webPatch = {}; if (isCloudSlot(web.tavilyBaseURL, 'tavily')) Object.assign(webPatch, { tavilyKey: '', tavilyBaseURL: '', tavilyNickname: '' }); if (isCloudSlot(web.firecrawlBaseURL, 'firecrawl')) Object.assign(webPatch, { firecrawlKey: '', firecrawlBaseURL: '', firecrawlNickname: '' }); return { providers, activeProviderId: keep(state.activeProviderId), activeImageProviderId: keep(state.activeImageProviderId), activeSttProviderId: keep(state.activeSttProviderId), activeJevProviderId: keep(state.activeJevProviderId), webPatch }; } /** The server's 402 body (see site server/proxy/metered.js), however the caller surfaced it. */ export function isCloudCreditsError(error) { const status = Number(error?.statusCode ?? error?.status) || 0; const text = `${error?.code || ''} ${error?.message || ''} ${typeof error?.responseBody === 'string' ? error.responseBody : ''}`; return /CREDITS_EXHAUSTED|Out of Browser Kitten credits/i.test(text) || (status === 402 && /browser ?kitten/i.test(text)); } function cloudErrorParts(error) { const status = Number(error?.statusCode ?? error?.status) || 0; const body = typeof error?.responseBody === 'string' ? error.responseBody : ''; const text = `${error?.code || ''} ${error?.message || ''} ${body}`; return { status, text, body }; } /** The proxy's 429 USAGE_LIMIT: the weekly cap on plan credits. Booster credits still work. */ export function isCloudUsageLimitError(error) { const { status, text } = cloudErrorParts(error); return /USAGE_LIMIT|usage limit on plan credits/i.test(text) || (status === 429 && /browser ?kitten/i.test(text) && /limit/i.test(text)); } /** When a usage limit frees up, from the error body or the server's sentence ("frees up by <ISO>"); '' if unknown. */ export function cloudLimitResetAt(error) { const { body, text } = cloudErrorParts(error); if (body) { try { const at = JSON.parse(body)?.error?.resetAt; if (typeof at === 'string' && !Number.isNaN(Date.parse(at))) return at; } catch { /* not JSON */ } } const m = text.match(/\bby (\d{4}-\d{2}-\d{2}T[\d:.]+Z?)/); return m && !Number.isNaN(Date.parse(m[1])) ? m[1] : ''; } /** The proxy's 426: this build is older than the server's minVersion. */ export function isCloudUpgradeError(error) { const { status, text } = cloudErrorParts(error); return /CLIENT_UPGRADE_REQUIRED/i.test(text) || (status === 426 && /browser ?kitten/i.test(text)); } /** The proxy's 503: the operator paused credits (all or one category). Another vendor's 503 is not ours. */ export function isCloudPausedError(error) { const { status, text } = cloudErrorParts(error); // status 0: the SDK retries 503s and its RetryError carries only the last message. return /SERVICE_PAUSED/i.test(text) || ((status === 503 || status === 0) && /browser ?kitten/i.test(text) && /\bpaused?\b/i.test(text)); } /** * The server's own words for a pause, or '' when there is nothing better than a * code / status to show (the caller then uses its localized default). */ export function cloudPausedMessage(error) { const { body } = cloudErrorParts(error); if (body) { try { const m = JSON.parse(body)?.error?.message; if (typeof m === 'string' && m.trim()) return m.trim(); } catch { /* not JSON */ } } const msg = String(error?.message || '') .replace(/^Failed after \d+ attempts?\.\s*Last error:\s*/i, '') .replace(/\bSERVICE_PAUSED\b:?\s*/g, '') .trim(); if (!msg || /^HTTP \d+$/i.test(msg) || /^(Service Unavailable|unknown( error)?)$/i.test(msg)) return ''; return msg; } /** A BYOK call refused because the account has no paid plan (byokGate.js), however it surfaced. */ export function isByokPlanError(error) { if (!error || typeof error !== 'object') return false; const text = `${error.code || ''} ${error.message || ''} ${error.error || ''}`; return new RegExp(`${BYOK_REQUIRES_PLAN}|needs a Browser Kitten plan`, 'i').test(text); } /** * Which account notice an error calls for, if any. Credits first (unchanged behavior), * then own keys without a plan. * @returns {{ kind: 'credits' } | { kind: 'limit', resetAt: string } | { kind: 'byok' } | { kind: 'upgrade' } | { kind: 'paused', message: string } | null} */ export function cloudNoticeFor(error) { if (isCloudCreditsError(error)) return { kind: 'credits' }; if (isCloudUsageLimitError(error)) return { kind: 'limit', resetAt: cloudLimitResetAt(error) }; if (isByokPlanError(error)) return { kind: 'byok' }; if (isCloudUpgradeError(error)) return { kind: 'upgrade' }; if (isCloudPausedError(error)) return { kind: 'paused', message: cloudPausedMessage(error) }; return null; } /** * Usage windows from /api/me, in the server's order. Keys may change (the 5-hour * window may go away); whatever is present is shown. * @returns {Array<{ key: string, used: number, limit: number|null }>} */ export function cloudUsageRows(usage, limits) { const u = usage && typeof usage === 'object' ? usage : {}; const l = limits && typeof limits === 'object' ? limits : {}; const keys = [...new Set([...Object.keys(u), ...Object.keys(l)])]; return keys.map((key) => { const raw = l[key]; const limit = raw == null || raw === '' || !Number.isFinite(Number(raw)) ? null : Number(raw); return { key, used: Number(u[key]) || 0, limit }; }); } // ---------- chrome-facing ---------- async function readAccount() { const bag = await chrome.storage.local.get(CLOUD_ACCOUNT_KEY); const acct = bag?.[CLOUD_ACCOUNT_KEY]; return acct && typeof acct.token === 'string' && acct.token ? acct : null; } export async function getCloudAccount() { try { return await readAccount(); } catch { return null; } } /** The backend this signed-in browser uses: the origin chosen at sign-in (production by default). */ export const accountOrigin = (acct) => cloudOriginOf(acct?.origin) || CLOUD_ORIGIN; /** Account page of the backend this browser is signed in to. */ export async function cloudAccountUrl() { return cloudBases(accountOrigin(await getCloudAccount())).account; } async function cloudJson(path, { token, method = 'GET', body, fetchImpl = fetch, origin = CLOUD_ORIGIN } = {}) { const res = await fetchImpl(`${origin}${path}`, { method, headers: { ...(bkClientHeaderValue() ? { [BK_CLIENT_HEADER]: bkClientHeaderValue() } : {}), ...(token ? { authorization: `Bearer ${token}` } : {}), ...(body ? { 'content-type': 'application/json' } : {}) }, body: body ? JSON.stringify(body) : undefined }); const json = await res.json().catch(() => ({})); if (!res.ok) { throw Object.assign(new Error(json?.error?.message || `HTTP ${res.status}`), { status: res.status, code: json?.error?.code, responseBody: JSON.stringify(json || {}) }); } return json; } function deviceLabel() { const ua = String(globalThis.navigator?.userAgent || ''); const browser = /Edg\//.test(ua) ? 'Edge' : 'Chrome'; const os = /Mac OS X/.test(ua) ? 'Mac' : /Windows/.test(ua) ? 'Windows' : /Linux/.test(ua) ? 'Linux' : ''; return os ? `${browser} on ${os}` : browser; } /** * Opens browserkitten.com in the identity window; the user signs in and approves * this browser; the page returns a one-time code, exchanged here for a device token. */ export async function signInWithBrowserKitten({ fetchImpl = fetch } = {}) { if (!chrome?.identity?.launchWebAuthFlow) throw new Error('chrome.identity is unavailable. Reload the extension.'); const origin = await resolveCloudOrigin(); const state = crypto.randomUUID(); const redirectUri = chrome.identity.getRedirectURL('bk'); const url = `${origin}/connect.html?redirect_uri=${encodeURIComponent(redirectUri)}&state=${encodeURIComponent(state)}`; const responseUrl = await chrome.identity.launchWebAuthFlow({ url, interactive: true }); const back = new URL(String(responseUrl || '')); if (back.searchParams.get('state') !== state) throw new Error('Sign-in was interrupted. Try again.'); const code = back.searchParams.get('code'); if (!code) throw new Error('Sign-in did not return a code.'); const { token, deviceId } = await cloudJson('/api/device/exchange', { method: 'POST', body: { code - src/agent/cloudModels.js (3459 bytes)
/** * Browser Kitten credit records: fixed ids, and the route each one always sends. * Credit users never pick a model; the server maps each route to a real model * (site server/modelRoutes.js). Leaf module so llm.js can pin records without * importing cloudAccount.js (which imports llm.js). */ export const CLOUD_ORIGIN = 'https://browserkitten.com'; /** * Staging backend for testing a release before production. Only an unpacked * (development) build can be pointed at it, by setting chrome.storage.local * `browserbot_cloud_origin` to this exact URL before signing in; a Web Store build * (manifest has update_url) always uses CLOUD_ORIGIN. Both count as "our" API, so * the x-bk-* headers and the credit records work against either. */ export const STAGING_ORIGIN = 'https://staging.browserkitten.com'; export const CLOUD_ORIGINS = [CLOUD_ORIGIN, STAGING_ORIGIN]; export const CLOUD_ORIGIN_OVERRIDE_KEY = 'browserbot_cloud_origin'; export const isCloudOrigin = (origin) => CLOUD_ORIGINS.includes(String(origin || '')); /** The origin of a URL when it is one of ours, else ''. */ export function cloudOriginOf(url) { try { const origin = new URL(String(url || '')).origin; return isCloudOrigin(origin) ? origin : ''; } catch { return ''; } } /** * Backend for a new sign-in. Store builds: always production. Unpacked builds: * staging only when explicitly opted in; anything else falls back to production. */ export async function resolveCloudOrigin({ storage = globalThis.chrome?.storage?.local, manifest = globalThis.chrome?.runtime?.getManifest?.() } = {}) { if (manifest?.update_url) return CLOUD_ORIGIN; try { const value = (await storage?.get?.(CLOUD_ORIGIN_OVERRIDE_KEY))?.[CLOUD_ORIGIN_OVERRIDE_KEY]; return value === STAGING_ORIGIN ? STAGING_ORIGIN : CLOUD_ORIGIN; } catch { return CLOUD_ORIGIN; } } /** * Marks cheap background calls (compaction, titles) so the server can route them to a * lighter model. Only ever sent to the Browser Kitten API; stripped for any other host. */ export const BK_PURPOSE_HEADER = 'x-bk-purpose'; export function isCloudApiUrl(url) { try { const u = new URL(String(url || '')); return isCloudOrigin(u.origin) && u.pathname.startsWith('/api/'); } catch { return false; } } export const CLOUD_IDS = { chat: 'bk_cloud_chat', image: 'bk_cloud_image', stt: 'bk_cloud_stt', jev: 'bk_cloud_jev' }; export const CLOUD_MODELS = { chat: 'bk-auto', image: 'bk-image', stt: 'bk-stt', jev: 'bk-system1' }; export const isCloudProviderId = (id) => typeof id === 'string' && id.startsWith('bk_cloud_'); export const isCloudProvider = (p) => isCloudProviderId(p?.id); /** The route a credit record must send, or null for the user's own provider. */ export function pinnedCloudModel(id) { for (const [purpose, cloudId] of Object.entries(CLOUD_IDS)) if (cloudId === id) return CLOUD_MODELS[purpose]; return null; } /** * Names this build on every Browser Kitten API request so the server can ask old * clients to update (HTTP 426 CLIENT_UPGRADE_REQUIRED). Only sent to our own API. * '' outside an extension (no manifest): callers then send no header at all. */ export const BK_CLIENT_HEADER = 'x-bk-client'; export function bkClientHeaderValue() { try { const version = globalThis.chrome?.runtime?.getManifest?.()?.version; return version ? `browser-kitten/${version}` : ''; } catch { return ''; } } - src/agent/contextWindowProbe.js (17300 bytes)
import { BRAND_NAME } from '../brand.js'; import { findCatalogModel, loadCachedModelsForBase, normalizeModelsBaseURL, parseContextWindow, resolveContextWindow } from './modelCatalog.js'; import { loadLlmSettings } from './llm.js'; import { byokAllowedForProvider } from './byokGate.js'; import { CONTEXT_WINDOW_AUTO_KEY, CONTEXT_WINDOW_CACHE_KEY, MAX_CONTEXT_WINDOW_EVIDENCE, contextWindowCacheKey, contextWindowEvidenceDue, contextWindowStorageGet, contextWindowStorageSet, normalizeContextWindowEvidence, normalizeContextWindowOverrides } from './contextWindowStore.js'; export const PROBE_MAX_TOKENS = 10_000_000; export const PROBE_FIRST_BYTE_MS = 2000; export const PROBE_TOTAL_MS = 8000; const CONTEXT_PHRASES = [ /maximum context length is\s+(\d{4,})/i, /(?:this model's\s+)?maximum context length(?:\s+is|\s+of)?\s*[:=]?\s*(\d{4,})/i, /context window(?:\s+is|\s+of)?\s*[:=]?\s*(\d{4,})/i, /context length(?:\s+is|\s+of)?\s*[:=]?\s*(\d{4,})/i, /context[_\s-]?length["'\s:=]+(\d{4,})/i, /context[_\s-]?window["'\s:=]+(\d{4,})/i, /(?:exceeds?|over)\s+the\s+context\s+window\s+of\s+(\d{4,})/i, /上下文长度[为是::\s]*(\d{4,})/ ]; const MAX_TOKENS_RANGE = /range of(?:\s+input)?\s+max[_\s-]?tokens\s*\[\s*\d+\s*,\s*\d+\s*\]/i; const MAX_TOKENS_ALLOWED = /max[_\s-]?tokens\b[^.\n]{0,64}(?:max(?:imum)?(?:\s+allowed)?|allowed|must be|<=|less than)\s*\d+/i; const HEADER_ADOPT = /^(?:x-)?context(?:-(?:length|window))?$/i; function adoptWindow(n) { const value = Number(n); if (!Number.isFinite(value) || value < 2048 || value >= PROBE_MAX_TOKENS) return null; return Math.round(value); } function nearbyIsMaxTokensCap(source, index, matched) { const start = Math.max(0, index - 72); const end = Math.min(source.length, index + matched.length + 48); const around = source.slice(start, end); if (MAX_TOKENS_RANGE.test(around) || MAX_TOKENS_ALLOWED.test(around)) return true; return /max[_\s-]?tokens/i.test(around) && !/context/i.test(around) && !/上下文/.test(around); } /** Parse ONLY a context window. Never an output cap. */ export function extractContextWindowFromText(text) { const s = String(text || ''); if (!s) return null; if (/\busage\.?prompt_tokens\b|\bprompt_tokens\b\s*[:=]\s*\d+/i.test(s) && !/context/i.test(s) && !/上下文/.test(s)) { return null; } for (const re of CONTEXT_PHRASES) { re.lastIndex = 0; const match = re.exec(s); if (!match) continue; if (nearbyIsMaxTokensCap(s, match.index, match[0])) continue; const n = adoptWindow(match[1]); if (n) return n; } return null; } export function extractContextWindowFromHeaders(headers) { if (!headers) return null; const pairs = []; if (typeof headers.forEach === 'function') { headers.forEach((value, name) => pairs.push([name, value])); } else if (typeof headers === 'object') { for (const [name, value] of Object.entries(headers)) pairs.push([name, value]); } for (const [name, value] of pairs) { const key = String(name || '').toLowerCase(); if (key.startsWith('x-ratelimit')) continue; if (!HEADER_ADOPT.test(key)) continue; const n = adoptWindow(String(value ?? '').replace(/[^\d.]/g, '')); if (n) return n; } return null; } export function extractContextWindowFromBody(raw) { if (!raw || typeof raw !== 'object') return null; const o = /** @type {any} */ (raw); const objects = [o, o.error, o.usage, o.model_info, o.top_provider, o.metadata, o.error?.metadata]; for (const obj of objects) { if (!obj || typeof obj !== 'object') continue; const n = parseContextWindow(obj); if (n) return n; } const texts = [o.error?.message, o.message, typeof o.error === 'string' ? o.error : '']; for (const text of texts) { const n = extractContextWindowFromText(text); if (n) return n; } return null; } export function extractContextWindowFromProbe({ headers, body, text } = {}) { const fromHeaders = extractContextWindowFromHeaders(headers); if (fromHeaders) return fromHeaders; if (body && typeof body === 'object') { const fromBody = extractContextWindowFromBody(body); if (fromBody) return fromBody; } const raw = String(text || ''); if (raw) { const fromText = extractContextWindowFromText(raw); if (fromText) return fromText; try { const parsed = JSON.parse(raw); const fromParsed = extractContextWindowFromBody(parsed); if (fromParsed) return fromParsed; } catch { /* SSE or HTML */ } } return null; } export function streamHasContentToken(text) { for (const line of String(text || '').split(/\r?\n/)) { const trimmed = line.trim(); if (!trimmed.startsWith('data:')) continue; const raw = trimmed.slice(5).trim(); if (!raw || raw === '[DONE]') continue; try { const json = JSON.parse(raw); const delta = json.choices?.[0]?.delta || json.choices?.[0]?.message || {}; const content = delta.content; if (typeof content === 'string' && content.length) return true; if (Array.isArray(content) && content.some((part) => part?.text || part?.content)) return true; } catch { /* keep scanning */ } } return false; } function extractFromSse(text) { const fromText = extractContextWindowFromText(text); if (fromText) return fromText; for (const line of String(text || '').split(/\r?\n/)) { const trimmed = line.trim(); if (!trimmed.startsWith('data:')) continue; const raw = trimmed.slice(5).trim(); if (!raw || raw === '[DONE]') continue; try { const n = extractContextWindowFromBody(JSON.parse(raw)); if (n) return n; } catch { /* ignore */ } } return null; } export function contextWindowChatCompletionsUrl(baseURL) { return `${normalizeModelsBaseURL(baseURL)}/chat/completions`; } export function contextWindowProbeBody(model) { return { model: String(model || '').trim(), messages: [{ role: 'user', content: '.' }], max_tokens: PROBE_MAX_TOKENS, stream: true }; } export function contextWindowProbeHeaders(settings = {}) { const headers = { Authorization: `Bearer ${String(settings.apiKey || '').trim()}`, 'Content-Type': 'application/json', Accept: 'text/event-stream' }; const base = normalizeModelsBaseURL(settings.apiBase || ''); if (base.includes('openrouter.ai')) { headers['HTTP-Referer'] = 'https://browserbot.local'; headers['X-Title'] = BRAND_NAME; } return headers; } function kindFromStatus(status, firstToken) { if (status === 401 || status === 403 || status === 404 || status === 405 || status === 429) return `HTTP_${status}`; if (status >= 500) return 'HTTP_5xx'; if (status === 200) return 'STREAM_STARTED'; if (status) return `HTTP_${status}`; return firstToken ? 'STREAM_STARTED' : 'CHECK_UNAVAILABLE'; } async function defaultProbeFetch(input, init) { if (typeof globalThis.chrome?.runtime?.id !== 'string') { throw Object.assign(new Error('CHECK_UNAVAILABLE'), { code: 'CHECK_UNAVAILABLE' }); } return fetch(input, init); } async function readProbeResponse(res, signal) { const headers = res.headers; const headerWindow = extractContextWindowFromHeaders(headers); if (!res.body || typeof res.body.getReader !== 'function') { const text = await res.text(); let body = null; try { body = JSON.parse(text); } catch { /* non-json */ } return { status: res.status, window: headerWindow || extractContextWindowFromProbe({ headers, body, text }), firstToken: false }; } const reader = res.body.getReader(); const decoder = new TextDecoder(); let buf = ''; let firstToken = false; try { while (!signal?.aborted) { const { done, value } = await reader.read(); if (done) break; buf += decoder.decode(value, { stream: true }); const found = headerWindow || extractFromSse(buf); firstToken = streamHasContentToken(buf); if (found || firstToken) { return { status: res.status, window: found || null, firstToken }; } } } finally { try { await reader.cancel(); } catch { /* already closed */ } } let body = null; try { body = JSON.parse(buf); } catch { /* SSE */ } return { status: res.status, window: headerWindow || extractContextWindowFromProbe({ headers, body, text: buf }), firstToken }; } /** * Dedicated offscreen POST {base}/chat/completions. Not ToolLoopAgent / sendMessage. * Abort on first token, ~2s first-byte, or ~8s total. A 200 stream is unknown. */ export async function runContextWindowProbe({ settings, signal, fetchImpl, firstByteMs = PROBE_FIRST_BYTE_MS, totalMs = PROBE_TOTAL_MS } = {}) { const fetchFn = typeof fetchImpl === 'function' ? fetchImpl : defaultProbeFetch; const url = contextWindowChatCompletionsUrl(settings?.apiBase); const ac = new AbortController(); const abort = (reason) => { try { ac.abort(reason); } catch { /* ignore */ } }; if (signal) { if (signal.aborted) abort(signal.reason); else signal.addEventListener('abort', () => abort(signal.reason), { once: true }); } let firstByteTimer = setTimeout(() => abort('FIRST_BYTE_TIMEOUT'), firstByteMs); const totalTimer = setTimeout(() => abort('CHECK_TIMEOUT'), totalMs); let parsed = null; try { const res = await fetchFn(url, { method: 'POST', headers: contextWindowProbeHeaders(settings), body: JSON.stringify(contextWindowProbeBody(settings?.model)), signal: ac.signal }); clearTimeout(firstByteTimer); firstByteTimer = null; parsed = await readProbeResponse(res, ac.signal); } catch (error) { if (parsed?.window) { return { window: parsed.window, source: 'measured', kind: 'measure_chat' }; } if (parsed) { return { window: null, source: 'unknown', kind: kindFromStatus(parsed.status, parsed.firstToken) }; } const aborted = ac.signal.aborted || signal?.aborted; const reason = String(ac.signal.reason || error?.message || ''); if (aborted && /FIRST_BYTE/i.test(reason)) { return { window: null, source: 'unknown', kind: 'FIRST_BYTE_TIMEOUT' }; } if (aborted) return { window: null, source: 'unknown', kind: 'CHECK_TIMEOUT' }; const status = Number(error?.status || error?.statusCode) || 0; return { window: null, source: 'unknown', kind: status ? kindFromStatus(status, false) : (error?.code || 'CHECK_UNAVAILABLE') }; } finally { clearTimeout(firstByteTimer); clearTimeout(totalTimer); } if (parsed?.window) return { window: parsed.window, source: 'measured', kind: 'measure_chat' }; return { window: null, source: 'unknown', kind: kindFromStatus(parsed?.status, parsed?.firstToken) }; } export function contextWindowSource(modelId, entry, record, override) { if (Number(override) >= 2048) return 'user'; if (Number(record?.window) >= 2048) return 'measured'; if (Number(entry?.contextWindow) >= 2048) return 'catalog'; if (resolveContextWindow(modelId, entry, { allowDefault: false, evidence: record, override })) return 'family'; return 'unknown'; } /** One manager per offscreen runtime. Ordinary sends only kick request(); they never await it. */ export class ContextWindowChecks { constructor({ getSettings = loadLlmSettings, getModels = loadCachedModelsForBase, get = contextWindowStorageGet, set = contextWindowStorageSet, run = runContextWindowProbe, now = Date.now, firstByteMs = PROBE_FIRST_BYTE_MS, totalMs = PROBE_TOTAL_MS, byokAllowed = byokAllowedForProvider } = {}) { Object.assign(this, { getSettings, getModels, get, set, run, now, firstByteMs, totalMs, byokAllowed }); this.tail = Promise.resolve(); this.writes = Promise.resolve(); this.job = null; this.sequence = 0; } identify(settings) { return { key: contextWindowCacheKey(settings.apiBase, settings.model), settings }; } overrideFor(settings) { const map = normalizeContextWindowOverrides(settings?.contextWindowOverrides); return map[contextWindowCacheKey(settings.apiBase, settings.model)]; } async read(settings, models) { const { key } = this.identify(settings); const saved = await this.get([CONTEXT_WINDOW_CACHE_KEY]); const record = normalizeContextWindowEvidence(saved[CONTEXT_WINDOW_CACHE_KEY]?.[key]); const entry = findCatalogModel(models || [], settings.model); const override = this.overrideFor(settings); return { key, record, entry, override, window: resolveContextWindow(settings.model, entry, { evidence: record, override, allowDefault: false }) }; } async update(key, record) { const write = this.writes.then(async () => { const data = await this.get([CONTEXT_WINDOW_CACHE_KEY]); const cache = { ...(data[CONTEXT_WINDOW_CACHE_KEY] || {}) }; cache[key] = normalizeContextWindowEvidence(record) || { window: null, source: 'unknown', kind: String(record?.kind || 'unknown'), checkedAt: this.now() }; const entries = Object.entries(cache).sort((a, b) => (b[1]?.checkedAt || 0) - (a[1]?.checkedAt || 0)); await this.set({ [CONTEXT_WINDOW_CACHE_KEY]: Object.fromEntries(entries.slice(0, MAX_CONTEXT_WINDOW_EVIDENCE)) }); }); this.writes = write.catch(() => {}); return write; } async status() { const settings = await this.getSettings(); const models = (await this.getModels(settings.apiBase)).models || []; const result = await this.read(settings, models); const prefs = await this.get([CONTEXT_WINDOW_AUTO_KEY]); return { providerId: settings.providerId, model: settings.model, base: settings.apiBase, window: result.window, catalogWindow: Number(result.entry?.contextWindow) >= 2048 ? Math.round(Number(result.entry.contextWindow)) : null, evidence: result.record, source: contextWindowSource(settings.model, result.entry, result.record, result.override), autoCheck: prefs[CONTEXT_WINDOW_AUTO_KEY] !== false, checking: this.job?.key === result.key && !this.job.done }; } async request({ force = false } = {}) { const seq = ++this.sequence; const settings = await this.getSettings(); const models = (await this.getModels(settings.apiBase)).models || []; const { key, entry } = await this.read(settings, models); const prefs = await this.get([CONTEXT_WINDOW_AUTO_KEY]); // A probe against the user's own key is BYOK use: skipped silently without a plan. const byokOk = !settings.apiKey || await this.byokAllowed('probe', settings.providerId); if (seq !== this.sequence) return { queued: false }; if (!settings.apiKey || (!force && prefs[CONTEXT_WINDOW_AUTO_KEY] === false)) { this.job?.controller.abort(); return { queued: false }; } if (!byokOk) { this.job?.controller.abort(); return { queued: false, reason: 'BYOK_REQUIRES_PLAN' }; } if (Number(entry?.contextWindow) >= 2048) return { queu - src/agent/contextWindowStore.js (2980 bytes)
import { normalizeModelsBaseURL } from './modelCatalog.js'; /** Configuration-scoped evidence. No credentials or provider response bodies are stored. */ export const CONTEXT_WINDOW_CACHE_KEY = 'browserbot_context_window_evidence_v1'; export const CONTEXT_WINDOW_AUTO_KEY = 'browserbot_context_window_auto_check'; export const MAX_CONTEXT_WINDOW_EVIDENCE = 100; export const DAY = 86400000; /** * Same grain as mediaModelKey: exact normalized base + exact model id. * @param {string} [base] * @param {string} [model] */ export function contextWindowCacheKey(base, model) { return `${normalizeModelsBaseURL(base)}\n${String(model || '').trim()}`; } /** * @param {unknown} raw * @returns {Record<string, number>} */ export function normalizeContextWindowOverrides(raw) { const out = {}; if (!raw || typeof raw !== 'object') return out; for (const [key, value] of Object.entries(raw)) { if (!String(key).includes('\n')) continue; const n = Number(value); if (Number.isFinite(n) && n >= 2048 && n < 10_000_000) out[key] = Math.round(n); } return out; } export async function contextWindowStorageGet(keys) { if (globalThis.chrome?.storage?.local) return (await chrome.storage.local.get(keys)) || {}; if (!globalThis.chrome?.runtime?.sendMessage) return {}; const res = await chrome.runtime.sendMessage({ target: 'browserbot-background', action: 'storage_local_get', keys }); if (!res?.ok) throw new Error('CONTEXT_WINDOW_STORAGE_UNAVAILABLE'); return res.result || {}; } export async function contextWindowStorageSet(values) { if (globalThis.chrome?.storage?.local) return chrome.storage.local.set(values); const res = await chrome.runtime.sendMessage({ target: 'browserbot-background', action: 'storage_local_set', values }); if (!res?.ok) throw new Error('CONTEXT_WINDOW_STORAGE_UNAVAILABLE'); } /** * Measured windows live 30d. Unknown / 401 / 429 / 5xx cool down for 24h. * @param {{ window?: number|null, checkedAt?: number }|null|undefined} record * @param {number} [now] */ export function contextWindowEvidenceDue(record, now = Date.now()) { if (!record || !Number.isFinite(record.checkedAt)) return true; const ttl = Number(record.window) >= 2048 ? 30 * DAY : DAY; return now - record.checkedAt >= ttl; } /** * @param {unknown} raw * @returns {{ window: number|null, source: string, kind: string, checkedAt: number }|null} */ export function normalizeContextWindowEvidence(raw) { if (!raw || typeof raw !== 'object') return null; const checkedAt = Number(/** @type {any} */ (raw).checkedAt); if (!Number.isFinite(checkedAt)) return null; const n = Number(/** @type {any} */ (raw).window); const window = Number.isFinite(n) && n >= 2048 && n < 10_000_000 ? Math.round(n) : null; return { window, source: window ? 'measured' : String(/** @type {any} */ (raw).source || 'unknown'), kind: String(/** @type {any} */ (raw).kind || (window ? 'measure_chat' : 'unknown')), checkedAt }; } - src/agent/dataTools/structuredData.js (11917 bytes)
/** * Structured table/card parsing + CSV/JSON export helpers (pure JS, no deps). */ const DEFAULT_MAX_ROWS = 500; const DEFAULT_MAX_COLS = 40; /** * Collect stable column order from list of row objects. * @param {Array<Record<string, any>>} rows * @returns {string[]} */ export function collectFieldNames(rows) { const names = []; const seen = new Set(); for (const row of rows || []) { if (!row || typeof row !== 'object') continue; for (const key of Object.keys(row)) { if (!seen.has(key)) { seen.add(key); names.push(key); } } } return names; } /** * Escape one CSV cell (RFC-style quotes). * @param {any} value * @returns {string} */ export function csvEscape(value) { if (value == null) return ''; const s = String(value).replace(/\r\n/g, '\n').replace(/\r/g, '\n'); if (/[",\t\n]/.test(s)) { return `"${s.replace(/"/g, '""')}"`; } return s; } /** * RFC 4180-ish CSV parse. Quoted commas/newlines stay in one cell; BOM stripped. * @param {string} text * @param {{ maxRows?: number, maxCols?: number }} [opts] * @returns {string[][]} */ export function parseCsv(text, opts = {}) { const maxRows = Math.max(1, Number(opts.maxRows) || DEFAULT_MAX_ROWS); const maxCols = Math.max(1, Number(opts.maxCols) || DEFAULT_MAX_COLS); const delim = opts.delimiter === '\t' ? '\t' : ','; const src = String(text || '').replace(/^\uFEFF/, ''); if (!src.trim()) return []; /** @type {string[][]} */ const rows = []; /** @type {string[]} */ let row = []; let cell = ''; let quoted = false; for (let i = 0; i < src.length; i++) { const ch = src[i]; if (quoted) { if (ch === '"') { if (src[i + 1] === '"') { cell += '"'; i += 1; } else { quoted = false; } } else { cell += ch; } continue; } if (ch === '"') { quoted = true; continue; } if (ch === delim) { if (row.length < maxCols) row.push(cell); cell = ''; continue; } if (ch === '\n' || ch === '\r') { if (ch === '\r' && src[i + 1] === '\n') i += 1; if (row.length < maxCols) row.push(cell); if (rows.length < maxRows) rows.push(row); row = []; cell = ''; if (rows.length >= maxRows) break; continue; } cell += ch; } if (quoted || cell.length || row.length) { if (row.length < maxCols) row.push(cell); if (rows.length < maxRows) rows.push(row); } const width = rows.reduce((w, r) => Math.max(w, r.length), 0); return rows.map((r) => { const out = r.slice(0, maxCols); while (out.length < width && out.length < maxCols) out.push(''); return out; }); } /** * @param {Array<Record<string, any>>} rows * @param {{ fieldNames?: string[], bom?: boolean }} [opts] * @returns {string} */ export function rowsToCsv(rows, opts = {}) { const list = Array.isArray(rows) ? rows : []; if (list.length === 0) return opts.bom ? '\uFEFF' : ''; const fields = opts.fieldNames && opts.fieldNames.length ? opts.fieldNames : collectFieldNames(list); const lines = [fields.map(csvEscape).join(',')]; for (const row of list) { lines.push(fields.map((f) => csvEscape(row?.[f])).join(',')); } const body = lines.join('\n'); return opts.bom === false ? body : `\uFEFF${body}`; } /** * @param {Array<Record<string, any>>} rows * @param {{ pretty?: boolean }} [opts] * @returns {string} */ export function rowsToJson(rows, opts = {}) { const list = Array.isArray(rows) ? rows : []; return JSON.stringify(list, null, opts.pretty === false ? 0 : 2); } /** * Parse HTML tables into row objects (pure string parser). * @param {string} html * @param {{ maxRows?: number, maxCols?: number }} [opts] * @returns {Array<{ source: string, headers: string[], rows: Array<Record<string, string>>, tableIndex: number }>} */ export function parseHtmlTables(html, opts = {}) { const maxRows = opts.maxRows ?? DEFAULT_MAX_ROWS; const maxCols = opts.maxCols ?? DEFAULT_MAX_COLS; if (!html) return []; const tables = []; const tableRe = /<table(\s[^>]*)?>([\s\S]*?)<\/table>/gi; let tm; let tableIndex = 0; while ((tm = tableRe.exec(html)) !== null) { const tableHtml = tm[2]; const headers = []; const thRe = /<th(\s[^>]*)?>([\s\S]*?)<\/th>/gi; let th; while ((th = thRe.exec(tableHtml)) !== null && headers.length < maxCols) { headers.push(plainCell(th[2]) || `Column_${headers.length + 1}`); } const rows = []; const trRe = /<tr(\s[^>]*)?>([\s\S]*?)<\/tr>/gi; let tr; while ((tr = trRe.exec(tableHtml)) !== null && rows.length < maxRows) { const cells = []; const tdRe = /<t[dh](\s[^>]*)?>([\s\S]*?)<\/t[dh]>/gi; let td; while ((td = tdRe.exec(tr[2])) !== null && cells.length < maxCols) { cells.push(plainCell(td[2])); } // Skip pure header rows (all th already captured, no td) const onlyTh = /<td[\s>]/i.test(tr[2]) === false && /<th[\s>]/i.test(tr[2]); if (onlyTh && headers.length > 0) continue; if (cells.length === 0) continue; // Skip if identical to headers if (headers.length && cells.every((c, i) => c === headers[i])) continue; const row = {}; for (let i = 0; i < cells.length; i++) { const key = headers[i] || `Column_${i + 1}`; row[key] = cells[i]; } if (Object.keys(row).length) rows.push(row); } if (rows.length > 0) { tables.push({ source: 'table', headers: headers.length ? headers : collectFieldNames(rows), rows, tableIndex }); } tableIndex++; } return tables; } /** * Heuristic card/list extraction from class*=card|item|product|row. * @param {string} html * @param {{ maxRows?: number, maxFields?: number }} [opts] * @returns {Array<Record<string, string>>} */ export function parseHtmlCards(html, opts = {}) { const maxRows = opts.maxRows ?? DEFAULT_MAX_ROWS; const maxFields = opts.maxFields ?? 12; if (!html) return []; const cardRe = /<(div|li|article|section)(\s[^>]*class=["'][^"']*(?:card|item|product|result|listing|row)[^"']*["'][^>]*)>([\s\S]*?)<\/\1>/gi; const results = []; const seen = new Set(); let m; while ((m = cardRe.exec(html)) !== null && results.length < maxRows) { const inner = m[3]; // Skip nested tiny wrappers if (inner.length < 20) continue; const texts = extractStrippedStrings(inner, maxFields + 4); if (texts.length < 1) continue; const row = {}; texts.slice(0, maxFields).forEach((t, i) => { row[`Field_${i + 1}`] = t; }); // Prefer named fields from common sub-nodes const title = matchFirstText(inner, /<(h[1-6]|a)(\s[^>]*)?>([\s\S]*?)<\/\1>/i) || matchFirstText(inner, /class=["'][^"']*title[^"']*["'][^>]*>([\s\S]*?)</i); const price = matchFirstText(inner, /class=["'][^"']*price[^"']*["'][^>]*>([\s\S]*?)</i); const link = (inner.match(/<a\s+[^>]*href=["']([^"']+)["']/i) || [])[1]; if (title) row.Title = title.slice(0, 200); if (price) row.Price = price.slice(0, 80); if (link && !link.startsWith('javascript:')) row.Link = link.slice(0, 500); const sig = JSON.stringify(row); if (seen.has(sig)) continue; seen.add(sig); results.push(row); } return results; } /** * Flatten best data source from tables + cards. * Preference: largest table → cards → empty. * @param {string} html * @param {{ prefer?: 'auto'|'table'|'cards', maxRows?: number }} [opts] * @returns {{ rows: Array<Record<string, string>>, source: string, tableCount: number, cardCount: number, meta: object }} */ export function parseStructuredDataFromHtml(html, opts = {}) { const prefer = opts.prefer || 'auto'; const maxRows = opts.maxRows ?? DEFAULT_MAX_ROWS; const tables = parseHtmlTables(html, { maxRows }); const cards = prefer === 'table' ? [] : parseHtmlCards(html, { maxRows }); let bestTable = null; for (const t of tables) { if (!bestTable || t.rows.length > bestTable.rows.length) bestTable = t; } if (prefer === 'cards') { return { rows: cards.slice(0, maxRows), source: cards.length ? 'cards' : 'none', tableCount: tables.length, cardCount: cards.length, meta: { preferred: prefer } }; } if (prefer === 'table' || (prefer === 'auto' && bestTable && bestTable.rows.length >= 2)) { if (bestTable) { return { rows: bestTable.rows.slice(0, maxRows), source: 'table', tableCount: tables.length, cardCount: cards.length, meta: { tableIndex: bestTable.tableIndex, headers: bestTable.headers } }; } } if (cards.length > 0) { return { rows: cards.slice(0, maxRows), source: 'cards', tableCount: tables.length, cardCount: cards.length, meta: {} }; } if (bestTable) { return { rows: bestTable.rows.slice(0, maxRows), source: 'table', tableCount: tables.length, cardCount: cards.length, meta: { tableIndex: bestTable.tableIndex, headers: bestTable.headers } }; } return { rows: [], source: 'none', tableCount: tables.length, cardCount: 0, meta: {} }; } /** * Convert selected elements summary → rows (selection export path). * @param {Array<{ tag?: string, selector?: string, text?: string, src?: string }>} elements * @returns {Array<Record<string, string>>} */ export function selectionToRows(elements) { return (elements || []).map((el, i) => ({ Index: String(i + 1), Tag: el.tag || '', Selector: el.selector || '', Text: (el.text || '').slice(0, 500), Src: (el.src || '').slice(0, 500) })); } /** * Build a downloadable data URL for text content. * @param {string} content * @param {string} mime * @returns {string} */ export function toDataUrl(content, mime = 'text/plain;charset=utf-8') { // Prefer percent-encoding for large unicode-safe CSV/JSON in content scripts return `data:${mime},${encodeURIComponent(content)}`; } /** * @param {Array<Record<string, any>>} rows * @param {'csv'|'json'} format * @returns {{ content: string, mime: string, extension: string, filenameHint: string }} */ export function formatRowsForDownload(rows, format = 'csv') { const fmt = String(format || 'csv').toLowerCase() === 'json' ? 'json' : 'csv'; if (fmt === 'json') { const content = rowsToJson(rows, { pretty: true }); return { content, mime: 'application/json;charset=utf-8', extension: 'json', filenameHint: `browserbot_data_${Date.now()}.json` }; } const content = rowsToCsv(rows, { bom: true }); return { content, mime: 'text/csv;charset=utf-8', extension: 'csv', filenameHint: `browserbot_data_${Date.now()}.csv` }; } // ── internals ────────────────────────────────────────────── function plainCell(html) { return String(html || '') .replace(/<br\s*\/?>/gi, ' ') .replace(/<[^>]+>/g, '') .replace(/ /gi, ' ') .replace(/&/gi, '&') .replace(/</gi, '<') .replace(/>/gi, '>') .replace(/"/gi, '"') .replace(/\s+/g, ' ') .trim() .slice(0, 500); } function extractStrippedStrings(html, max = 12) { const texts = []; const re = />([^<]{2,})</g; let m; while ((m = re.exec(html)) !== null && texts.length < max) { const t = m[1].replace(/\s+/g, ' ').trim(); if (t.length < 2) continue; if (/^[\d\s.,¥$€£%]+$/.test(t) && t.length < 2) continue; texts.push(t.slice(0, 300)); } return texts; } function matchFirstText(html, re) { const m = html.match(re); if (!m) return ''; const raw = m[3] != null ? m[3] : m[1]; return plainCell(raw); } export const STRUCTURED_MAX_ROWS = DEFAULT_MAX_ROWS; - src/agent/documentRender.js (24816 bytes)
/** * BrowserBot document compiler — single atomic renderer used by render_document. * Product profile **A′**: extension-only; no end-user Pandoc/npm. * PDF = print-ready HTML → system print → Save as PDF. */ import { createTextArtifact, createTableArtifact, createBinaryArtifact, packZipArtifact } from './artifacts.js'; import { RENDER_FORMATS, PRODUCT_FORMATS, FORMAT_ENGINE_MATRIX, resolveEngine, getRenderCapabilities } from './render/engine.js'; /** @typedef {import('./draftStore.js').DraftBlock} DraftBlock */ /** @typedef {import('./draftStore.js').BrowserBotDraft} BrowserBotDraft */ export { RENDER_FORMATS, PRODUCT_FORMATS, FORMAT_ENGINE_MATRIX, resolveEngine, getRenderCapabilities }; /** @deprecated A′ does not use pandoc in product path */ export function isPandocReady() { return false; } export function onPandocProgress() { return () => {}; } export async function ensurePandoc() { return { ok: false, code: 'ENGINE_NOT_IN_PRODUCT', message: 'Pandoc is not shipped in extension-only product A′' }; } export function getPandocLoadState() { return { status: 'disabled_a_prime' }; } export function setPandocConvertForTests() {} export function resetPandocLoaderForTests() {} /** * @param {DraftBlock[]} blocks * @returns {string} */ export function draftBlocksToMarkdown(blocks) { const lines = []; for (const b of blocks || []) { if (!b || b.type === 'slot') continue; if (b.type === 'heading') { lines.push(`## ${b.text || ''}`); lines.push(''); } else if (b.type === 'paragraph') { lines.push(b.text || ''); if (b.href) lines.push(`[link](${b.href})`); lines.push(''); } else if (b.type === 'image') { const alt = b.alt || b.text || 'image'; if (b.src) lines.push(``); else lines.push(`*[image: ${alt}]*`); lines.push(''); } else if (b.type === 'record' && b.fields) { for (const [k, v] of Object.entries(b.fields)) { lines.push(`- **${k}**: ${v}`); } lines.push(''); } else if (b.type === 'table' && b.columns && b.rows) { lines.push(`| ${b.columns.join(' | ')} |`); lines.push(`| ${b.columns.map(() => '---').join(' | ')} |`); for (const row of b.rows) { lines.push(`| ${b.columns.map((c) => String(row[c] ?? '').replace(/\|/g, '\\|')).join(' | ')} |`); } lines.push(''); } else if (b.type === 'divider') { lines.push('---'); lines.push(''); } } return lines.join('\n').trim() + '\n'; } /** * @param {DraftBlock[]} blocks * @returns {string} */ export function draftBlocksToPlainText(blocks) { return draftBlocksToMarkdown(blocks) .replace(/!\[[^\]]*\]\([^)]*\)/g, '[image]') .replace(/\[([^\]]+)\]\([^)]*\)/g, '$1') .replace(/[#*_`]/g, '') .replace(/\n{3,}/g, '\n\n') .trim() + '\n'; } /** * Extract first table from blocks for CSV. * @param {DraftBlock[]} blocks * @returns {{ columns: string[], rows: Array<Record<string, string>> }|null} */ export function extractTableFromBlocks(blocks) { for (const b of blocks || []) { if (b?.type === 'table' && Array.isArray(b.columns) && b.columns.length && Array.isArray(b.rows)) { return { columns: b.columns, rows: b.rows }; } } // Synthesize from records const records = (blocks || []).filter((b) => b?.type === 'record' && b.fields); if (records.length) { const colSet = new Set(); for (const r of records) { Object.keys(r.fields || {}).forEach((k) => colSet.add(k)); } const columns = [...colSet]; const rows = records.map((r) => { /** @type {Record<string, string>} */ const o = {}; for (const c of columns) o[c] = String(r.fields?.[c] ?? ''); return o; }); return { columns, rows }; } // heading+image pairs → simple table const pairs = []; let pendingTitle = ''; for (const b of blocks || []) { if (b?.type === 'heading' || (b?.type === 'paragraph' && (b.text || '').length < 80)) { pendingTitle = b.text || ''; } else if (b?.type === 'image') { pairs.push({ title: pendingTitle || b.alt || '', image: b.src || '', link: b.href || '' }); pendingTitle = ''; } } if (pairs.length) { return { columns: ['title', 'image', 'link'], rows: pairs }; } return null; } /** * Preview HTML for draft tab (includes insert slots). * @param {BrowserBotDraft|object} draft * @returns {string} */ export function draftToPreviewHtml(draft) { const title = escapeHtml(draft?.title || 'Browser Kitten Draft'); const version = draft?.version ?? 1; const blocks = Array.isArray(draft?.blocks) ? draft.blocks : []; const body = blocks .map((b) => { if (!b) return ''; const id = escapeHtml(b.id || ''); if (b.type === 'slot') { return `<div class="pw-block pw-slot browserbot-draft-slot" data-block-id="${id}" data-slot="1" tabindex="0"> <span class="pw-slot-plus">+</span> <span>${escapeHtml(b.text || '添加内容 / Insert')}</span> </div>`; } if (b.type === 'heading') { return `<h2 class="pw-block" data-block-id="${id}">${escapeHtml(b.text || '')}</h2>`; } if (b.type === 'paragraph') { return `<p class="pw-block" data-block-id="${id}">${escapeHtml(b.text || '')}</p>`; } if (b.type === 'image') { const src = b.src ? escapeAttr(b.src) : ''; const img = src ? `<img src="${src}" alt="${escapeAttr(b.alt || '')}" />` : `<div class="pw-img-missing">${escapeHtml(b.alt || 'image')}</div>`; return `<figure class="pw-block pw-figure" data-block-id="${id}">${img}<figcaption>${escapeHtml(b.alt || b.text || '')}</figcaption></figure>`; } if (b.type === 'table' && b.columns) { const head = b.columns.map((c) => `<th>${escapeHtml(c)}</th>`).join(''); const rows = (b.rows || []) .slice(0, 100) .map( (r) => `<tr>${b.columns.map((c) => `<td>${escapeHtml(String(r[c] ?? ''))}</td>`).join('')}</tr>` ) .join(''); return `<div class="pw-block pw-table-wrap" data-block-id="${id}"><table><thead><tr>${head}</tr></thead><tbody>${rows}</tbody></table></div>`; } if (b.type === 'record' && b.fields) { const lis = Object.entries(b.fields) .map(([k, v]) => `<li><strong>${escapeHtml(k)}</strong>: ${escapeHtml(v)}</li>`) .join(''); return `<ul class="pw-block pw-record" data-block-id="${id}">${lis}</ul>`; } if (b.type === 'divider') return `<hr class="pw-block" data-block-id="${id}" />`; return ''; }) .join('\n'); return `<!DOCTYPE html> <html lang="zh-CN" data-browserbot-role="draft-preview" data-draft-id="${escapeAttr(draft?.draftId || '')}"> <head> <meta charset="UTF-8" /> <meta name="viewport" content="width=device-width, initial-scale=1" /> <title>${title} · v${version}</title> <style> :root { font-family: system-ui, "Segoe UI", "PingFang SC", sans-serif; color: #0f172a; } body { margin: 0; background: #f1f5f9; } .pw-chrome { position: sticky; top: 0; z-index: 10; background: #0f172a; color: #e2e8f0; padding: 10px 16px; display: flex; justify-content: space-between; align-items: center; gap: 12px; font-size: 12px; } .pw-chrome strong { color: #a5b4fc; } .pw-main { max-width: 720px; margin: 20px auto; padding: 24px; background: #fff; border-radius: 12px; box-shadow: 0 4px 24px rgba(15,23,42,.08); } h2 { margin: 1.2em 0 .4em; font-size: 1.25rem; } p { line-height: 1.6; color: #334155; } figure { margin: 1em 0; } img { max-width: 100%; border-radius: 8px; display: block; } figcaption { font-size: 12px; color: #64748b; margin-top: 6px; } table { border-collapse: collapse; width: 100%; font-size: 13px; } th, td { border: 1px solid #e2e8f0; padding: 6px 8px; text-align: left; } th { background: #f8fafc; } .pw-slot { border: 2px dashed #6366f1; border-radius: 10px; padding: 16px; margin: 16px 0; color: #4f46e5; display: flex; align-items: center; gap: 10px; cursor: pointer; background: #eef2ff; user-select: none; } .pw-slot:hover { background: #e0e7ff; } .pw-slot-plus { font-size: 22px; font-weight: 700; } .pw-block.browserbot-selected, .pw-block.browserbot-scope-preview { outline: 3px solid #8b5cf6; outline-offset: 2px; } .pw-hint { font-size: 11px; color: #94a3b8; margin-top: 24px; } </style> </head> <body> <div class="pw-chrome"> <div><strong>Browser Kitten 草稿预览</strong> · ${title} · v${version} · 独立标签页 · 自动保存 · 刷新不丢</div> <div data-draft-id="${escapeAttr(draft?.draftId || '')}">${escapeHtml(draft?.draftId || '')}</div> </div> <main class="pw-main" data-browserbot-role="draft-preview"> ${body} <p class="pw-hint">点已有块 → 工具栏编辑/删除/后面插入;点紫色「+」新增。确认下载请在侧栏操作。</p> </main> </body> </html>`; } function escapeHtml(s) { return String(s ?? '') .replace(/&/g, '&') .replace(/</g, '<') .replace(/>/g, '>') .replace(/"/g, '"'); } function escapeAttr(s) { return escapeHtml(s).replace(/'/g, '''); } /** * Minimal multi-line text PDF (no external deps). * @param {string} text * @param {string} [docTitle] * @returns {Uint8Array} */ export function textToSimplePdf(text, docTitle = 'Browser Kitten') { const lines = String(text || '') .replace(/\r\n/g, '\n') .split('\n') .map((l) => l.slice(0, 90)); const maxLinesPerPage = 48; const pages = []; for (let i = 0; i < lines.length || pages.length === 0; i += maxLinesPerPage) { pages.push(lines.slice(i, i + maxLinesPerPage)); if (i >= lines.length) break; } /** @type {string[]} */ const objects = []; const add = (s) => { objects.push(s); return objects.length; }; // 1 catalog placeholder filled later add('<< /Type /Catalog /Pages 2 0 R >>'); // 2 pages tree const pageObjIds = []; const fontId = 3; add(''); // placeholder for pages — index 1 (object 2) add('<< /Type /Font /Subtype /Type1 /BaseFont /Helvetica >>'); /** @type {number[]} */ const contentIds = []; for (let p = 0; p < pages.length; p++) { const contentLines = [ 'BT', '/F1 11 Tf', '50 780 Td', '14 TL', `(${pdfEscape(docTitle.slice(0, 60))}) Tj`, 'T*', 'T*', ...pages[p].map((line) => `(${pdfEscape(line)}) Tj T*`), 'ET' ]; const stream = contentLines.join('\n'); const contentId = add( `<< /Length ${stream.length} >>\nstream\n${stream}\nendstream` ); contentIds.push(contentId); const pageId = add( `<< /Type /Page /Parent 2 0 R /MediaBox [0 0 612 792] /Contents ${contentId} 0 R /Resources << /Font << /F1 ${fontId} 0 R >> >> >>` ); pageObjIds.push(pageId); } // Fix pages object (index 1) objects[1] = `<< /Type /Pages /Kids [${pageObjIds.map((id) => `${id} 0 R`).join(' ')}] /Count ${pageObjIds.length} >>`; const enc = new TextEncoder(); /** @type {Uint8Array[]} */ const parts = [enc.encode('%PDF-1.4\n')]; /** @type {number[]} */ const offsets = [0]; let pos = parts[0].length; for (let i = 0; i < objects.length; i++) { offsets.push(pos); const chunk = enc.encode(`${i + 1} 0 obj\n${objects[i]}\nendobj\n`); parts.push(chunk); pos += chunk.length; } const xrefPos = pos; let xref = `xref\n0 ${objects.length + 1}\n0000000000 65535 f \n`; for (let i = 1; i <= objects.length; i++) { xref += `${String(offsets[i]).padStart(10, '0')} 00000 n \n`; } xref += `trailer\n<< /Size ${objects.length + 1} /Root 1 0 R >>\nstartxref\n${xrefPos}\n%%EOF\n`; parts.push(enc.encode(xref)); let total = 0; for (const p of parts) total += p.length; const out = new Uint8Array(total); let o = 0; for (const p of parts) { out.set(p, o); o += p.length; } return out; } function pdfEscape(s) { return String(s ?? '') .replace(/\\/g, '\\\\') .replace(/\(/g, '\\(') .replace(/\)/g, '\\)') // PDF basic fonts: strip non-latin-1 for Helvetica safety .replace(/[^\x20-\x7E]/g, '?'); } /** * Minimal PPTX-like HTML deck (downloadable .html slides; real pptx later). * @param {DraftBlock[]} blocks * @param {string} title * @returns {string} */ export function draftToSlideHtml(blocks, title) { /** @type {string[]} */ const slides = []; let buf = []; const flush = () => { if (!buf.length) return; slides.push(buf.join('\n')); buf = []; }; for (const b of blocks || []) { if (!b || b.type === 'slot') continue; if (b.type === 'heading') { flush(); buf.push(`<h1>${escapeHtml(b.text || '')}</h1>`); } else if (b.type === 'paragraph') { buf.push(`<p>${escapeHtml(b.text || '')}</p>`); } else if (b.type === 'image' && b.src) { buf.push(`<img src="${escapeAttr(b.src)}" alt="${escapeAttr(b.alt || '')}" style="max-height:50vh"/>`); } else if (b.type === 'divider') { flush(); } } flush(); if (!slides.length) slides.push(`<h1>${escapeHtml(title || 'Slides')}</h1>`); const sections = slides .map( (s, i) => `<section class="slide" id="s${i}"><div class="inner">${s}</div><div class="num">${i + 1}/${slides.length}</div></section>` ) .join('\n'); return `<!DOCTYPE html><html><head><meta charset="utf-8"/><title>${escapeHtml(title)}</title> <style> body{margin:0;font-family:system-ui,sans-serif;background:#0f172a;color:#f8fafc} .slide{min-height:100vh;display:flex;align-items:center;justify-content:center;padding:40px;box-sizing:border-box;border-bottom:1px solid #334155} .inner{max-width:900px;width:100%} h1{font-size:2.2rem;margin:0 0 1rem} p{font-size:1.2rem;line-height:1.5;color:#cbd5e1} img{max-width:100%;border-radius:8px} .num{position:fixed;right:16px;bottom:12px;opacity:.5;font-size:12px} </style></head><body>${sections}</body></html>`; } /** * Compile draft to an in-memory artifact (A′ extension-only). * Single tool surface: format enum → builtin or builtin_print (pdf). Never new tool names. * * @param {BrowserBotDraft|object} draft * @param {{ * format?: string, * runId?: string, * name?: string, * markdown?: string, * engine?: 'auto'|'builtin'|'builtin_print'|'pandoc', * forceBuiltin?: boolean, * forcePandoc?: boolean, * onEngineProgress?: (e: { phase: string, message?: string, error?: string }) => void * }} opts */ export async function renderDocumentFromDraft(draft, opts = {}) { const format = String(opts.format || draft?.targetFormat || 'md').toLowerCase(); if (!RENDER_FORMATS.includes(format)) { return { status: 'error', code: 'UNSUPPORTED_FORMAT', message: `Unsupported format: ${format}. Product A′ formats: ${RENDER_FORMATS.join(', ')}` }; } const runId = opts.runId || draft?.runId || 'draft_render'; const title = draft?.title || op - src/agent/draftStore.js (35351 bytes)
/** * BrowserBot Draft Store — durable draft for preview/refine (refresh-safe). * * Source of truth = serialized draft JSON in chrome.storage.local (or memory in tests). * Large image blobs stay as dataUrl strings on image blocks (capped); optional IDB later. * * Lifecycle: create/save while editing → purge after successful download (or explicit discard). */ /** @typedef {'heading'|'paragraph'|'image'|'record'|'table'|'slot'|'divider'} DraftBlockType */ /** * @typedef {Object} DraftBlock * @property {string} id * @property {DraftBlockType} type * @property {string} [text] * @property {string} [src] * @property {string} [alt] * @property {string} [href] * @property {string[]} [columns] * @property {Array<Record<string, string>>} [rows] * @property {Record<string, string>} [fields] * @property {string} [slotType] * @property {string} [sourceRef] */ /** * @typedef {Object} BrowserBotDraft * @property {string} draftId * @property {'editing'|'ready_for_export'|'delivered'|'purged'} status * @property {number} version * @property {number} createdAt * @property {number} updatedAt * @property {string} [title] * @property {string} [template] * @property {string} [targetFormat] * @property {DraftBlock[]} blocks * @property {{ pageUrl?: string, title?: string, domain?: string }} [source] * @property {string} [runId] * @property {string} [sessionId] */ export const DRAFTS_STORAGE_KEY = 'browserbot_drafts_v1'; export const MAX_DRAFTS = 20; export const MAX_BLOCKS = 200; export const MAX_IMAGE_DATAURL_CHARS = 400_000; /** @type {Map<string, BrowserBotDraft>|null} */ let memoryFallback = null; function storageAvailable() { return typeof chrome !== 'undefined' && chrome?.storage?.local; } function storageGet(keys) { if (!storageAvailable()) { return Promise.resolve({}); } return new Promise((resolve) => { chrome.storage.local.get(keys, (res) => resolve(res || {})); }); } function storageSet(obj) { if (!storageAvailable()) { return Promise.resolve(); } return new Promise((resolve) => { chrome.storage.local.set(obj, () => resolve()); }); } /** @returns {string} */ export function generateDraftId() { return `dr_${Date.now().toString(36)}_${Math.random().toString(36).slice(2, 8)}`; } /** @returns {string} */ export function generateBlockId() { return `b_${Date.now().toString(36)}_${Math.random().toString(36).slice(2, 6)}`; } /** * @param {Partial<DraftBlock>} b * @returns {DraftBlock|null} */ export function normalizeBlock(b) { if (!b || typeof b !== 'object') return null; const type = String(b.type || 'paragraph'); const allowed = new Set([ 'heading', 'paragraph', 'image', 'record', 'table', 'slot', 'divider' ]); if (!allowed.has(type)) return null; /** @type {DraftBlock} */ const out = { id: typeof b.id === 'string' && b.id ? b.id : generateBlockId(), type: /** @type {DraftBlockType} */ (type) }; if (b.text != null) out.text = String(b.text).slice(0, 8000); if (b.src != null) { let src = String(b.src); if (src.length > MAX_IMAGE_DATAURL_CHARS) { src = src.slice(0, MAX_IMAGE_DATAURL_CHARS); } out.src = src; } if (b.alt != null) out.alt = String(b.alt).slice(0, 500); if (b.href != null) out.href = String(b.href).slice(0, 2000); if (Array.isArray(b.columns)) { out.columns = b.columns.map((c) => String(c).slice(0, 80)).slice(0, 40); } if (Array.isArray(b.rows)) { out.rows = b.rows.slice(0, 500).map((row) => { if (!row || typeof row !== 'object') return {}; /** @type {Record<string, string>} */ const o = {}; for (const [k, v] of Object.entries(row)) { o[String(k).slice(0, 80)] = String(v ?? '').slice(0, 2000); } return o; }); } if (b.fields && typeof b.fields === 'object') { /** @type {Record<string, string>} */ const f = {}; for (const [k, v] of Object.entries(b.fields)) { f[String(k).slice(0, 80)] = String(v ?? '').slice(0, 2000); } out.fields = f; } if (b.slotType != null) out.slotType = String(b.slotType).slice(0, 40); if (b.sourceRef != null) out.sourceRef = String(b.sourceRef).slice(0, 120); return out; } /** * @param {Partial<BrowserBotDraft>} input * @returns {BrowserBotDraft|null} */ export function normalizeDraft(input) { if (!input || typeof input !== 'object') return null; const now = Date.now(); const blocks = (Array.isArray(input.blocks) ? input.blocks : []) .map(normalizeBlock) .filter(Boolean) .slice(0, MAX_BLOCKS); // Ensure at least one trailing insert slot for "add component" const hasSlot = blocks.some((b) => b.type === 'slot'); if (!hasSlot) { blocks.push({ id: generateBlockId(), type: 'slot', slotType: 'append', text: '点击此处添加内容 / Click to insert' }); } const statusRaw = String(input.status || 'editing'); const status = ['editing', 'ready_for_export', 'delivered', 'purged'].includes(statusRaw) ? /** @type {BrowserBotDraft['status']} */ (statusRaw) : 'editing'; return { draftId: typeof input.draftId === 'string' && input.draftId ? input.draftId : generateDraftId(), status, version: typeof input.version === 'number' && Number.isFinite(input.version) ? Math.max(1, Math.floor(input.version)) : 1, createdAt: typeof input.createdAt === 'number' && Number.isFinite(input.createdAt) ? input.createdAt : now, updatedAt: now, title: input.title != null ? String(input.title).slice(0, 200) : '', template: input.template != null ? String(input.template).slice(0, 64) : 'scrapbook', targetFormat: input.targetFormat != null ? String(input.targetFormat).slice(0, 16) : 'md', blocks, source: input.source && typeof input.source === 'object' ? { pageUrl: input.source.pageUrl ? String(input.source.pageUrl).slice(0, 2000) : undefined, title: input.source.title ? String(input.source.title).slice(0, 300) : undefined, domain: input.source.domain ? String(input.source.domain).slice(0, 200) : undefined } : undefined, runId: input.runId != null ? String(input.runId).slice(0, 80) : undefined, sessionId: input.sessionId != null ? String(input.sessionId).slice(0, 80) : undefined }; } /** * @returns {Promise<Record<string, BrowserBotDraft>>} */ async function loadAllMap() { if (!storageAvailable()) { if (!memoryFallback) memoryFallback = new Map(); /** @type {Record<string, BrowserBotDraft>} */ const o = {}; for (const [k, v] of memoryFallback) o[k] = v; return o; } const res = await storageGet([DRAFTS_STORAGE_KEY]); const raw = res[DRAFTS_STORAGE_KEY]; if (!raw || typeof raw !== 'object') return {}; /** @type {Record<string, BrowserBotDraft>} */ const out = {}; for (const [id, d] of Object.entries(raw)) { const n = normalizeDraft(d); if (n && n.status !== 'purged') out[id] = n; } return out; } /** * @param {Record<string, BrowserBotDraft>} map */ async function saveAllMap(map) { // Cap count by updatedAt const entries = Object.values(map).sort((a, b) => b.updatedAt - a.updatedAt); const kept = entries.slice(0, MAX_DRAFTS); /** @type {Record<string, BrowserBotDraft>} */ const next = {}; for (const d of kept) next[d.draftId] = d; if (!storageAvailable()) { memoryFallback = new Map(Object.entries(next)); return; } await storageSet({ [DRAFTS_STORAGE_KEY]: next }); } /** * @param {Partial<BrowserBotDraft>} input * @returns {Promise<BrowserBotDraft>} */ export async function createDraft(input = {}) { const draft = normalizeDraft({ ...input, draftId: input.draftId || generateDraftId(), version: 1, status: input.status || 'editing' }); if (!draft) throw new Error('invalid draft'); const map = await loadAllMap(); map[draft.draftId] = draft; await saveAllMap(map); return draft; } /** * @param {string} draftId * @returns {Promise<BrowserBotDraft|null>} */ export async function loadDraft(draftId) { if (!draftId) return null; const map = await loadAllMap(); return map[draftId] || null; } /** * @param {string} draftId * @param {Partial<BrowserBotDraft> & { blocks?: DraftBlock[], bumpVersion?: boolean }} patch * @returns {Promise<BrowserBotDraft|null>} */ export async function saveDraft(draftId, patch = {}) { const map = await loadAllMap(); const prev = map[draftId]; if (!prev) return null; const bump = patch.bumpVersion !== false; const merged = normalizeDraft({ ...prev, ...patch, draftId, version: bump ? (prev.version || 1) + 1 : prev.version, createdAt: prev.createdAt, blocks: patch.blocks != null ? patch.blocks : prev.blocks }); if (!merged) return null; map[draftId] = merged; await saveAllMap(map); return merged; } /** * @param {string} draftId * @returns {Promise<boolean>} */ export async function purgeDraft(draftId) { if (!draftId) return false; const map = await loadAllMap(); if (!map[draftId]) return false; delete map[draftId]; await saveAllMap(map); return true; } /** * @returns {Promise<BrowserBotDraft[]>} */ export async function listDrafts() { const map = await loadAllMap(); return Object.values(map).sort((a, b) => b.updatedAt - a.updatedAt); } /** * Build draft blocks from selection-like items + optional table rows. * @param {{ * elements?: Array<{ text?: string, tag?: string, src?: string, href?: string, kind?: string }>, * records?: Array<Record<string, string>>, * columns?: string[], * title?: string * }} opts * @returns {DraftBlock[]} */ export function blocksFromSelectionMaterial(opts = {}) { /** @type {DraftBlock[]} */ const blocks = []; if (opts.title) { blocks.push({ id: generateBlockId(), type: 'heading', text: String(opts.title).slice(0, 200) }); } const els = Array.isArray(opts.elements) ? opts.elements : []; els.slice(0, 80).forEach((el, i) => { const kind = String(el.kind || '').toLowerCase(); const tag = String(el.tag || '').toLowerCase(); const text = String(el.text || '').trim(); const src = el.src ? String(el.src) : ''; if (kind === 'image' || tag === 'img' || src) { blocks.push({ id: generateBlockId(), type: 'image', src: src || undefined, alt: text.slice(0, 200) || `image_${i + 1}`, href: el.href ? String(el.href) : undefined, sourceRef: `sel:${i}` }); if (text) { blocks.push({ id: generateBlockId(), type: 'paragraph', text: text.slice(0, 500), sourceRef: `sel:${i}:cap` }); } return; } if (/^h[1-6]$/.test(tag) || (text && text.length < 80 && !text.includes('\n'))) { blocks.push({ id: generateBlockId(), type: text.length < 80 ? 'heading' : 'paragraph', text: text || `(${tag || 'el'})`, href: el.href ? String(el.href) : undefined, sourceRef: `sel:${i}` }); return; } if (text) { blocks.push({ id: generateBlockId(), type: 'paragraph', text, href: el.href ? String(el.href) : undefined, sourceRef: `sel:${i}` }); } }); const records = Array.isArray(opts.records) ? opts.records : []; if (records.length) { const cols = Array.isArray(opts.columns) && opts.columns.length ? opts.columns.map(String) : Object.keys(records[0] || {}); blocks.push({ id: generateBlockId(), type: 'table', columns: cols, rows: records.map((r) => { /** @type {Record<string, string>} */ const o = {}; for (const c of cols) o[c] = String(r[c] ?? ''); return o; }) }); } blocks.push({ id: generateBlockId(), type: 'slot', slotType: 'append', text: '点击此处添加内容 / Click to insert' }); return blocks; } /** Layout / multi-block content → must open draft preview (not chat code walls). */ export const PREVIEW_HTML_CHAR_THRESHOLD = 1200; export const PREVIEW_MD_CHAR_THRESHOLD = 800; export const PREVIEW_MULTI_HEADING_MIN = 2; export const PREVIEW_MULTI_BLOCK_MIN = 3; /** User-facing finish/summary cap after HTML strip. */ export const USER_FACING_HTML_STRIP_THRESHOLD = 600; /** * Detect full / layout-hard HTML that must not be dumped into chat. * @param {string} text * @returns {boolean} */ export function looksLikeFullHtmlDocument(text) { const s = String(text || ''); if (!s) return false; if (/<!DOCTYPE\s+html/i.test(s)) return true; if (/<html[\s>]/i.test(s) && /<\/html>/i.test(s)) return true; if (s.length >= PREVIEW_HTML_CHAR_THRESHOLD) { if (/<\/?(?:body|head|style|section|article|main|table|div)[\s>]/i.test(s)) return true; const tagCount = (s.match(/<\/?[a-zA-Z][^>]*>/g) || []).length; if (tagCount >= 12) return true; } return false; } /** Reasons that force HARD tier (host auto-open preview on materialize). */ const HARD_PREVIEW_REASONS = new Set([ 'forcePreview', 'full_html_document', 'large_layout_markup', 'html_mime_or_name', 'layout_format_hint', 'media_or_table_blocks' ]); /** * @param {string} reason * @returns {boolean} */ export function isHardPreviewReason(reason) { const r = String(reason || ''); if (HARD_PREVIEW_REASONS.has(r)) return true; if (r.startsWith('template:')) { const t = r.slice('template:'.length); return ( t === 'report' || t === 'product_cards' || t === 'title_image_pairs' || t === 'slides' || t === 'pptx' ); } return false; } /** * Collect preview-need reasons (no tier). forcePreview short-circuits as sole hard signal * when no other content signals exist, but is still listed. * @param {{ * markdown?: string, * html?: string, * content?: string, * text?: string, * mime?: string, * name?: string, * template?: string, * blocks?: Array<Partial<DraftBlock>|DraftBlock>, * records?: unknown[], * forcePreview?: boolean * }} input * @returns {string[]} */ export function collectPreviewReasons(input = {}) { /** @type {string[]} */ const reasons = []; if (input?.forcePreview) { reasons.push('forcePreview'); } const template = String(input?.template || '').toLowerCase(); if ( template === 'report' || template === 'product_cards' || template === 'title_image_pairs' || template === 'slides' || template === 'pptx' ) { reasons.push(`template:${template || 'layout'}`); } const blocks = Array.isArray(input?.blocks) ? input.blocks : []; const contentBlocks = blocks.filter((b) => b && b.type !== 'slot'); if (contentBlocks.length >= PREVIEW_MULTI_BLOCK_MIN) { reasons.push(`blocks:${contentBlocks.length}`); } - src/agent/llm.js (35351 bytes)
/** * Provider settings for BYOK HTTPS inference. * Inference goes through AI SDK (`createBrowserBotLanguageModel`), not this file. * * browserbot_providers: [{ id, name, nickname?, baseURL, apiKey, model, protocol?, purpose?, createdAt, image?, lastProbe? }] * browserbot_active_provider_id: string * Flat copies (browserbot_api_key / browserbot_api_base / selected_model) stay in sync with the active chat provider. * * Chat `protocol` is the wire for ANY user base URL: * openai-compatible (default) | anthropic | openai-responses | auto * Explicit protocol always wins. Omitted → openai-compatible, except host * api.anthropic.com → anthropic. `auto` is optional model-id multiplexing. * Image keeps nested image.protocol. STT is purpose:'stt'. Jev is purpose:'jev' * + /systemone and must never share the chat LanguageModel path. */ import { STORAGE_ACTIVE_IMAGE_PROVIDER_ID, STORAGE_ACTIVE_PROVIDER_ID, STORAGE_ACTIVE_STT_PROVIDER_ID, STORAGE_ACTIVE_JEV_PROVIDER_ID, STORAGE_PROVIDERS } from '../brand.js'; import { normalizeMediaOverrides } from './chatMedia.js'; import { normalizeContextWindowOverrides } from './contextWindowStore.js'; import { CLOUD_IDS, pinnedCloudModel } from './cloudModels.js'; export const DEFAULT_BASE = 'https://api.deepseek.com/v1'; /** Canonical OpenRouter OpenAI-compatible origin (chat + dedicated POST /images). */ export const OPENROUTER_API_BASE = 'https://openrouter.ai/api/v1'; /** MiniMax-oriented image-gen defaults (path is relative to image base / chat origin). */ export const DEFAULT_IMAGE_PROTOCOL = 'minimax-image'; export const DEFAULT_IMAGE_PATH = '/image_generation'; export const DEFAULT_IMAGE_MODEL = 'image-01'; export const DEFAULT_OPENROUTER_IMAGE_PROTOCOL = 'openrouter-image'; export const DEFAULT_OPENROUTER_IMAGE_PATH = '/images'; export const DEFAULT_OPENROUTER_IMAGE_MODEL = 'google/gemini-2.5-flash-image'; /** * Suggested OpenCode Zen origin (OpenAI-compatible; client appends /systemone as needed). * DEFAULT_JEV_MODEL is Zen's published id only — never a silent fallback for other gateways. */ export const DEFAULT_JEV_BASE = 'https://opencode.ai/zen/v1'; export const DEFAULT_JEV_MODEL = 'jev-1.13'; /** Chat wire on the provider record (applies to any base URL). */ export const CHAT_PROTOCOL_OPENAI = 'openai-compatible'; export const CHAT_PROTOCOL_ANTHROPIC = 'anthropic'; export const CHAT_PROTOCOL_RESPONSES = 'openai-responses'; export const CHAT_PROTOCOL_AUTO = 'auto'; export const DEFAULT_CHAT_PROTOCOL = CHAT_PROTOCOL_OPENAI; /** chrome.storage.local keys */ export const PROVIDERS_STORAGE_KEY = STORAGE_PROVIDERS; export const ACTIVE_PROVIDER_ID_KEY = STORAGE_ACTIVE_PROVIDER_ID; export const ACTIVE_IMAGE_PROVIDER_ID_KEY = STORAGE_ACTIVE_IMAGE_PROVIDER_ID; export const ACTIVE_STT_PROVIDER_ID_KEY = STORAGE_ACTIVE_STT_PROVIDER_ID; export const ACTIVE_JEV_PROVIDER_ID_KEY = STORAGE_ACTIVE_JEV_PROVIDER_ID; export const DEFAULT_STT_MODEL = 'whisper-1'; export const DEFAULT_STT_BASE = 'https://api.openai.com/v1'; /** UI presets — fill form templates; catalog lives in providerPresets.js. */ export { PROVIDER_PRESETS } from './providerPresets.js'; /** * Normalize a stored chat protocol. Empty → null (omitted; resolver applies defaults). * @param {unknown} raw * @returns {typeof CHAT_PROTOCOL_OPENAI | typeof CHAT_PROTOCOL_ANTHROPIC | typeof CHAT_PROTOCOL_RESPONSES | typeof CHAT_PROTOCOL_AUTO | null} */ export function normalizeChatProtocol(raw) { if (raw == null) return null; const s = String(raw || '') .trim() .toLowerCase(); if (!s) return null; if (s === CHAT_PROTOCOL_ANTHROPIC || s === 'claude' || s === 'messages') { return CHAT_PROTOCOL_ANTHROPIC; } if (s === CHAT_PROTOCOL_RESPONSES || s === 'responses' || s === 'openai-responses') { return CHAT_PROTOCOL_RESPONSES; } if (s === CHAT_PROTOCOL_AUTO || s === 'model-auto' || s === 'by-model') { return CHAT_PROTOCOL_AUTO; } if ( s === CHAT_PROTOCOL_OPENAI || s === 'openai' || s === 'chat' || s === 'openai-compatible' ) { return CHAT_PROTOCOL_OPENAI; } return null; } /** * OpenAI-compatible chat completions URL. Base already ending in /chat/completions stays. * @param {string} baseURL */ export function openaiChatCompletionsUrl(baseURL) { const base = String(baseURL || '') .trim() .replace(/\/$/, ''); if (!base) return ''; if (/\/chat\/completions$/i.test(base)) return base; return `${base}/chat/completions`; } /** * Anthropic Messages API URL. Base ending in /v1 → {base}/messages (no double /v1). * Base already ending in /messages stays. Otherwise append /v1/messages. * @param {string} baseURL */ export function anthropicMessagesUrl(baseURL) { const base = String(baseURL || '') .trim() .replace(/\/$/, ''); if (!base) return ''; if (/\/messages$/i.test(base)) return base; if (/\/v1$/i.test(base)) return `${base}/messages`; return `${base}/v1/messages`; } /** * OpenAI Responses API URL (generic). Adapter may still be unsupported. * @param {string} baseURL */ export function openaiResponsesUrl(baseURL) { const base = String(baseURL || '') .trim() .replace(/\/$/, ''); if (!base) return ''; if (/\/responses$/i.test(base)) return base; if (/\/v1$/i.test(base)) return `${base}/responses`; return `${base}/v1/responses`; } /** * System One (Jev) URL — generic path rules, not a vendor allowlist. * @param {string} baseURL */ export function systemOneUrl(baseURL) { const base = String(baseURL || '') .trim() .replace(/\/$/, ''); if (!base) return ''; if (/\/systemone$/i.test(base)) return base; if (/\/v1$/i.test(base)) return `${base}/systemone`; return `${base}/v1/systemone`; } /** * Build HTTPS endpoint for chat/jev from purpose + protocol + model. * Prefer resolveChatWire from chatWireRoute.js for full routing. * @param {string} baseURL * @param {{ purpose?: string, protocol?: string, model?: string }} [opts] */ export function buildProviderEndpointUrl(baseURL, opts = {}) { const purpose = opts.purpose || 'chat'; if (purpose === 'jev') return systemOneUrl(baseURL); if (purpose === 'stt') { const base = String(baseURL || '') .trim() .replace(/\/$/, ''); if (!base) return ''; if (/\/audio\/transcriptions$/i.test(base)) return base; return `${base}/audio/transcriptions`; } const proto = normalizeChatProtocol(opts.protocol) || DEFAULT_CHAT_PROTOCOL; if (proto === CHAT_PROTOCOL_ANTHROPIC) return anthropicMessagesUrl(baseURL); if (proto === CHAT_PROTOCOL_RESPONSES) return openaiResponsesUrl(baseURL); return openaiChatCompletionsUrl(baseURL); } /** * @typedef {{ * enabled: boolean, * protocol: string, * baseURL?: string, * apiKey?: string, * path?: string, * model?: string * }} BrowserBotProviderImage * * @typedef {{ * ok: boolean, * at: number, * count: number, * error?: string * }} BrowserBotProviderProbe * * @typedef {{ * id: string, * name: string, * baseURL: string, * apiKey: string, * model: string, * createdAt: number, * purpose?: 'chat'|'image'|'stt'|'jev', * protocol?: string, * image?: BrowserBotProviderImage, * lastProbe?: BrowserBotProviderProbe * }} BrowserBotProvider */ /** Dedicated 生图 vendor (not a chat API). */ export function isImageOnlyProvider(provider) { if (!provider || typeof provider !== 'object') return false; if (provider.purpose === 'image') return true; if (provider.purpose === 'stt' || provider.purpose === 'jev') return false; if (provider.purpose === 'chat') return false; const hasChat = !!String(provider.model || '').trim() || !!String(provider.apiKey || '').trim(); return !!(provider.image && provider.image.enabled && !hasChat); } /** Dedicated speech-to-text vendor (OpenAI-compatible /audio/transcriptions). */ export function isSttOnlyProvider(provider) { return !!(provider && provider.purpose === 'stt'); } /** Selection-only vendor. Never a chat or image model. */ export function isJevProvider(provider) { return !!(provider && provider.purpose === 'jev'); } /** Appears in the inference settings list / chat picker. */ export function isChatProvider(provider) { return !!provider && !isImageOnlyProvider(provider) && !isSttOnlyProvider(provider) && !isJevProvider(provider); } /** Appears in the image settings list / 生图 picker. */ export function isImageProvider(provider) { return !!(provider?.image && provider.image.enabled); } export function listChatProviders(providers) { return (Array.isArray(providers) ? providers : []).filter(isChatProvider); } export function listImageProviders(providers) { return (Array.isArray(providers) ? providers : []).filter(isImageProvider); } export function listSttProviders(providers) { return (Array.isArray(providers) ? providers : []).filter(isSttOnlyProvider); } export function resolveActiveSttProvider(providers, activeSttProviderId) { const list = listSttProviders(providers); if (!list.length) return { activeSttProviderId: null, activeStt: null }; const want = String(activeSttProviderId || ''); const hit = want && list.find((p) => p.id === want); const activeStt = hit || list[0]; return { activeSttProviderId: activeStt.id, activeStt }; } export function listJevProviders(providers) { return (Array.isArray(providers) ? providers : []).filter(isJevProvider); } export function resolveActiveJevProvider(providers, activeJevProviderId) { const list = listJevProviders(providers); if (!list.length) return { activeJevProviderId: null, activeJev: null }; const want = String(activeJevProviderId || ''); const hit = want && list.find((p) => p.id === want); const activeJev = hit || list[0]; return { activeJevProviderId: activeJev.id, activeJev }; } export function sttVendorLabel(provider) { const base = String(provider?.baseURL || ''); if (/openai\.com/i.test(base) && !/openrouter/i.test(base)) return 'OpenAI'; if (/openrouter/i.test(base)) return 'OpenRouter'; return provider?.name || 'STT'; } export function resolveActiveImageProvider(providers, activeImageProviderId) { const list = listImageProviders(providers); if (!list.length) return { activeImageProviderId: null, activeImage: null }; const want = String(activeImageProviderId || ''); const hit = want && list.find((p) => p.id === want); const activeImage = hit || list[0]; return { activeImageProviderId: activeImage.id, activeImage }; } export function imageVendorLabel(provider) { const img = provider?.image && typeof provider.image === 'object' ? provider.image : {}; const proto = String(img.protocol || ''); const base = String(img.baseURL || provider?.baseURL || ''); if (/openrouter/i.test(proto) || /openrouter/i.test(base)) return 'OpenRouter'; if (/minimax/i.test(proto) || /minimax/i.test(base) || /minimaxi/i.test(base)) return 'MiniMax'; if (/openai/i.test(base) && !/openrouter/i.test(base)) return 'OpenAI'; if (provider?.purpose === 'image' && provider?.name) return provider.name; return provider?.name || 'Image'; } /** Card title: optional nickname, else vendor/name. Missing nickname stays compatible. */ export function providerDisplayName(provider, fallback) { const nick = typeof provider?.nickname === 'string' ? provider.nickname.trim() : ''; if (nick) return nick; return String(fallback || provider?.name || '').trim() || 'Provider'; } export function imageCardTitle(provider) { const img = provider?.image && typeof provider.image === 'object' ? provider.image : {}; const imageNick = typeof img.nickname === 'string' ? img.nickname.trim() : ''; if (imageNick) return imageNick; return providerDisplayName(provider, imageVendorLabel(provider)); } export function sttCardTitle(provider) { return providerDisplayName(provider, sttVendorLabel(provider)); } export function jevVendorLabel(provider) { const base = String(provider?.baseURL || ''); if (/opencode\.ai/i.test(base)) return 'OpenCode Zen'; if (/openrouter/i.test(base)) return 'OpenRouter'; if (/typesafe/i.test(base)) return 'TypeSafe'; if (/localhost:8000|127\.0\.0\.1:8000/i.test(base)) return 'Laya (local)'; return provider?.name || 'System1'; } export function jevCardTitle(provider) { return providerDisplayName(provider, jevVendorLabel(provider)); } function normalizeLastProbe(raw) { if (!raw || typeof raw !== 'object') return undefined; const at = Number(raw.at); const count = Number(raw.count); /** @type {BrowserBotProviderProbe} */ const out = { ok: raw.ok === true, at: Number.isFinite(at) ? at : 0, count: Number.isFinite(count) && count >= 0 ? Math.round(count) : 0 }; if (typeof raw.error === 'string' && raw.error.trim()) out.error = raw.error.trim().slice(0, 240); return out; } /** * MiniMax-style image config defaults. enabled is always false unless overridden. * @param {Partial<BrowserBotProviderImage>} [overrides] * @returns {BrowserBotProviderImage} */ export function defaultImageConfig(overrides = {}) { const o = overrides && typeof overrides === 'object' ? overrides : {}; /** @type {BrowserBotProviderImage} */ const cfg = { enabled: typeof o.enabled === 'boolean' ? o.enabled : false, protocol: typeof o.protocol === 'string' && o.protocol.trim() ? o.protocol.trim() : DEFAULT_IMAGE_PROTOCOL, path: typeof o.path === 'string' && o.path.trim() ? o.path.trim() : DEFAULT_IMAGE_PATH, model: typeof o.model === 'string' && o.model.trim() ? o.model.trim() : DEFAULT_IMAGE_MODEL }; if (typeof o.baseURL === 'string' && o.baseURL.trim()) { cfg.baseURL = o.baseURL.trim().replace(/\/$/, ''); } if (typeof o.apiKey === 'string' && o.apiKey.trim()) { cfg.apiKey = o.apiKey.trim(); } if (typeof o.nickname === 'string' && o.nickname.trim()) { cfg.nickname = o.nickname.trim(); } return cfg; } /** * Normalize optional provider.image. Returns undefined when input is absent/invalid * so providers without image config stay lean. * @param {unknown} image * @returns {BrowserBotProviderImage|undefined} */ export function normalizeImageConfig(image) { if (image == null || typeof image !== 'object') return undefined; return defaultImageConfig(/** @type {Partial<BrowserBotProviderImage>} */ (image)); } function hasChromeLocalStorage() { return typeof chrome !== 'undefined' && !!chrome.storage?.local; } function hasRuntimeBridge() { return typeof chrome !== 'undefined' && typeof chrome.runtime?.sendMessage === 'function'; } /** * chrome.storage.local is available in the SW / sidepanel, not in offscreen. * Offscreen (and any other chrome.runtime-only context) goes through the SW. */ async function storageGet(keys) { const list = Array.isArray(keys) ? keys : [keys]; if (hasChromeLocalStorage()) { return (await chrome.storage.local.get(list)) || {}; } if (hasRuntimeBridge()) { const r - src/agent/llmProxyFetch.js (5204 bytes)
/** * Shared LLM HTTPS proxy helpers for offscreen/sidepanel → SW `llm_proxy_fetch`. * No chrome.* / AI SDK imports — SW can static-import this file. */ const STRIP_HEADER = /^(user-agent|origin|referer|referrer|host|content-length|connection|keep-alive|accept-encoding|cookie|cookie2|sec-.*|proxy-.*)$/i; export function resolveLlmFetchUrl(input) { if (typeof input === 'string') return input; if (input && typeof input === 'object') { if (typeof input.url === 'string' && input.url) return input.url; if (typeof input.href === 'string' && input.href) return input.href; } return String(input || ''); } export function headerEntries(raw) { if (!raw) return []; if (typeof Headers !== 'undefined' && raw instanceof Headers) return [...raw.entries()]; if (Array.isArray(raw)) return raw.map(([k, v]) => [k, v]); if (typeof raw === 'object') return Object.entries(raw); return []; } /** Drop headers that Chrome fetch (SW or document) may reject as unsafe. */ export function sanitizeLlmProxyHeaders(raw) { const out = {}; for (const [key, value] of headerEntries(raw)) { const name = String(key || ''); if (!name || STRIP_HEADER.test(name)) continue; if (name.toLowerCase() === 'x-paw-internal-no-retry') continue; if (value == null) continue; out[name] = String(value); } return out; } export async function serializeLlmProxyBody(body) { if (body == null) return null; if (typeof body === 'string') return body; try { if (body instanceof Uint8Array) return new TextDecoder().decode(body); if (body instanceof ArrayBuffer) return new TextDecoder().decode(body); if (ArrayBuffer.isView(body)) return new TextDecoder().decode(body); } catch { /* fall through */ } if (typeof Blob !== 'undefined' && body instanceof Blob) return body.text(); if (typeof body === 'object' && typeof body.getReader === 'function') { const reader = body.getReader(); const chunks = []; for (;;) { const { done, value } = await reader.read(); if (done) break; if (value == null) continue; chunks.push(typeof value === 'string' ? new TextEncoder().encode(value) : new Uint8Array(value)); } let total = 0; for (const chunk of chunks) total += chunk.byteLength; const out = new Uint8Array(total); let offset = 0; for (const chunk of chunks) { out.set(chunk, offset); offset += chunk.byteLength; } return new TextDecoder().decode(out); } if (typeof body === 'object' && typeof body.text === 'function') return body.text(); return String(body); } export function isAllowedLlmProxyUrl(url) { if (!url || typeof url !== 'string') return { ok: false, error: 'Missing url' }; let parsed; try { parsed = new URL(url); } catch { return { ok: false, error: 'Invalid url' }; } const allowed = parsed.protocol === 'https:' || parsed.hostname === '127.0.0.1' || parsed.hostname === 'localhost'; if (!allowed) return { ok: false, error: 'Blocked non-HTTPS LLM endpoint' }; return { ok: true, url: parsed.href }; } export function reconstructLlmProxyResponse(res) { const text = res?.text != null ? res.text : res?.json != null ? JSON.stringify(res.json) : ''; const contentType = res?.contentType || res?.headers?.['content-type'] || res?.headers?.['Content-Type'] || 'application/json'; return new Response(text, { status: res?.status || (res?.ok ? 200 : 500), statusText: res?.statusText || '', headers: { 'Content-Type': contentType } }); } export function llmProxyFailure(directErr, proxyErr) { const proxyMsg = proxyErr instanceof Error ? proxyErr.message : String(proxyErr || 'proxy failed'); const err = new Error(`Background LLM proxy failed: ${proxyMsg}`); err.code = 'LLM_PROXY_FAILED'; if (proxyErr != null) err.cause = proxyErr; if (directErr != null) err.directError = directErr; return err; } /** * Privileged SW/document fetch. Caller supplies fetchImpl in tests. * @param {{ url?: string, method?: string, headers?: object, body?: string|null }} request * @param {typeof fetch} [fetchImpl] */ export async function executeLlmProxyFetch(request, fetchImpl = fetch) { const allowed = isAllowedLlmProxyUrl(request?.url); if (!allowed.ok) return { ok: false, error: allowed.error }; const methodUpper = String(request?.method || 'POST').toUpperCase(); const init = { method: methodUpper, headers: sanitizeLlmProxyHeaders(request?.headers || {}), credentials: 'omit' }; if (request?.body != null && methodUpper !== 'GET' && methodUpper !== 'HEAD') { init.body = request.body; } const res = await fetchImpl(allowed.url, init); const text = await res.text(); let json = null; try { json = JSON.parse(text); } catch { /* SSE / non-json */ } const contentType = (typeof res.headers?.get === 'function' && res.headers.get('content-type')) || (json != null ? 'application/json' : 'text/plain'); return { ok: res.ok, status: res.status, statusText: res.statusText || '', text, json, contentType, headers: { 'content-type': contentType } }; } - src/agent/mediaCapabilityChecks.js (11587 bytes)
import { loadLlmSettings } from './llm.js'; import { byokAllowedForProvider } from './byokGate.js'; import { loadCachedModelsForBase } from './modelCatalog.js'; import { resolveChatMediaCapabilities } from './chatMedia.js'; import { MEDIA_AUTO_KEY, MEDIA_CACHE_KEY, mediaStorageGet, mediaStorageSet, mediaConnectionKey, evidenceDue } from './mediaCapabilityStore.js'; import { createMediaCheckSample, scoreMediaCheck } from './mediaCheckSamples.js'; import { classifyMediaRejection } from './mediaRejection.js'; const FORMATS = { image: 'image/jpeg', audio: 'audio/wav', video: 'video/webm' }; const OFFICIAL_IMAGE_HOSTS = ['api.openai.com', 'api.deepseek.com', 'generativelanguage.googleapis.com']; export function checkTransport(caps, kind) { // Automatic calibration requires an identified profile, including for image samples. // Official Chat Completions hosts accept standard image_url without a media-wire profile. if (kind === 'image') { if (['openai-audio', 'openrouter'].includes(caps.transport)) return true; try { return OFFICIAL_IMAGE_HOSTS.includes(new URL(caps.base).hostname); } catch { return false; } } return (kind === 'audio' && ['openai-audio', 'openrouter'].includes(caps.transport)) || (kind === 'video' && caps.transport === 'openrouter'); } export async function runMediaCheck({ settings, caps, kind, signal, sampleFactory = createMediaCheckSample }) { let sample; try { sample = await sampleFactory(kind, signal); } catch (error) { throw Object.assign(new Error('SAMPLE_UNAVAILABLE'), { code: 'SAMPLE_UNAVAILABLE' }); } signal.throwIfAborted(); const { createBrowserBotLanguageModel } = await import('./provider.js'); const trialCaps = structuredClone(caps); trialCaps.inputs[kind] = { state: 'supported', source: 'check-only' }; const built = await createBrowserBotLanguageModel({ settings, mediaCapabilities: trialCaps, noRetry: true }); // doGenerate is a single SDK transport call, without generateText/ToolLoopAgent retries. const result = await built.model.doGenerate({ abortSignal: signal, maxOutputTokens: 1024, prompt: [ { role: 'user', content: [{ type: 'text', text: sample.prompt }] }, { role: 'assistant', content: [{ type: 'tool-call', toolCallId: 'media-check', toolName: 'acquire', input: { action: 'fetch' } }] }, { role: 'tool', content: [{ type: 'tool-result', toolCallId: 'media-check', toolName: 'acquire', output: { type: 'content', value: [{ type: 'text', text: '{"source":"synthetic capability sample"}' }, sample.part] } }] } ] }); return scoreMediaCheck(kind, sample, (result.content || []).filter(p => p.type === 'text').map(p => p.text).join('')); } /** One manager per offscreen runtime. Ordinary sends only call context(), never request(). */ export class MediaCapabilityChecks { constructor({ getSettings = loadLlmSettings, getModels = loadCachedModelsForBase, get = mediaStorageGet, set = mediaStorageSet, run = runMediaCheck, now = Date.now, timeoutMs = 20000, byokAllowed = byokAllowedForProvider } = {}) { Object.assign(this, { getSettings, getModels, get, set, run, now, timeoutMs, byokAllowed }); this.tail = Promise.resolve(); this.writes = Promise.resolve(); this.job = null; this.sequence = 0; } async identify(settings, models) { const caps = resolveChatMediaCapabilities(settings, models); return { caps, key: await mediaConnectionKey(settings, caps) }; } async read(settings, models) { const { key } = await this.identify(settings, models); const saved = await this.get([MEDIA_CACHE_KEY]); const evidence = saved[MEDIA_CACHE_KEY]?.[key] || {}; return { key, evidence, caps: resolveChatMediaCapabilities(settings, models, evidence, this.now()) }; } async update(key, kind, record) { const write = this.writes.then(async () => { const data = await this.get([MEDIA_CACHE_KEY]); const cache = data[MEDIA_CACHE_KEY] || {}; const previous = cache[key]?.[kind]; if (record.state !== 'supported') record = { ...record, previous: previous?.state === 'supported' ? previous : previous?.previous }; const rejections = { ...previous?.rejections }; for (const format of record.formats || []) { if (record.state === 'unsupported') rejections[format] = record.checkedAt; if (record.state === 'supported') delete rejections[format]; } record = { ...record, rejections }; cache[key] = { ...cache[key], [kind]: record }; // Bounded metadata storage, preserving the connection being updated. const entries = Object.entries(cache).sort((a, b) => Math.max(...Object.values(b[1]).map(r => r.checkedAt || 0)) - Math.max(...Object.values(a[1]).map(r => r.checkedAt || 0))); await this.set({ [MEDIA_CACHE_KEY]: Object.fromEntries(entries.slice(0, 100)) }); }); this.writes = write.catch(() => {}); return write; } async status() { const settings = await this.getSettings(); const models = (await this.getModels(settings.apiBase)).models || []; const result = await this.read(settings, models); const prefs = await this.get([MEDIA_AUTO_KEY]); return { providerId: settings.providerId, model: settings.model, base: result.caps.base, capabilities: result.caps, autoCheck: prefs[MEDIA_AUTO_KEY] !== false, checking: this.job?.key === result.key && !this.job.done ? this.job.kind || 'queued' : null }; } async request({ force = false } = {}) { const seq = ++this.sequence; const settings = await this.getSettings(); const models = (await this.getModels(settings.apiBase)).models || []; const { key, caps } = await this.read(settings, models); const prefs = await this.get([MEDIA_AUTO_KEY]); // A probe against the user's own key is BYOK use: skipped silently without a plan. const byokOk = !settings.apiKey || await this.byokAllowed('probe', settings.providerId); if (seq !== this.sequence) return { queued: false }; if (!settings.apiKey || (!force && prefs[MEDIA_AUTO_KEY] === false)) { this.job?.controller.abort(); return { queued: false }; } if (!byokOk) { this.job?.controller.abort(); return { queued: false, reason: 'BYOK_REQUIRES_PLAN' }; } const configTag = JSON.stringify(settings.chatMediaOverrides || {}); if (this.job?.key === key && this.job.configTag === configTag && !this.job.done && !this.job.controller.signal.aborted) return { queued: true }; this.job?.controller.abort(); const job = { key, configTag, controller: new AbortController(), done: false, kind: null }; this.job = job; // Work begins outside the RPC response; all jobs share a single serial lane. job.promise = this.tail.then(() => this.perform(job, settings, models, caps, force)) .catch(() => {}) // storage failures leave prior evidence intact; never break startup/settings .finally(() => { job.done = true; job.kind = null; }); this.tail = job.promise; return { queued: true }; } async stillCurrent(job) { if (job.controller.signal.aborted) return false; const settings = await this.getSettings(); const models = (await this.getModels(settings.apiBase)).models || []; return (await this.identify(settings, models)).key === job.key && JSON.stringify(settings.chatMediaOverrides || {}) === job.configTag; } async perform(job, settings, models, caps, force) { for (const kind of Object.keys(FORMATS)) { if (!(await this.stillCurrent(job))) return; const latestModels = (await this.getModels(settings.apiBase)).models || models; const { evidence, caps: fresh } = await this.read(settings, latestModels); if (!checkTransport(caps, kind) || fresh.inputs[kind].source === 'user') continue; if (!force && !evidenceDue(evidence[kind], this.now())) continue; if (!force && ['catalog', 'builtin'].includes(fresh.inputs[kind].source) && evidence[kind]?.state !== 'unsupported') continue; job.kind = kind; // Persist before dispatch: a crashed offscreen must not repeatedly charge on boot. await this.update(job.key, kind, { state: 'unknown', error: 'CHECK_INTERRUPTED', checkedAt: this.now(), formats: [FORMATS[kind]] }); const controller = new AbortController(); const abort = () => controller.abort(job.controller.signal.reason); job.controller.signal.addEventListener('abort', abort, { once: true }); if (job.controller.signal.aborted) abort(); let result, stop = false, timer; try { controller.signal.throwIfAborted(); const work = this.run({ settings, caps: fresh, kind, signal: controller.signal }); const timed = new Promise((_, reject) => { timer = setTimeout(() => { controller.abort(); reject(new Error('CHECK_TIMEOUT')); }, this.timeoutMs); }); result = await Promise.race([work, timed]); } catch (error) { const status = error?.statusCode || error?.status; const rejected = classifyMediaRejection(status, error?.responseBody || error?.message, [FORMATS[kind]]); stop = (!status && error?.code !== 'SAMPLE_UNAVAILABLE') || status === 401 || status === 403 || status === 429 || status >= 500; result = { state: rejected ? 'unsupported' : 'unknown', error: rejected ? 'FORMAT_REJECTED' : status ? `HTTP_${status}` : controller.signal.aborted ? 'CHECK_TIMEOUT' : 'CHECK_UNAVAILABLE' }; } finally { clearTimeout(timer); job.controller.signal.removeEventListener('abort', abort); } if (!(await this.stillCurrent(job))) return; await this.update(job.key, kind, { ...result, checkedAt: this.now(), formats: [FORMATS[kind]] }); if (stop) { // Cool down the rest of the batch as well, so restart does not retry a bad credential. for (const remaining of Object.keys(FORMATS).slice(Object.keys(FORMATS).indexOf(kind) + 1)) { await this.update(job.key, remaining, { state: 'unknown', error: 'CHECK_DEFERRED', checkedAt: this.now(), formats: [FORMATS[remaining]] }); } break; } } } async context(settings, models) { let first; try { first = await this.read(settings, models); } catch { first = await this.identify(settings, models); } const manager = this; let current = first.caps; return { get current() { return current; }, async refresh() { try { const latest = (await manager.getModels(settings.apiBase)).models; current = (await manager.read(settings, Array.isArray(latest) && latest.length ? latest : models)).caps; } catch { /* retain the last snapshot if storage is temporarily unavailable */ } return current; }, async wait(kind, signal) { const job = manager.job; if (current.inputs[kind]?.state !== 'unknown' || !job || job.key !== first.key || job.done || job.kind !== kind) return current; let timer, cancel; try { await Promise.race([job.promise, new Promise(resolve => { timer = setTimeout(resolve, 2000); cancel = resolve; signal?.addEventListener('abort', cancel, { once: true }); if (signal?.aborted) resolve(); })]); } finally { clearTimeout(timer); if (cancel) signal?.removeEventListener('abort', cancel); } return this.refresh(); }, async reject(mime) { await manager.update(first.key, mime.split('/')[0], { state: 'unsupported', error: 'FORMAT_REJECTED', formats: [mime], checkedAt: manager.now() }); } }; } } - src/agent/mediaCapabilityStore.js (2919 bytes)
import { MSG_TARGET_BACKGROUND } from '../brand.js'; /** Configuration-scoped evidence. No credentials or provider response bodies are stored. */ export const MEDIA_CACHE_KEY = 'browserbot_chat_media_evidence_v1'; export const MEDIA_AUTO_KEY = 'browserbot_chat_media_auto_check'; export const MEDIA_CHECK_VERSION = 1; export const DAY = 86400000; export async function mediaStorageGet(keys) { if (globalThis.chrome?.storage?.local) return (await chrome.storage.local.get(keys)) || {}; if (!globalThis.chrome?.runtime?.sendMessage) return {}; const res = await chrome.runtime.sendMessage({ target: MSG_TARGET_BACKGROUND, action: 'storage_local_get', keys }); if (!res?.ok) throw new Error('MEDIA_STORAGE_UNAVAILABLE'); return res.result || {}; } export async function mediaStorageSet(values) { if (globalThis.chrome?.storage?.local) return chrome.storage.local.set(values); const res = await chrome.runtime.sendMessage({ target: MSG_TARGET_BACKGROUND, action: 'storage_local_set', values }); if (!res?.ok) throw new Error('MEDIA_STORAGE_UNAVAILABLE'); } export async function mediaConnectionKey(settings, caps) { const bytes = new TextEncoder().encode(JSON.stringify([ MEDIA_CHECK_VERSION, settings.providerId || 'legacy', caps.base, caps.model, settings.apiKey || '', caps.transport, settings.routing || null ])); const digest = await crypto.subtle.digest('SHA-256', bytes); return Array.from(new Uint8Array(digest), b => b.toString(16).padStart(2, '0')).join(''); } export function evidenceDue(record, now = Date.now()) { if (!record) return true; const ttl = record.state === 'supported' ? 30 * DAY : record.state === 'unsupported' ? 7 * DAY : DAY; return now - record.checkedAt >= ttl; } export function mergeMediaEvidence(caps, entry, now = Date.now()) { const out = structuredClone(caps); for (const kind of ['image', 'audio', 'video']) { const record = entry?.[kind]; if (!record) continue; out.inputs[kind].lastCheck = { ...record, stale: evidenceDue(record, now) }; if (out.inputs[kind].source === 'user') continue; // A failed refresh preserves the last positive evidence; transient failures never deny input. const proof = record.state === 'supported' ? record : record.previous; if (proof?.state === 'supported') out.inputs[kind] = { ...out.inputs[kind], state: 'supported', source: 'verified', formats: proof.formats, checkedAt: proof.checkedAt, stale: evidenceDue(proof, now) }; const rejections = record.rejections || (record.state === 'unsupported' ? Object.fromEntries((record.formats || []).map(format => [format, record.checkedAt])) : {}); out.inputs[kind].rejectedFormats = Object.entries(rejections) .filter(([, checkedAt]) => now - checkedAt < 7 * DAY).map(([format]) => format); } out.videoAudio = entry?.video?.videoAudio || entry?.video?.previous?.videoAudio || 'unknown'; return out; } - src/agent/mediaCheckSamples.js (5613 bytes)
/** Local synthetic challenges; expected answers never enter the request text. */ import { pcmToWav } from './vnext/sessionWorkspace/mediaInput.js'; const random = n => crypto.getRandomValues(new Uint32Array(1))[0] % n; function tones() { const result = Array.from({ length: 8 }, () => random(2) ? 'H' : 'L'); result[0] = 'L'; result[1] = 'H'; // audible low/high calibration; remaining six are unpredictable return result.join(''); } function samples(sequence, rate = 16000) { const data = new Float32Array(sequence.length * rate / 2); for (let i = 0; i < data.length; i++) { const within = i % (rate / 2), index = Math.floor(i / (rate / 2)); if (within < rate * 0.3) { const envelope = Math.min(1, within / 160, (rate * 0.3 - within) / 160); data[i] = 0.3 * envelope * Math.sin(2 * Math.PI * (sequence[index] === 'H' ? 1000 : 300) * i / rate); } } return data; } function encode(bytes) { let result = ''; for (let i = 0; i < bytes.length; i += 8192) result += String.fromCharCode(...bytes.subarray(i, i + 8192)); return btoa(result); } const audioPrompt = 'Return the sequence of eight separated tones as H for high and L for low, in order. The first two establish low then high.'; function canvas() { const el = document.createElement('canvas'); el.width = 480; el.height = 240; return el; } function draw(el, text) { const ctx = el.getContext('2d'); ctx.fillStyle = '#ffffff'; ctx.fillRect(0, 0, el.width, el.height); ctx.fillStyle = '#000000'; ctx.font = 'bold 60px monospace'; ctx.textAlign = 'center'; ctx.textBaseline = 'middle'; ctx.fillText(text, el.width / 2, el.height / 2); } export async function createMediaCheckSample(kind, signal) { signal.throwIfAborted(); if (kind === 'image') { const alphabet = 'ABCDEFGHJKLMNPQRSTUVWXYZ23456789'; const answer = Array.from({ length: 6 }, () => alphabet[random(alphabet.length)]).join(''); const el = canvas(); draw(el, answer); const data = el.toDataURL('image/jpeg', 0.9).split(',')[1]; return { part: { type: 'file', mediaType: 'image/jpeg', data }, expected: { image: answer }, prompt: 'Read the six characters in the image. Reply only with JSON {"image":"characters"}.' }; } const audio = tones(); const pcm = samples(audio); if (kind === 'audio') return { part: { type: 'file', mediaType: 'audio/wav', data: encode(pcmToWav([pcm], 16000)) }, expected: { audio }, prompt: `${audioPrompt} Reply only with JSON {"audio":"sequence"}.` }; const letters = ['A', 'B', 'C', 'D', 'E', 'F']; for (let i = letters.length - 1; i > 0; i--) { const j = random(i + 1); [letters[i], letters[j]] = [letters[j], letters[i]]; } const el = canvas(); draw(el, letters[0]); const stream = el.captureStream(8); const ctx = new AudioContext({ sampleRate: 16000 }); let recorder, source, timer, rejectRecording; const abort = () => { clearInterval(timer); if (recorder?.state === 'recording') recorder.stop(); stream.getTracks().forEach(track => track.stop()); void ctx.close().catch(() => {}); rejectRecording?.(signal.reason); }; signal.addEventListener('abort', abort, { once: true }); try { const dest = ctx.createMediaStreamDestination(); const buffer = ctx.createBuffer(1, pcm.length, 16000); buffer.copyToChannel(pcm, 0); source = ctx.createBufferSource(); source.buffer = buffer; source.connect(dest); dest.stream.getAudioTracks().forEach(track => stream.addTrack(track)); await ctx.resume(); signal.throwIfAborted(); const mimeType = 'video/webm;codecs=vp8,opus'; if (!MediaRecorder.isTypeSupported(mimeType)) throw new Error('SAMPLE_CODEC_UNAVAILABLE'); recorder = new MediaRecorder(stream, { mimeType, videoBitsPerSecond: 200000 }); const bytes = await new Promise((resolve, reject) => { rejectRecording = reject; const chunks = []; recorder.ondataavailable = event => { if (event.data.size) chunks.push(event.data); }; recorder.onerror = () => reject(new Error('SAMPLE_RECORDING_FAILED')); recorder.onstop = () => new Blob(chunks).arrayBuffer().then(data => resolve(new Uint8Array(data)), reject); recorder.start(); source.start(); let frame = 0; timer = setInterval(() => { frame++; if (frame === letters.length) { clearInterval(timer); recorder.stop(); } else draw(el, letters[frame]); }, 700); }); signal.throwIfAborted(); return { part: { type: 'file', mediaType: 'video/webm', data: encode(bytes) }, expected: { video: letters.join(''), audio }, prompt: `Read all six letters in the video in temporal order. For its soundtrack: ${audioPrompt} Reply only with JSON {"video":"letters","audio":"sequence"}.` }; } finally { clearInterval(timer); signal.removeEventListener('abort', abort); if (recorder?.state === 'recording') recorder.stop(); try { source?.stop(); } catch { /* already stopped */ } stream.getTracks().forEach(track => track.stop()); if (ctx.state !== 'closed') await ctx.close(); } } export function scoreMediaCheck(kind, sample, text) { let result; try { result = JSON.parse(String(text).trim().replace(/^```(?:json)?\s*|\s*```$/g, '')); } catch { return { state: 'unknown', error: 'ANSWER_MISMATCH' }; } const match = key => typeof result?.[key] === 'string' && result[key].trim().toUpperCase() === sample.expected[key]; return { state: match(kind) ? 'supported' : 'unknown', ...(match(kind) ? {} : { error: 'ANSWER_MISMATCH' }), ...(kind === 'video' ? { videoAudio: match('audio') ? 'verified' : 'unknown' } : {}) }; } - src/agent/mediaRejection.js (1501 bytes)
/** Only explicit input-format rejection is evidence. Do not persist upstream bodies. */ export function classifyMediaRejection(status, text, formats) { if (![400, 415, 422].includes(status)) return null; const message = String(text || '').toLowerCase(); if (/size|too large|duration|length|corrupt|decode|invalid.*(?:base64|url)|quota|billing/.test(message)) return null; const unsupported = /not support|unsupported|not allowed|does not accept|only supports/.test(message); if (!unsupported) return null; const matched = formats.filter(mime => message.includes(mime) || (mime.startsWith('audio/') && /audio|input_audio/.test(message)) || (mime.startsWith('video/') && /video|video_url/.test(message)) || (mime.startsWith('image/') && /image|vision/.test(message))); return matched.length === 1 ? matched[0] : null; } export function requestMediaFormats(init) { try { const body = JSON.parse(init.body); const formats = new Set(); for (const message of body.messages || []) for (const part of Array.isArray(message.content) ? message.content : []) { if (part.type === 'input_audio') formats.add(part.input_audio?.format === 'wav' ? 'audio/wav' : 'audio/mpeg'); if (part.type === 'video_url' || part.type === 'image_url') { const url = (part.video_url || part.image_url)?.url || ''; const mime = /^data:([^;,]+)/.exec(url)?.[1]; if (mime) formats.add(mime); } } return [...formats]; } catch { return []; } } - src/agent/modelCatalog.js (46343 bytes)
/** * BrowserBot model catalog (IG-6) — OpenAI-compatible GET /models + shrink UI helpers. * * Purpose: aggregators (e.g. OpenRouter) may return 200+ models. Never dump the full * list as the default settings view — use Recommended / Recent / Favorites + search * with a display cap. * * chrome.storage.local keys (documented): * browserbot_model_catalog_cache — { [hostKey]: { baseURL, models: ModelEntry[], fetchedAt } } * browserbot_model_recent — string[] of chat model ids (most-recent first) * browserbot_model_favorites — string[] of favorited chat model ids * * Image models: static MiniMax list only + always-available manual entry (not fetched). * * Never log or export API keys. */ import { CLOUD_MODELS } from './cloudModels.js'; import { BRAND_NAME } from '../brand.js'; /** * @typedef {{ * supported: boolean, * efforts: string[]|null, * defaultEffort: string|null, * defaultEnabled: boolean, * mandatory: boolean * }} ModelReasoningInfo * * @typedef {{ id: string, name?: string, image?: boolean, inputModalities?: string[], inputModalitiesSource?: 'architecture.input_modalities'|'input_modalities'|'inference_metadata.request_modality', reasoning?: ModelReasoningInfo, contextWindow?: number }} ModelEntry */ /** Gateway effort order (highest first), matching OpenRouter reasoning.effort. */ export const GATEWAY_REASONING_EFFORTS = ['max', 'xhigh', 'high', 'medium', 'low', 'minimal']; const EFFORT_SET = new Set(GATEWAY_REASONING_EFFORTS); /** Low → high rank for slider layout. */ export const REASONING_EFFORT_RANK = { minimal: 0, low: 1, medium: 2, high: 3, xhigh: 4, max: 5 }; /** * Collapse a model's advertised efforts into a 3- or 5-stop slider (low → high). * 6-stop gateway lists drop to 5 (keep min/low/mid/high/max). * @param {string[]|null|undefined} levels * @returns {string[]|null} */ export function normalizeEffortSteps(levels) { if (!Array.isArray(levels) || !levels.length) return null; const uniq = [...new Set(levels.map((e) => String(e || '').toLowerCase()))].filter((e) => EFFORT_SET.has(e) ); uniq.sort((a, b) => REASONING_EFFORT_RANK[a] - REASONING_EFFORT_RANK[b]); if (!uniq.length) return null; if (uniq.length <= 5) return uniq; const prefer = ['minimal', 'low', 'medium', 'high', 'max']; const picked = prefer.filter((e) => uniq.includes(e)); if (picked.length >= 3) return picked.slice(0, 5); const last = uniq.length - 1; return [...new Set([0, Math.round(last * 0.25), Math.round(last * 0.5), Math.round(last * 0.75), last].map((i) => uniq[i]))]; } /** * Parse OpenRouter (or compatible) per-model reasoning metadata. * @param {object} [raw] * @returns {ModelReasoningInfo|null} */ export function parseModelReasoning(raw) { if (!raw || typeof raw !== 'object') return null; const r = /** @type {any} */ (raw).reasoning; const params = Array.isArray(/** @type {any} */ (raw).supported_parameters) ? /** @type {any} */ (raw).supported_parameters : []; const paramHint = params.some((p) => String(p).toLowerCase() === 'reasoning'); if (!r || typeof r !== 'object') { if (!paramHint) return null; return { supported: true, efforts: null, defaultEffort: null, defaultEnabled: false, mandatory: false }; } let efforts = null; if (Array.isArray(r.supported_efforts)) { efforts = r.supported_efforts .map((e) => String(e || '').toLowerCase()) .filter((e) => EFFORT_SET.has(e)); } else if (r.supported_efforts === null) { efforts = GATEWAY_REASONING_EFFORTS.slice(); } const def = String(r.default_effort || '').toLowerCase(); return { supported: true, efforts, defaultEffort: EFFORT_SET.has(def) ? def : null, defaultEnabled: r.default_enabled === true, mandatory: r.mandatory === true }; } /** * @param {ModelEntry[]} models * @param {string} modelId * @returns {ModelEntry|null} */ export function findCatalogModel(models, modelId) { const want = String(modelId || '').trim(); if (!want || !Array.isArray(models)) return null; const exact = models.find((m) => m.id === want); if (exact) return exact; const lower = want.toLowerCase(); const ci = models.find((m) => String(m.id || '').toLowerCase() === lower); if (ci) return ci; const tail = models.find((m) => { const id = String(m.id || '').toLowerCase(); return id.endsWith(`/${lower}`) || lower.endsWith(`/${id}`); }); return tail || null; } /** * Effort chips for a model. null = do not show a selector (unknown or unsupported). * @param {ModelReasoningInfo|null|undefined} info * @returns {string[]|null} */ /** Compact-math fallback when override / catalog / probe / family yield no window. UI must not paint this as measured. */ export const DEFAULT_CONTEXT_WINDOW = 256000; /** * Read advertised context window from OpenRouter / OpenAI-compatible /models rows. * @param {object} [raw] * @returns {number|null} */ export function parseContextWindow(raw) { if (!raw || typeof raw !== 'object') return null; const o = /** @type {any} */ (raw); const top = o.top_provider && typeof o.top_provider === 'object' ? o.top_provider : {}; const modelInfo = o.model_info && typeof o.model_info === 'object' ? o.model_info : {}; const candidates = [ o.context_length, o.contextLength, o.context_window, o.max_context_length, o.max_input_tokens, top.context_length, top.max_input_tokens, // Alibaba Cloud Model Studio / DashScope GET /api/v1/models. modelInfo.context_window, modelInfo.max_input_tokens ]; for (const c of candidates) { const n = Number(c); if (Number.isFinite(n) && n >= 2048) return Math.round(n); } return null; } function familyContextWindow(modelId) { const s = String(modelId || '').toLowerCase(); if (!s) return null; if (s.includes('gemini')) return 1_000_000; if (s.includes('claude')) return 200_000; if (s.includes('grok')) return 131_072; if (/gpt-4\.1|o3|o4-mini/.test(s)) return 1_000_000; // DeepSeek /models does not advertise a window; V3-era 128k is not a // measurement of current Flash (1M). Leave unknown so UI/probe do not // present a family guess as a fact. if (s.includes('gpt-4o') || s.includes('gpt-4')) return 128_000; return null; } /** * Last-resort guess from model id when the catalog has no window. * Unknown ids use the compacting default; UI should call resolveContextWindow(..., { allowDefault: false }). * @param {string} [modelId] * @returns {number} */ export function guessContextWindow(modelId) { return familyContextWindow(modelId) ?? DEFAULT_CONTEXT_WINDOW; } /** * Adaptive window: user override → measured evidence → catalog row → family → * optional 256k compacting default. UI must pass { allowDefault: false }. * Never treat an 8k output cap as a window; callers only pass adopted ≥2048 values. * @param {string} [modelId] * @param {ModelEntry|null|undefined} [entry] * @param {{ * allowDefault?: boolean, * evidence?: { window?: number|null }|null, * override?: number|null, * overrides?: Record<string, number>, * baseURL?: string * }} [opts] * @returns {number|null} */ export function resolveContextWindow(modelId, entry, opts = {}) { let override = Number(opts.override); if (!Number.isFinite(override) && opts.overrides && opts.baseURL != null) { const key = `${normalizeModelsBaseURL(opts.baseURL)}\n${String(modelId || entry?.id || '').trim()}`; override = Number(opts.overrides[key]); } if (Number.isFinite(override) && override >= 2048) return Math.round(override); const measured = Number(opts.evidence?.window); if (Number.isFinite(measured) && measured >= 2048) return Math.round(measured); const advertised = Number(entry?.contextWindow); if (Number.isFinite(advertised) && advertised >= 2048) return Math.round(advertised); const family = familyContextWindow(modelId || entry?.id); if (family) return family; return opts.allowDefault === false ? null : DEFAULT_CONTEXT_WINDOW; } /** GET /models 404/405: vendor has no catalog. Not an auth or server failure. */ export function isMissingModelsCatalogError(error) { const status = Number(error?.status); if (status === 404 || status === 405) return true; return /\bHTTP\s*40[45]\b/.test(String(error?.message || error || '')); } /** * Strip aggregator/provider prefix (`x-ai/`, `openai/`, `anthropic/`, …) and lowercase. * @param {string} [modelId] * @returns {string} */ export function normalizeChatModelId(modelId) { const raw = String(modelId || '').toLowerCase().trim(); if (!raw) return ''; const slash = raw.lastIndexOf('/'); return slash >= 0 ? raw.slice(slash + 1) : raw; } /** * Explicit vision tokens on the *leaf* id (after prefix strip). * `revision` / `television` must not count. * @param {string} id */ function hasExplicitVisionHint(id) { const s = String(id || '').toLowerCase(); if (!s) return false; if (/(?:^|[-_.:])vl\d*(?:[-_.:]|$)/.test(s)) return true; return /(?:^|[-_./:])vision(?:[-_./:]|$)/.test(s); } /** * Known text-only chat families (leaf id). Vision exceptions (`vl` / `vision`) win. * Unknown ids are NOT listed — BYOK defaults permissive. * Official DeepSeek Pro has no vision. Do not blanket-match `deepseek-*`: * current Flash / chat aliases accept image_url on api.deepseek.com. */ const TEXT_ONLY_CHAT_PATTERNS = [ /^deepseek-v4-pro(?:[-_.]|$)/, /^o1-mini(?:[-_.]|$)/, /^o3-mini(?:[-_.]|$)/, /^gpt-3\.5/, /^text-davinci/, /^claude-instant/, /^claude-2(?:[-_.]|$)/ ]; /** * Known multimodal / vision chat families (leaf id). Informational — unknown is still capable. */ const VISION_CHAT_PATTERNS = [ /^deepseek-(?:v\d+(?:\.\d+)?-)?flash(?:[-_.]|$)/, /^deepseek-chat(?:[-_.]|$)/, /^grok-[34](?:[.\-]|$)/, /^grok-2-vision/, /^gpt-4o/, /^gpt-4\.1/, /^gpt-4-turbo/, /^gpt-4-vision/, /^gpt-5/, /^claude-3/, /^claude-(?:sonnet|opus|haiku)/, /^gemini/, /^qwen(?:\d+(?:\.\d+)?)?-?vl/, /^qwen-vl/, /^glm-4v/, /^doubao(?:[-_].*)?(?:vision|vl)/, /llava/, /(?:^|[-_.:])vl\d*(?:[-_.:]|$)/, /(?:^|[-_./:])vision(?:[-_./:]|$)/ ]; /** * @param {string} [modelId] * @returns {boolean} */ export function isKnownTextOnlyChatModel(modelId) { const id = normalizeChatModelId(modelId); if (!id) return false; if (hasExplicitVisionHint(id)) return false; return TEXT_ONLY_CHAT_PATTERNS.some((re) => re.test(id)); } /** * @param {string} [modelId] * @returns {boolean} */ export function isKnownVisionChatModel(modelId) { const id = normalizeChatModelId(modelId); if (!id) return false; if (isKnownTextOnlyChatModel(id)) return false; if (hasExplicitVisionHint(id)) return true; return VISION_CHAT_PATTERNS.some((re) => re.test(id)); } /** * `'vision' | 'text-only' | 'unknown'` * Empty / unset id is `unknown` (BYOK: do not block). * @param {string} [modelId] * @returns {'vision'|'text-only'|'unknown'} */ export function classifyChatVisionCapability(modelId) { const id = normalizeChatModelId(modelId); if (!id) return 'unknown'; if (isKnownTextOnlyChatModel(id)) return 'text-only'; if (isKnownVisionChatModel(id)) return 'vision'; return 'unknown'; } /** * Whether the host should treat this chat model as able to receive image parts. * Known text-only families → false. Known vision *and unknown* → true (permissive). * A stale allow-list must not block a BYOK send; provider errors surface at runtime. * @param {string} [modelId] * @returns {boolean} */ export function isVisionCapableModel(modelId) { return classifyChatVisionCapability(modelId) !== 'text-only'; } export function effortLevelsForReasoning(info) { const core = info && info.supported ? normalizeEffortSteps(info.efforts) : ['low', 'medium', 'high']; const steps = core && core.length ? core : ['low', 'medium', 'high']; const gears = steps.filter((e) => e !== 'none'); if (info?.mandatory) return gears.length ? gears : ['medium']; return ['none', ...gears]; } /** chrome.storage.local keys */ export const MODEL_CATALOG_CACHE_KEY = 'browserbot_model_catalog_cache'; export const MODEL_RECENT_KEY = 'browserbot_model_recent'; export const MODEL_FAVORITES_KEY = 'browserbot_model_favorites'; /** Initial list display cap (search can narrow; still capped for DOM). */ export const DEFAULT_DISPLAY_CAP = 30; export const MAX_SEARCH_RESULTS = 50; export const MAX_RECENT = 12; export const MAX_FAVORITES = 40; export const MAX_CACHE_HOSTS = 12; /** Soft cap on models kept in cache per host (search still works offline on cache). */ export const MAX_CACHED_MODELS_PER_HOST = 500; /** * Hardcoded recommended chat models (small; shown before any fetch). * Not vendor-locked — free-text model always wins. * @type {ModelEntry[]} */ export const RECOMMENDED_CHAT_MODELS = [ { id: 'deepseek-v4-flash', name: 'DeepSeek V4 Flash' }, { id: 'deepseek-v4-pro', name: 'DeepSeek V4 Pro' }, { id: 'gpt-4o', name: 'GPT-4o' }, { id: 'gpt-4.1', name: 'GPT-4.1' }, { id: 'gpt-4o-mini', name: 'GPT-4o mini' }, { id: 'openai/gpt-4o', name: 'OpenRouter · GPT-4o' }, { id: 'anthropic/claude-sonnet-4', name: 'OpenRouter · Claude Sonnet' }, { id: 'MiniMax-Text-01', name: 'MiniMax Text-01' }, { id: 'gemini-2.5-flash', name: 'Gemini 2.5 Flash' } ]; /** * Static image models for protocol minimax-image (manual entry always available). * @type {ModelEntry[]} */ export const STATIC_IMAGE_MODELS_MINIMAX = [ { id: 'image-01', name: 'image-01 (MiniMax t2i/i2i)' } ]; export const DEFAULT_IMAGE_MODEL_ID = 'image-01'; // ── pure helpers ──────────────────────────────────────────────────────────── /** * @param {string} baseURL * @returns {string} */ export function normalizeModelsBaseURL(baseURL) { if (!baseURL || typeof baseURL !== 'string') return ''; return baseURL.trim().replace(/\/+$/, ''); } /** * Host key for cache map (no secrets). * @param {string} baseURL * @returns {string} */ export function catalogHostKey(baseURL) { const base = normalizeModelsBaseURL(baseURL); if (!base) return ''; try { const u = new URL(base.includes('://') ? base : `https://${base}`); return (u.host + u.pathname).replace(/\/+$/, '') || u.host; } catch { return base.replace(/^https?:\/\//i, '').slice(0, 120); } } /** * Build GET /models URL from chat base. * @param {string} baseURL * @returns {string} */ export function modelsEndpointURL(baseURL) { const base = normalizeModelsBaseURL(baseURL); if (!base) throw new Error('baseURL required'); // DashScope exposes OpenAI-compatible chat at /compatible-mode/v1, but its // documented rich model catalog lives at /api/v1/models on the same regional // host. This is an endpoint adapter only: authentication stays Bearer and // no vendor-specific request stack is introduced. try { const url = new URL(base); const isDashScopeHost = /(?:^|\.)(?:dashscope(?:-intl)?|[a-z0-9-]+\.(?:cn-[a- - src/agent/modelReachProbe.js (9283 bytes)
/** * Settings form: probe whether the configured model answers a tiny chat turn. * Uses resolveChatWire (same protocol-first routing as real send). Not GET /models. * Jev / purpose:jev stays on systemone (see jevSelect.probeJevSystemOne). */ import { resolveChatWire } from './chatWireRoute.js'; import { anthropicRequestHeaders } from './anthropicProvider.js'; import { CHAT_PROTOCOL_ANTHROPIC, CHAT_PROTOCOL_OPENAI } from './llm.js'; export const MODEL_REACH_PROBE_PROMPT = 'ping'; export const MODEL_REACH_PROBE_TIMEOUT_MS = 18000; export const MODEL_REACH_PROBE_MAX_TOKENS = 1; /** * @param {string} msg * @param {string} [apiKey] */ export function sanitizeProbeErrorMessage(msg, apiKey = '') { let s = String(msg || '').trim() || 'probe failed'; const key = String(apiKey || '').trim(); if (key && key.length >= 8) { s = s.split(key).join('[redacted]'); } s = s.replace(/\bsk-[A-Za-z0-9_\-]{8,}\b/g, '[redacted]'); s = s.replace(/\bBearer\s+\S+/gi, 'Bearer [redacted]'); if (s.length > 220) s = `${s.slice(0, 217)}…`; return s; } /** * @param {unknown} body * @param {string} [text] */ function shortVendorError(body, text) { if (body && typeof body === 'object') { const err = /** @type {any} */ (body).error; if (typeof err === 'string' && err.trim()) return err.trim(); if (err && typeof err === 'object') { const m = err.message || err.msg || err.type; if (typeof m === 'string' && m.trim()) return m.trim(); } const top = /** @type {any} */ (body).message || /** @type {any} */ (body).detail || /** @type {any} */ (body).error_description; if (typeof top === 'string' && top.trim()) return top.trim(); } const t = String(text || '').trim(); if (t && t.length < 180 && !t.startsWith('{') && !t.startsWith('<')) return t; return ''; } /** * Build POST request for a one-token model reachability check. * Does not send; never invents a default model id. * * @param {{ * baseURL?: string, * apiKey?: string, * model?: string, * protocol?: string|null, * purpose?: string * }} input * @returns {{ * ok: true, * wire: string, * protocol: string|null, * url: string, * method: 'POST', * headers: Record<string, string>, * body: Record<string, unknown>, * bodyJson: string * } | { * ok: false, * code: string, * error: string, * missing?: 'baseURL'|'apiKey'|'model', * wire?: string, * url?: string * }} */ export function buildModelReachProbeRequest(input = {}) { const baseURL = String(input.baseURL || '').trim().replace(/\/$/, ''); const apiKey = String(input.apiKey || '').trim(); const model = String(input.model || '').trim(); if (!baseURL) { return { ok: false, code: 'NEED_BASE', error: 'Base URL is required.', missing: 'baseURL' }; } if (!apiKey) { return { ok: false, code: 'NEED_KEY', error: 'API key is required.', missing: 'apiKey' }; } if (!model) { return { ok: false, code: 'NEED_MODEL', error: 'Model id is required.', missing: 'model' }; } if (input.purpose === 'jev') { return { ok: false, code: 'CHAT_WIRE_JEV', error: 'System1 providers use System One probe, not chat completions.' }; } const route = resolveChatWire({ baseURL, model, protocol: input.protocol, purpose: input.purpose }); if (!route.supported) { return { ok: false, code: route.code || 'CHAT_WIRE_UNSUPPORTED', error: String(route.error || 'Unsupported chat wire'), wire: route.wire, url: route.url || '' }; } if (route.wire === CHAT_PROTOCOL_ANTHROPIC) { const body = { model, max_tokens: MODEL_REACH_PROBE_MAX_TOKENS, messages: [ { role: 'user', content: [{ type: 'text', text: MODEL_REACH_PROBE_PROMPT }] } ] }; return { ok: true, wire: CHAT_PROTOCOL_ANTHROPIC, protocol: CHAT_PROTOCOL_ANTHROPIC, url: route.url, method: 'POST', headers: anthropicRequestHeaders(apiKey), body, bodyJson: JSON.stringify(body) }; } // openai-compatible (explicit, auto, host-default) const body = { model, messages: [{ role: 'user', content: MODEL_REACH_PROBE_PROMPT }], max_tokens: MODEL_REACH_PROBE_MAX_TOKENS, stream: false }; /** @type {Record<string, string>} */ const headers = { Authorization: `Bearer ${apiKey}`, 'Content-Type': 'application/json' }; if (baseURL.includes('openrouter.ai')) { headers['HTTP-Referer'] = 'https://browserbot.local'; headers['X-Title'] = 'Browser Kitten'; } return { ok: true, wire: CHAT_PROTOCOL_OPENAI, protocol: CHAT_PROTOCOL_OPENAI, url: route.url, method: 'POST', headers, body, bodyJson: JSON.stringify(body) }; } /** * HTTP 2xx + a parseable assistant/message envelope (content may be empty). * @param {{ wire?: string, status?: number, body?: unknown, text?: string }} input */ export function interpretModelReachProbeResponse(input = {}) { const status = Number(input.status) || 0; if (status < 200 || status >= 300) { const detail = shortVendorError(input.body, input.text); return { ok: false, code: status ? `HTTP_${status}` : 'HTTP_ERROR', error: sanitizeProbeErrorMessage(detail ? `HTTP ${status}: ${detail}` : `HTTP ${status || 0}`) }; } let body = input.body; if (body == null && typeof input.text === 'string' && input.text.trim()) { try { body = JSON.parse(input.text); } catch { return { ok: false, code: 'BAD_RESPONSE', error: 'Response was not JSON.' }; } } if (!body || typeof body !== 'object') { return { ok: false, code: 'BAD_RESPONSE', error: 'Empty or non-object response.' }; } const wire = String(input.wire || ''); if (wire === CHAT_PROTOCOL_ANTHROPIC || /** @type {any} */ (body).type === 'message') { if (/** @type {any} */ (body).type === 'message' || Array.isArray(/** @type {any} */ (body).content)) { return { ok: true }; } } const choices = /** @type {any} */ (body).choices; if (Array.isArray(choices) && choices.length > 0) { const first = choices[0]; if ( first && typeof first === 'object' && (first.message != null || first.delta != null || first.text != null || first.finish_reason != null) ) { return { ok: true }; } // Some gateways return choices:[{}] with only index — still a completions envelope. if (first && typeof first === 'object') return { ok: true }; } // OpenAI-ish root fields without choices (rare) — require message-like shape. if (/** @type {any} */ (body).message && typeof /** @type {any} */ (body).message === 'object') { return { ok: true }; } return { ok: false, code: 'BAD_RESPONSE', error: 'HTTP ok but no assistant/message payload.' }; } /** * @param {{ * baseURL?: string, * apiKey?: string, * model?: string, * protocol?: string|null, * purpose?: string * }} config * @param {typeof fetch} [fetchImpl] * @param {{ timeoutMs?: number }} [opts] */ export async function probeModelReach(config, fetchImpl = fetch, opts = {}) { const built = buildModelReachProbeRequest(config); if (!built.ok) { return { ok: false, code: built.code, error: built.error, missing: built.missing, wire: built.wire, url: built.url || '' }; } const timeoutMs = typeof opts.timeoutMs === 'number' && opts.timeoutMs > 0 ? opts.timeoutMs : MODEL_REACH_PROBE_TIMEOUT_MS; const ac = new AbortController(); const timer = setTimeout(() => ac.abort(), timeoutMs); let response; try { response = await fetchImpl(built.url, { method: built.method, headers: built.headers, body: built.bodyJson, signal: ac.signal }); } catch (err) { clearTimeout(timer); const aborted = err?.name === 'AbortError' || /aborted|timeout/i.test(String(err?.message || err || '')); if (aborted) { return { ok: false, code: 'TIMEOUT', error: `Probe timed out after ${Math.round(timeoutMs / 1000)}s.`, wire: built.wire, url: built.url }; } return { ok: false, code: 'NETWORK', error: sanitizeProbeErrorMessage( err instanceof Error ? err.message : String(err), config?.apiKey ), wire: built.wire, url: built.url }; } clearTimeout(timer); let text = ''; let body = null; try { text = await response.text(); } catch { text = ''; } if (text) { try { body = JSON.parse(text); } catch { body = null; } } const interpreted = interpretModelReachProbeResponse({ wire: built.wire, status: response.status, body, text }); if (!interpreted.ok) { return { ...interpreted, error: sanitizeProbeErrorMessage(interpreted.error, config?.apiKey), wire: built.wire, url: built.url, status: response.status }; } return { ok: true, wire: built.wire, url: built.url, status: response.status }; } - src/agent/provider.js (18589 bytes)
/** * BrowserBot — OpenAI-compatible or Anthropic Messages language model via Vercel AI SDK * BYOK from active provider in chrome.storage; HTTPS inference only (no cloud agent orchestrator). */ import { BK_CLIENT_HEADER, BK_PURPOSE_HEADER, bkClientHeaderValue, isCloudApiUrl } from './cloudModels.js'; import { createOpenAICompatible } from './vnext/adapters/vendor/ai-sdk-loader.mjs'; import { loadLlmSettings, DEFAULT_BASE, CHAT_PROTOCOL_ANTHROPIC, CHAT_PROTOCOL_OPENAI } from './llm.js'; import { resolveChatWire } from './chatWireRoute.js'; import { createAnthropicLanguageModel } from './anthropicProvider.js'; import { loadCachedModelsForBase } from './modelCatalog.js'; import { resolveChatMediaCapabilities, canSendChatMedia, MAX_CHAT_MEDIA_BYTES } from './chatMedia.js'; import { isAbortLike, toAbortError } from './vnext/host/userStop.js'; import { classifyMediaRejection, requestMediaFormats } from './mediaRejection.js'; import { llmProxyFailure, reconstructLlmProxyResponse, resolveLlmFetchUrl, sanitizeLlmProxyHeaders, serializeLlmProxyBody } from './llmProxyFetch.js'; const DEFAULT_MODEL_ID = 'deepseek-v4-flash'; function resolveModelName(raw) { const s = String(raw || '') .trim() .replace(/\s+/g, '-'); return s || DEFAULT_MODEL_ID; } /** * Fields many OpenAI-compatible proxies (Groq, etc.) reject on re-sent history. * AI SDK may re-inject reasoning as reasoning_content during multi-step tool loops. */ const STRIP_MESSAGE_KEYS = new Set([ 'reasoning_content', 'reasoning', 'reasoning_details', 'providerMetadata', 'provider_metadata', 'providerOptions', 'provider_options' ]); /** * Strip non-standard / rejected fields from an OpenAI-style chat message. * Keeps role, content, name, tool_calls, tool_call_id, function_call. * @param {object} msg * @returns {object} */ export function sanitizeOpenAiMessage(msg, opts = {}) { if (!msg || typeof msg !== 'object') return msg; const keepReasoning = opts.keepReasoning === true; const out = {}; for (const [k, v] of Object.entries(msg)) { if (!keepReasoning && STRIP_MESSAGE_KEYS.has(k)) continue; if (keepReasoning && (k === 'providerMetadata' || k === 'provider_metadata' || k === 'providerOptions' || k === 'provider_options')) { continue; } out[k] = v; } if (Array.isArray(out.content)) { out.content = out.content .filter((part) => { if (!part || typeof part !== 'object') return true; const t = part.type; if (keepReasoning) return t !== 'reasoning-delta'; return t !== 'reasoning' && t !== 'reasoning-delta'; }) .map((part) => { if (!part || typeof part !== 'object') return part; const cleaned = { ...part }; delete cleaned.providerMetadata; delete cleaned.providerOptions; delete cleaned.provider_metadata; delete cleaned.provider_options; return cleaned; }); } return out; } function filePartToDataUrl(part) { if (!part || typeof part !== 'object') return null; const mediaType = String(part.mediaType || part.mimeType || 'image/jpeg').split(';')[0].trim(); if (!mediaType.startsWith('image/')) return null; const raw = (typeof part.data === 'object' && part.data && part.data.data != null ? part.data.data : null) ?? part.data ?? part.image; if (typeof raw !== 'string' || raw.length < 32) return null; if (raw.startsWith('data:image/')) return raw; return `data:${mediaType};base64,${raw}`; } function extractImagesFromToolContent(content, capabilities) { const images = []; const texts = []; if (typeof content !== 'string') { return { content, images }; } let parsed; try { parsed = JSON.parse(content); } catch { return { content, images }; } if (!Array.isArray(parsed)) { return { content, images }; } let lifted = false; for (const p of parsed) { if (!p || typeof p !== 'object') continue; if (p.type === 'text' && p.text) { texts.push(String(p.text)); continue; } if (p.type === 'file' || p.type === 'file-data' || p.type === 'image') { const mime = String(p.mediaType || p.mimeType || '').split(';')[0].trim().toLowerCase(); if (/^(audio|video)\//.test(mime)) { lifted = true; // Never leave rejected media base64 in tool text. const raw = (typeof p.data === 'object' ? p.data?.data : p.data) || ''; const data = typeof raw === 'string' ? raw.replace(/^data:[^,]+,/, '') : ''; if (data && data.length <= Math.ceil(MAX_CHAT_MEDIA_BYTES / 3) * 4 && canSendChatMedia(capabilities, mime)) { images.push(mime.startsWith('audio/') ? { type: 'input_audio', input_audio: { data, format: mime === 'audio/wav' ? 'wav' : 'mp3' } } : { type: 'video_url', video_url: { url: `data:${mime};base64,${data}` } }); } else texts.push('[Media omitted: current chat model/transport/size does not permit this input. Use the saved file with STT; no media was perceived.]'); continue; } const url = filePartToDataUrl(p); if (url) { if (canSendChatMedia(capabilities, mime || 'image/jpeg')) images.push({ type: 'image_url', image_url: { url } }); else texts.push('[Image omitted: current model declares image input unsupported.]'); lifted = true; } } } if (!lifted) return { content, images: [] }; return { content: texts.join('\n') || '{"ok":true,"hasMedia":true}', images, changed: lifted }; } /** * OpenAI-compatible chat maps tool `content` file parts to JSON.stringify(value). * Gemini/OpenRouter then treat that base64 as *text*, not pixels (~350k tokens/photo * and the model hallucinates). Lift images onto a user message after the tool batch. * @param {object[]} messages * @returns {object[]} */ export function liftToolResultImages(messages, capabilities) { if (!Array.isArray(messages) || messages.length === 0) return messages; const out = []; let pending = []; const flush = () => { if (!pending.length) return; out.push({ role: 'user', content: [ { type: 'text', text: 'Media from the preceding tool results. Analyze the attached content; follow each result’s source and scope.' }, ...pending ] }); pending = []; }; for (const msg of messages) { if (msg && msg.role === 'tool') { const extracted = extractImagesFromToolContent(msg.content, capabilities); out.push(extracted.changed ? { ...msg, content: extracted.content } : msg); if (extracted.images.length) pending.push(...extracted.images); continue; } flush(); out.push(msg); } flush(); return out; } /** * Parse JSON chat body and strip unsupported message fields before HTTPS send. * Safe no-op for non-JSON / non-chat bodies. * @param {RequestInit} [init] * @returns {RequestInit} */ function decodeRequestBody(raw) { if (raw == null) return null; if (typeof raw === 'string') return raw; try { if (raw instanceof Uint8Array) return new TextDecoder().decode(raw); if (raw instanceof ArrayBuffer) return new TextDecoder().decode(raw); if (ArrayBuffer.isView(raw)) { return new TextDecoder().decode(raw); } } catch { return null; } return null; } export function stripUnsupportedFieldsFromRequestInit(init = {}, reasoning = null, opts = {}) { if (!init || init.body == null) return init; const raw = decodeRequestBody(init.body); if (raw == null) return init; try { const body = JSON.parse(raw); if (!body || typeof body !== 'object' || !Array.isArray(body.messages)) { return init; } const keepReasoning = opts.keepReasoning === true; body.messages = liftToolResultImages( body.messages.map((m) => sanitizeOpenAiMessage(m, { keepReasoning })), opts.mediaCapabilities?.model === body.model ? opts.mediaCapabilities : undefined ); const cfg = openRouterReasoningBody(reasoning); if (cfg) body.reasoning = cfg; // Browser Kitten decides the default per route, so "off" must be said, not implied by omission. else if (opts.explicitOff && reasoning && reasoning.enabled === false) body.reasoning = { effort: 'none' }; return { ...init, body: JSON.stringify(body) }; } catch { return init; } } const REASONING_EFFORTS = new Set(['minimal', 'low', 'medium', 'high', 'xhigh', 'max', 'none']); /** * OpenRouter unified thinking body. Switch off → omit (do not send effort:none; * some models reject disable). Switch on → reasoning.effort (default medium). * @param {{ enabled?: boolean, effort?: string }|null|undefined} reasoning * @returns {{ effort: string }|null} */ export function openRouterReasoningBody(reasoning) { if (!reasoning || reasoning.enabled !== true) return null; const effort = String(reasoning.effort || '').toLowerCase(); if (effort && REASONING_EFFORTS.has(effort) && effort !== 'none') return { effort }; return { enabled: true }; } /** * Inject OpenRouter `reasoning` onto a chat/completions JSON body. * @param {RequestInit} [init] * @param {{ enabled?: boolean, effort?: string }|null} [reasoning] * @returns {RequestInit} */ export function applyChatReasoningToInit(init = {}, reasoning) { return stripUnsupportedFieldsFromRequestInit(init, reasoning); } /** * Build a LanguageModel for streamText / ToolLoopAgent. * Uses the **active** chat provider. Wire path comes from provider.protocol * (openai-compatible | anthropic | openai-responses | auto) for any base URL. * @param {{ model?: string, reasoning?: { enabled?: boolean, effort?: string } }} [opts] * @returns {Promise<{ model: import('ai').LanguageModel, settings: object, modelId: string }>} */ export async function createBrowserBotLanguageModel(opts = {}) { const settings = opts.settings || await loadLlmSettings(); if (!settings.apiKey) { const err = new Error('NO_API_KEY'); err.code = 'NO_API_KEY'; throw err; } const modelId = resolveModelName(opts.model || settings.model); const baseURL = settings.apiBase || DEFAULT_BASE; const providerName = (settings.providerName && String(settings.providerName).trim()) || 'browserbot'; const route = resolveChatWire({ baseURL, model: modelId, protocol: settings.protocol }); if (!route.supported) { const err = new Error(route.error || 'CHAT_WIRE_UNSUPPORTED'); err.code = route.code || 'CHAT_WIRE_UNSUPPORTED'; throw err; } const protocol = route.protocol || CHAT_PROTOCOL_OPENAI; const mediaCapabilities = opts.mediaCapabilities || resolveChatMediaCapabilities( { ...settings, model: modelId }, (await loadCachedModelsForBase(baseURL)).models ); if (protocol === CHAT_PROTOCOL_ANTHROPIC) { const model = createAnthropicLanguageModel({ modelId, apiKey: settings.apiKey, baseURL, providerName, fetch: createExtensionFetch({ reasoning: null, mediaCapabilities, mediaContext: opts.mediaContext, noRetry: opts.noRetry === true, anthropic: true }) }); return { model, settings: { ...settings, protocol }, modelId }; } const provider = createOpenAICompatible({ name: providerName, apiKey: settings.apiKey, baseURL, includeUsage: true, fetch: createExtensionFetch({ reasoning: opts.reasoning || null, mediaCapabilities, mediaContext: opts.mediaContext, noRetry: opts.noRetry === true }) }); return { model: provider(modelId), settings: { ...settings, protocol }, modelId }; } /** * Fetch for AI SDK. Extension pages with host_permissions usually succeed * directly (and must, for live SSE). Offscreen can still throw TypeError * Failed to fetch (CORS / unsafe headers / PNA). Then SW `llm_proxy_fetch` * buffers the full HTTPS response — including SSE — so the turn can finish. * OpenRouter keeps reasoning / reasoning_details; Groq-class hosts strip them. * Anthropic Messages bodies must not go through OpenAI message sanitizers. * @param {{ reasoning?: { enabled?: boolean, effort?: string }|null, anthropic?: boolean }} [opts] * @returns {typeof fetch} */ export function createExtensionFetch(opts = {}) { return async function browserBotFetch(input, init = {}) { const headers = new Headers(init.headers || input?.headers); const requestNoRetry = headers.get('x-paw-internal-no-retry') === '1'; headers.delete('x-paw-internal-no-retry'); const url = resolveLlmFetchUrl(input); if (isCloudApiUrl(url)) { const client = bkClientHeaderValue(); if (client) headers.set(BK_CLIENT_HEADER, client); else headers.delete(BK_CLIENT_HEADER); } else { headers.delete(BK_PURPOSE_HEADER); headers.delete(BK_CLIENT_HEADER); } const keepReasoning = /openrouter\.ai/i.test(String(url)); const cleanedInit = opts.anthropic ? { ...init, headers: sanitizeLlmProxyHeaders(headers) } : stripUnsupportedFieldsFromRequestInit({ ...init, headers: sanitizeLlmProxyHeaders(headers) }, opts.reasoning || null, { keepReasoning, explicitOff: isCloudApiUrl(url), mediaCapabilities: opts.mediaContext?.current || opts.mediaCapabilities }); const body = await serializeLlmProxyBody(cleanedInit.body); const fetchInit = { ...cleanedInit, headers: sanitizeLlmProxyHeaders(cleanedInit.headers), body }; if (body == null) delete fetchInit.body; try { const res = await fetch(url || input, fetchInit); if (opts.mediaContext && [400, 415, 422].includes(res.status)) { try { const mime = classifyMediaRejection(res.status, (await res.clone().text()).slice(0, 16000), requestMediaFormats(fetchInit)); if (mime) await opts.mediaContext.reject(mime); } catch { /* evidence persistence must not replace the actual request error */ } } return keepReasoning ? liftOpenRouterReasoningResponse(res) : res; } catch (directErr) { if (opts.noRetry || requestNoRetry) throw directErr; if (isAbortLike(directErr, fetchInit?.signal)) throw toAbortError(directErr); console.warn('[BrowserBot] direct fetch failed, trying background proxy:', directErr?.message); return proxyFetchAsResponse(url || input, fetchInit, directErr, { keepReasoning }); } }; } function detailsPlainText(details) { if (!Array.isArray(details)) return ''; return details .map((d) => { if (!d || typeof d !== 'object') return ''; if (typeof d.text === 'string') return d.text; if (typeof d.summary === 'string') return d.summary; return ''; }) .filter(Boolean) .join(''); } /** * Gemini/OpenRouter often put plaintext in reasoning_details[].text and leave * delta.reasoning empty. AI SDK only maps reasoning / reasoning_content strings. */ export function liftOpenRouterReasoningPay - src/agent/providerPresets.js (11039 bytes)
/** * BYOK provider form templates (inference / image / stt / jev / web). * Base URLs are vendor-documented OpenAI-compatible or Anthropic Messages origins. * Inference `protocol` is always set: openai-compatible | anthropic | auto. */ /** @typedef {'openai-compatible'|'anthropic'|'auto'} InferencePresetProtocol */ /** * @typedef {{ * id: string, * name: string, * baseURL: string, * model?: string, * protocol: InferencePresetProtocol, * apiKeyPlaceholder?: string, * image?: object * }} InferenceProviderPreset */ const OPENROUTER_API_BASE = 'https://openrouter.ai/api/v1'; /** @type {InferenceProviderPreset[]} */ export const PROVIDER_PRESETS = [ { id: 'deepseek', name: 'DeepSeek', baseURL: 'https://api.deepseek.com/v1', model: 'deepseek-v4-flash', protocol: 'openai-compatible', apiKeyPlaceholder: 'sk-...' }, { id: 'openai', name: 'OpenAI', baseURL: 'https://api.openai.com/v1', model: 'gpt-4o', protocol: 'openai-compatible', apiKeyPlaceholder: 'sk-...' }, { id: 'anthropic', name: 'Anthropic', baseURL: 'https://api.anthropic.com/v1', model: 'claude-sonnet-4-20250514', protocol: 'anthropic', apiKeyPlaceholder: 'sk-ant-...' }, { id: 'google-gemini', name: 'Google Gemini', baseURL: 'https://generativelanguage.googleapis.com/v1beta/openai', model: 'gemini-2.0-flash', protocol: 'openai-compatible', apiKeyPlaceholder: 'AIza...' }, { id: 'mistral', name: 'Mistral', baseURL: 'https://api.mistral.ai/v1', model: 'mistral-large-latest', protocol: 'openai-compatible', apiKeyPlaceholder: '...' }, { id: 'xai', name: 'xAI Grok', baseURL: 'https://api.x.ai/v1', model: '', protocol: 'openai-compatible', apiKeyPlaceholder: 'xai-...' }, { id: 'groq', name: 'Groq', baseURL: 'https://api.groq.com/openai/v1', model: '', protocol: 'openai-compatible', apiKeyPlaceholder: 'gsk_...' }, { id: 'moonshot', name: 'Moonshot Kimi', baseURL: 'https://api.moonshot.ai/v1', model: 'kimi-k2.6', protocol: 'openai-compatible', apiKeyPlaceholder: 'sk-...' }, { id: 'moonshot-cn', name: 'Moonshot Kimi (中国)', baseURL: 'https://api.moonshot.cn/v1', model: '', protocol: 'openai-compatible', apiKeyPlaceholder: 'sk-...' }, { id: 'zhipu', name: '智谱 GLM', baseURL: 'https://open.bigmodel.cn/api/paas/v4', model: 'glm-5.2', protocol: 'openai-compatible', apiKeyPlaceholder: '...' }, { id: 'zai', name: 'Z.AI', baseURL: 'https://api.z.ai/api/paas/v4', model: 'glm-5.2', protocol: 'openai-compatible', apiKeyPlaceholder: '...' }, { id: 'dashscope', name: '阿里云 DashScope', baseURL: 'https://dashscope.aliyuncs.com/compatible-mode/v1', model: 'qwen-plus', protocol: 'openai-compatible', apiKeyPlaceholder: 'sk-...' }, { id: 'volcengine-ark', name: '火山方舟 Ark', baseURL: 'https://ark.cn-beijing.volces.com/api/v3', model: '', protocol: 'openai-compatible', apiKeyPlaceholder: '...' }, { id: 'minimax', name: 'MiniMax', baseURL: 'https://api.minimaxi.com/v1', model: 'MiniMax-Text-01', protocol: 'openai-compatible', apiKeyPlaceholder: 'eyJ... / API key', image: { enabled: false, protocol: 'minimax-image', path: '/image_generation', model: 'image-01' } }, { id: 'stepfun', name: '阶跃星辰 StepFun', baseURL: 'https://api.stepfun.com/v1', model: 'step-3.7-flash', protocol: 'openai-compatible', apiKeyPlaceholder: '...' }, { id: 'cohere', name: 'Cohere', baseURL: 'https://api.cohere.ai/compatibility/v1', model: 'command-a-plus-05-2026', protocol: 'openai-compatible', apiKeyPlaceholder: '...' }, { id: 'openrouter', name: 'OpenRouter', baseURL: OPENROUTER_API_BASE, model: 'openai/gpt-4o', protocol: 'openai-compatible', apiKeyPlaceholder: 'sk-or-...', image: { enabled: false, protocol: 'openrouter-image', path: '/images', model: 'google/gemini-2.5-flash-image', baseURL: OPENROUTER_API_BASE } }, { id: 'opencode-zen', name: 'OpenCode Zen', baseURL: 'https://opencode.ai/zen/v1', model: 'deepseek-v4-flash', protocol: 'auto', apiKeyPlaceholder: 'sk-...' }, { id: 'together', name: 'Together AI', baseURL: 'https://api.together.ai/v1', model: '', protocol: 'openai-compatible', apiKeyPlaceholder: '...' }, { id: 'fireworks', name: 'Fireworks', baseURL: 'https://api.fireworks.ai/inference/v1', model: '', protocol: 'openai-compatible', apiKeyPlaceholder: '...' }, { id: 'deepinfra', name: 'DeepInfra', baseURL: 'https://api.deepinfra.com/v1/openai', model: '', protocol: 'openai-compatible', apiKeyPlaceholder: '...' }, { id: 'novita', name: 'Novita', baseURL: 'https://api.novita.ai/openai/v1', model: '', protocol: 'openai-compatible', apiKeyPlaceholder: '...' }, { id: 'siliconflow', name: 'SiliconFlow', baseURL: 'https://api.siliconflow.cn/v1', model: 'deepseek-ai/DeepSeek-V4-Flash', protocol: 'openai-compatible', apiKeyPlaceholder: 'sk-...' }, { id: 'cerebras', name: 'Cerebras', baseURL: 'https://api.cerebras.ai/v1', model: '', protocol: 'openai-compatible', apiKeyPlaceholder: '...' }, { id: 'sambanova', name: 'SambaNova', baseURL: 'https://api.sambanova.ai/v1', model: '', protocol: 'openai-compatible', apiKeyPlaceholder: '...' }, { id: 'nebius', name: 'Nebius Token Factory', baseURL: 'https://api.tokenfactory.nebius.com/v1', model: '', protocol: 'openai-compatible', apiKeyPlaceholder: '...' }, { id: 'hyperbolic', name: 'Hyperbolic', baseURL: 'https://api.hyperbolic.xyz/v1', model: '', protocol: 'openai-compatible', apiKeyPlaceholder: '...' }, { id: 'featherless', name: 'Featherless', baseURL: 'https://api.featherless.ai/v1', model: '', protocol: 'openai-compatible', apiKeyPlaceholder: '...' }, { id: 'nvidia-nim', name: 'NVIDIA NIM', baseURL: 'https://integrate.api.nvidia.com/v1', model: '', protocol: 'openai-compatible', apiKeyPlaceholder: 'nvapi-...' }, { id: 'ollama', name: 'Ollama', baseURL: 'http://localhost:11434/v1', model: '', protocol: 'openai-compatible', apiKeyPlaceholder: 'ollama' }, { id: 'lmstudio', name: 'LM Studio', baseURL: 'http://localhost:1234/v1', model: '', protocol: 'openai-compatible', apiKeyPlaceholder: 'lm-studio' }, { id: 'openai-compatible', name: 'OpenAI Compatible', baseURL: '', model: '', protocol: 'openai-compatible', apiKeyPlaceholder: 'sk-... / any key' } ]; /** Image editor templates (not chat wire). */ export const IMAGE_PROVIDER_PRESETS = [ { id: 'openrouter', name: 'OpenRouter', baseURL: OPENROUTER_API_BASE }, { id: 'openai', name: 'OpenAI', baseURL: 'https://api.openai.com/v1' }, { id: 'minimax', name: 'MiniMax', baseURL: 'https://api.minimaxi.com/v1' }, { id: 'openai-compatible', name: 'OpenAI Compatible', baseURL: '' } ]; /** STT editor templates. */ export const STT_PROVIDER_PRESETS = [ { id: 'openai', name: 'OpenAI', baseURL: 'https://api.openai.com/v1', model: 'whisper-1' }, { id: 'groq', name: 'Groq', baseURL: 'https://api.groq.com/openai/v1', model: 'whisper-large-v3' }, { id: 'openai-compatible', name: 'OpenAI Compatible', baseURL: '' } ]; /** * System1 / System One templates (purpose:'jev'). * Model ids are vendor-specific (confirmed from current docs); custom leaves model empty. * Laya local base follows laya-serve Quickstart (`POST …/v1/systemone`); * model id `multilingual` (alias `laya` maps to english-only — do not use). */ export const JEV_PROVIDER_PRESETS = [ { id: 'opencode', name: 'OpenCode Zen', baseURL: 'https://opencode.ai/zen/v1', model: 'jev-1.13', apiKeyPlaceholder: 'sk-...' }, { id: 'typesafe', name: 'TypeSafe', baseURL: 'https://api.typesafe.ai/v1', model: 'jev-1.13.0', apiKeyPlaceholder: 'sk-...' }, { id: 'openrouter', name: 'OpenRouter', baseURL: OPENROUTER_API_BASE, model: 'typesafe/jev-1.13', apiKeyPlaceholder: 'sk-or-...' }, { id: 'laya-local', name: 'Laya (local)', baseURL: 'http://localhost:8000/v1', model: 'multilingual', apiKeyPlaceholder: 'any key / LAYA_SERVE_API_KEY' }, { id: 'openai-compatible', name: 'OpenAI Compatible', baseURL: '', model: '', apiKeyPlaceholder: 'sk-... / any key' } ]; /** Web search / extract templates. */ export const WEB_PROVIDER_PRESETS = [ { id: 'tavily', name: 'Tavily', baseURL: '' }, { id: 'brave', name: 'Brave', baseURL: '' }, { id: 'firecrawl', name: 'Firecrawl', baseURL: '' } ]; /** * Template combobox options: vendor presets only. * Blank `openai-compatible` stays in the catalog for the separate「自定义」action. * @template {{ id?: string }} T * @param {T[]} [presets] * @returns {T[]} */ export function presetsForTemplateCombobox(presets) { return (Array.isArray(presets) ? presets : []).filter((p) => p?.id !== 'openai-compatible'); } const ALLOWED_PROTOCOLS = new Set(['openai-compatible', 'anthropic', 'auto']); /** * Normalize base for uniqueness (strip trailing slash). Empty stays empty. * @param {string} baseURL */ export function normalizePresetBaseURL(baseURL) { return String(baseURL || '') .trim() .replace(/\/$/, ''); } /** * Validate inference preset catalog shape (pure; for tests + boot asserts). * @param {InferenceProviderPreset[]} [presets] */ export function assertInferencePresetsValid(presets = PROVIDER_PRESETS) { const ids = new Set(); const bases = new Set(); for (const p of presets) { if (!p || typeof p !== 'object') throw new Error('preset must be an object'); const id = String(p.id || '').trim(); const name = String(p.name || '').trim(); if (!id) throw new Error('preset missing id'); if (!name) throw new Error(`preset ${id} missing name`); if (ids.has(id)) throw new Error(`duplicate preset id: ${id}`); ids.add(id); if (!ALLOWED_PROTOCOLS.has(p.protocol)) { throw new Error(`preset ${id} has invalid protocol: ${p.protocol}`); } const base = normalizePresetBaseURL(p.baseURL); if (id === 'openai-compatible') { if (base) throw new Error('openai-compatible preset must keep empty baseURL'); continue; } if (!/^https?:\/\//i.test(base)) { throw new Error(`preset ${id} needs http(s) baseURL`); } const key = base.toLowerCase(); if (bases.has(key)) throw new Error(`duplicate baseURL: ${base}`); bases.add(key); } return true; } - src/agent/render/engine.js (4720 bytes)
/** * render_document engine routing — Product profile **A′** (2026-08-07) * * Install constraint: **extension only** — no npm, no desktop Pandoc for end users. * All product formats must work with files shipped inside the extension package. * * Public API: only `render_document({ draftId, format })`. * Never expose call_pandoc / md_to_* tools. * * | format | engine | delivery | * |--------|--------|----------| * | md, txt, csv, html, zip, pptx | builtin | download artifact | * | pdf | builtin_print | HTML print dialog → Save as PDF | * * docx / pandoc-wasm are **not** product formats under A′ (optional lab only, not default). */ /** Formats shown in UI and accepted as product capabilities (extension-only). */ export const PRODUCT_FORMATS = Object.freeze([ 'md', 'txt', 'csv', 'html', 'pdf', 'pptx', 'zip' ]); /** Alias used by render_document — A′: same as product surface. */ export const RENDER_FORMATS = PRODUCT_FORMATS; /** * @type {Readonly<Record<string, { * defaultEngine: 'builtin'|'builtin_print', * preferPandoc: boolean, * builtin: boolean, * pandoc: boolean, * fallback: null, * delivery: 'download'|'browser_print', * notes: string * }>>} */ export const FORMAT_ENGINE_MATRIX = Object.freeze({ md: { defaultEngine: 'builtin', preferPandoc: false, builtin: true, pandoc: false, fallback: null, delivery: 'download', notes: 'A′ extension-only markdown.' }, txt: { defaultEngine: 'builtin', preferPandoc: false, builtin: true, pandoc: false, fallback: null, delivery: 'download', notes: 'A′ plain text.' }, csv: { defaultEngine: 'builtin', preferPandoc: false, builtin: true, pandoc: false, fallback: null, delivery: 'download', notes: 'First table/records only.' }, html: { defaultEngine: 'builtin', preferPandoc: false, builtin: true, pandoc: false, fallback: null, delivery: 'download', notes: 'Export HTML file.' }, zip: { defaultEngine: 'builtin', preferPandoc: false, builtin: true, pandoc: false, fallback: null, delivery: 'download', notes: 'md + csv + images zip.' }, pptx: { defaultEngine: 'builtin', preferPandoc: false, builtin: true, pandoc: false, fallback: null, delivery: 'download', notes: 'Native OOXML PPTX (zip + ppt/presentation.xml), one slide per plate.' }, pdf: { defaultEngine: 'builtin_print', preferPandoc: false, builtin: true, pandoc: false, fallback: null, delivery: 'browser_print', notes: 'A′: print-ready HTML → system print dialog → Save as PDF. No Pandoc, no extra install. Optional simple binary PDF not product default.' } }); /** * @param {string} format * @param {{ * engine?: 'auto'|'builtin'|'builtin_print'|'pandoc', * forceBuiltin?: boolean, * forcePandoc?: boolean, * pandocReady?: boolean * }} [options] */ export function resolveEngine(format, options = {}) { const fmt = String(format || '').toLowerCase(); const row = FORMAT_ENGINE_MATRIX[fmt]; if (!row) { return { engine: null, fallback: null, format: fmt, reason: 'unsupported_format', delivery: null, errorCode: 'UNSUPPORTED_FORMAT' }; } // A′: never route product traffic to pandoc if (options.forcePandoc || options.engine === 'pandoc') { return { engine: null, fallback: null, format: fmt, reason: 'pandoc_not_in_product_A_prime', delivery: row.delivery, errorCode: 'ENGINE_NOT_IN_PRODUCT', message: 'Pandoc is not part of the extension-only product (A′). Use md/txt/csv/html/pdf(print)/zip/pptx.' }; } if (fmt === 'pdf' || row.defaultEngine === 'builtin_print') { return { engine: 'builtin_print', fallback: null, format: fmt, reason: 'a_prime_html_print_pdf', delivery: 'browser_print' }; } return { engine: 'builtin', fallback: null, format: fmt, reason: 'a_prime_builtin', delivery: row.delivery || 'download' }; } /** * @returns {{ formats: string[], matrix: typeof FORMAT_ENGINE_MATRIX, productProfile: string }} */ export function getRenderCapabilities() { return { productProfile: 'A_prime', install: 'extension_only', formats: [...PRODUCT_FORMATS], matrix: FORMAT_ENGINE_MATRIX, pandoc: false, notes: 'All formats ship inside the extension. PDF = HTML print → Save as PDF. No npm/Pandoc for end users.' }; } /** @deprecated lab only — always false in A′ product profile */ export function isPandocReady() { return false; } - src/agent/render/README.md (997 bytes)
# Document render engines — Product **A′** **Install rule:** end users only install the Chrome extension. No npm, no desktop Pandoc. **Public API:** only `render_document({ draftId, format })`. ## Product formats (shipped in extension) | format | engine | delivery | |--------|--------|----------| | md, txt, csv, html, zip | builtin | file download | | pptx | builtin | HTML slide deck download | | **pdf** | **builtin_print** | open print HTML → system print → **Save as PDF** | ## Not in A′ product surface | format | reason | |--------|--------| | docx | would need large WASM in package or extra install — deferred | | pandoc-wasm | not bundled; optional lab only, not user path | ## PDF flow 1. `render_document({ format: 'pdf' })` → print-ready HTML artifact + `delivery: 'browser_print'` 2. Side panel opens `src/preview/print.html?draftId=…&autoprint=1` 3. User chooses **Save as PDF** in the print dialog No extra software. Chinese / layout use browser engines. - src/agent/skills.js (10723 bytes)
/** * BrowserBot user Skills — local persistence (chrome.storage.local) * Product persistence model: solidify successful workflows → re-run via Agent Runtime. */ export const SKILLS_STORAGE_KEY = 'browserbot_user_skills'; export const MAX_USER_SKILLS = 80; export const MAX_SCRIPT_HINTS = 5; export const MAX_SCRIPT_HINT_CHARS = 8000; /** * @typedef {Object} UserSkill * @property {string} id * @property {string} name * @property {string} description * @property {number} createdAt * @property {number} updatedAt * @property {string|null} urlPattern * @property {string} promptTemplate * @property {string[]} preferredTools * @property {string[]} scriptHints * @property {string} lastTraceSummary * @property {string|null} sourceSessionId */ /** @returns {string} */ export function generateSkillId() { if (typeof crypto !== 'undefined' && crypto.randomUUID) { return crypto.randomUUID(); } return `skill_${Date.now()}_${Math.random().toString(36).slice(2, 10)}`; } /** * Normalize / validate skill input into a UserSkill record. * Pure — no storage side effects. * @param {object} input * @param {UserSkill|null} [existing] * @returns {UserSkill} */ export function normalizeSkill(input = {}, existing = null) { const now = Date.now(); const name = String(input.name || existing?.name || 'Untitled Skill').trim().slice(0, 80); const description = String(input.description || existing?.description || '').trim().slice(0, 500); const promptTemplate = String( input.promptTemplate || input.prompt_template || existing?.promptTemplate || '' ) .trim() .slice(0, 4000); let preferredTools = input.preferredTools || input.preferred_tools || existing?.preferredTools || []; if (!Array.isArray(preferredTools)) preferredTools = []; preferredTools = preferredTools.map((t) => String(t).slice(0, 64)).filter(Boolean).slice(0, 20); let scriptHints = input.scriptHints || input.script_hints || existing?.scriptHints || []; if (!Array.isArray(scriptHints)) scriptHints = []; scriptHints = scriptHints .map((s) => String(s || '').trim()) .filter(Boolean) .map((s) => (s.length > MAX_SCRIPT_HINT_CHARS ? s.slice(0, MAX_SCRIPT_HINT_CHARS) + '…' : s)) .slice(0, MAX_SCRIPT_HINTS); let urlPattern = input.urlPattern ?? input.url_pattern ?? existing?.urlPattern ?? null; if (urlPattern != null) { urlPattern = String(urlPattern).trim().slice(0, 300) || null; } const lastTraceSummary = String( input.lastTraceSummary || input.last_trace_summary || input.notes || existing?.lastTraceSummary || '' ) .trim() .slice(0, 2000); return { id: existing?.id || input.id || generateSkillId(), name: name || 'Untitled Skill', description, createdAt: existing?.createdAt || input.createdAt || now, updatedAt: now, urlPattern, promptTemplate: promptTemplate || name, preferredTools, scriptHints, lastTraceSummary, sourceSessionId: input.sourceSessionId ?? existing?.sourceSessionId ?? null }; } /** * Build skill from a completed agent run snapshot (UI or save_skill tool). * @param {object} opts */ export function skillFromRunSnapshot({ name, description, prompt, pageMeta = null, extractedCode = '', traces = [], sessionId = null, notes = '' } = {}) { const preferredTools = []; const seen = new Set(); for (const step of traces || []) { for (const tc of step.toolCalls || []) { if (tc?.name && !seen.has(tc.name) && tc.name !== 'finish' && tc.name !== 'save_skill') { seen.add(tc.name); preferredTools.push(tc.name); } } } const scriptHints = []; if (extractedCode && extractedCode.trim()) { scriptHints.push(extractedCode.trim()); } const urlPattern = pageMeta?.domain ? pageMeta.domain : pageMeta?.url ? safeOrigin(pageMeta.url) : null; const lastTraceSummary = notes || summarizeTraces(traces) || (description || '').slice(0, 500); return normalizeSkill({ name: name || deriveNameFromPrompt(prompt), description: description || deriveDescription(prompt, preferredTools), promptTemplate: prompt || '', preferredTools, scriptHints, urlPattern, lastTraceSummary, sourceSessionId: sessionId }); } export function deriveNameFromPrompt(prompt) { const p = String(prompt || '').trim().replace(/\s+/g, ' '); if (!p) return 'Saved Skill'; return p.length > 36 ? p.slice(0, 36) + '…' : p; } function deriveDescription(prompt, tools) { const p = String(prompt || '').trim().replace(/\s+/g, ' ').slice(0, 120); const t = (tools || []).slice(0, 5).join(', '); if (p && t) return `${p} · tools: ${t}`; return p || (t ? `Tools: ${t}` : ''); } export function summarizeTraces(traces) { if (!Array.isArray(traces) || !traces.length) return ''; const parts = []; for (const step of traces.slice(0, 12)) { const names = (step.toolCalls || []).map((c) => c.name).filter(Boolean); if (names.length) parts.push(`step${step.step}: ${names.join(', ')}`); } return parts.join(' → ').slice(0, 500); } export function safeOrigin(url) { try { return new URL(url).origin; } catch { return null; } } /** * Compact list for list_skills / UI cards. * @param {UserSkill[]} skills * @param {{ host?: string|null, limit?: number }} [opts] */ export function listSkillsCompact(skills, opts = {}) { const limit = opts.limit || 50; let list = Array.isArray(skills) ? [...skills] : []; if (opts.host) { const host = opts.host.toLowerCase(); list.sort((a, b) => { const as = skillMatchesHost(a, host) ? 0 : 1; const bs = skillMatchesHost(b, host) ? 0 : 1; if (as !== bs) return as - bs; return (b.updatedAt || 0) - (a.updatedAt || 0); }); } else { list.sort((a, b) => (b.updatedAt || 0) - (a.updatedAt || 0)); } return list.slice(0, limit).map((s) => ({ id: s.id, name: s.name, description: s.description, urlPattern: s.urlPattern, updatedAt: s.updatedAt, hasScriptHint: !!(s.scriptHints && s.scriptHints.length) })); } export function skillMatchesHost(skill, host) { if (!skill?.urlPattern || !host) return false; const p = String(skill.urlPattern).toLowerCase(); const h = String(host).toLowerCase(); return p === h || p.includes(h) || h.includes(p.replace(/^https?:\/\//, '').split('/')[0]); } /** * Find by id or name (case-insensitive name). * @param {UserSkill[]} skills * @param {string} idOrName * @returns {UserSkill|null} */ export function findSkill(skills, idOrName) { if (!idOrName || !Array.isArray(skills)) return null; const q = String(idOrName).trim(); const byId = skills.find((s) => s.id === q); if (byId) return byId; const lower = q.toLowerCase(); return skills.find((s) => (s.name || '').toLowerCase() === lower) || null; } /** * Build prompt + **user-side** skill context for re-running a skill. * Skill recipes are dynamic context (not system constitution) so system stays cache-stable. * * Skill `description` should be outcome-oriented (what / where), 1–2 sentences, no secrets. * * @param {UserSkill} skill * @returns {{ prompt: string, skillContext: string, skillSystemAppendix: string }} * skillSystemAppendix is always '' (legacy alias); use skillContext on the user turn. */ export function buildSkillRunPrompt(skill) { if (!skill) { return { prompt: '', skillContext: '', skillSystemAppendix: '' }; } const prompt = skill.promptTemplate || skill.name || 'Execute the saved Browser Kitten skill on the current page.'; // User-turn recipe: compact, actionable; no permission expansion claims const lines = [ 'Reusable procedure for this run only. Does not expand tools or permissions.', `name: ${skill.name}`, skill.description ? `outcome: ${skill.description}` : '', skill.urlPattern ? `site_hint: ${skill.urlPattern}` : '', skill.preferredTools?.length ? `preferred_tools: ${skill.preferredTools.join(', ')} (use when still relevant)` : '', skill.lastTraceSummary ? `prior_notes: ${skill.lastTraceSummary}` : '', skill.scriptHints?.length ? `script_hints (reference only; prefer structured tools; adapt if DOM changed):\n${skill.scriptHints .map((h, i) => ` --- hint ${i + 1} ---\n${h}`) .join('\n')}` : '', 'Execute end-to-end on the LIVE page with registered tools. Adapt selectors if the DOM changed. Call finish when done.' ].filter(Boolean); const skillContext = lines.join('\n'); return { prompt, skillContext, /** @deprecated always empty — skill is injected on user turn */ skillSystemAppendix: '' }; } // ── chrome.storage.local I/O ── function storageGet(keys) { return new Promise((resolve) => { if (typeof chrome === 'undefined' || !chrome.storage?.local) { resolve({}); return; } chrome.storage.local.get(keys, (res) => resolve(res || {})); }); } function storageSet(obj) { return new Promise((resolve, reject) => { if (typeof chrome === 'undefined' || !chrome.storage?.local) { reject(new Error('chrome.storage.local unavailable')); return; } chrome.storage.local.set(obj, () => { if (chrome.runtime?.lastError) { reject(new Error(chrome.runtime.lastError.message)); } else { resolve(); } }); }); } /** @returns {Promise<UserSkill[]>} */ export async function loadUserSkills() { const res = await storageGet([SKILLS_STORAGE_KEY]); const arr = res[SKILLS_STORAGE_KEY]; if (!Array.isArray(arr)) return []; return arr.map((s) => normalizeSkill(s, s)).filter((s) => s.id && s.name); } /** @param {UserSkill[]} skills */ export async function saveUserSkills(skills) { const capped = (Array.isArray(skills) ? skills : []).slice(-MAX_USER_SKILLS); await storageSet({ [SKILLS_STORAGE_KEY]: capped }); return capped; } /** * Upsert a skill (by id). * @param {object|UserSkill} input * @returns {Promise<UserSkill>} */ export async function upsertUserSkill(input) { const skills = await loadUserSkills(); const existing = input.id ? skills.find((s) => s.id === input.id) : null; const skill = normalizeSkill(input, existing || null); const next = existing ? skills.map((s) => (s.id === skill.id ? skill : s)) : [...skills, skill]; await saveUserSkills(next); return skill; } /** * @param {string} id * @returns {Promise<boolean>} */ export async function deleteUserSkill(id) { const skills = await loadUserSkills(); const next = skills.filter((s) => s.id !== id); if (next.length === skills.length) return false; await saveUserSkills(next); return true; } - src/agent/state.js (3435 bytes)
/** * BrowserBot Browser Agent — session / run state helpers */ /** Internal safety ceiling only — not a user setting (loops stop on finish/abort). */ /** Safety ceiling — sidepanel may pass a lower maxSteps (e.g. 20). */ export const DEFAULT_MAX_STEPS = 20; export const MAX_HISTORY_TURNS = 12; export const MAX_OBSERVATION_CHARS = 6000; export const MAX_DOM_SUMMARY_CHARS = 2500; /** * @typedef {Object} AgentTraceStep * @property {number} step * @property {string} [thought] * @property {Array<{id?: string, name: string, arguments: object}>} [toolCalls] * @property {Array<{toolCallId?: string, name: string, result: string}>} [observations] * @property {string} [assistantText] * @property {number} ts */ /** * @typedef {Object} AgentRunState * @property {string} runId * @property {string} sessionId * @property {number} step * @property {number} maxSteps * @property {boolean} done * @property {boolean} aborted * @property {boolean} waitingForUser * @property {AgentTraceStep[]} traces * @property {Array<{role: string, content?: string, tool_calls?: any[], tool_call_id?: string, name?: string}>} messages * @property {string} finalText * @property {string} extractedCode * @property {object|null} metrics */ export function createRunState({ sessionId = 'default', maxSteps = DEFAULT_MAX_STEPS } = {}) { return { runId: `run_${Date.now()}_${Math.random().toString(36).slice(2, 8)}`, sessionId, step: 0, maxSteps: maxSteps || DEFAULT_MAX_STEPS, done: false, aborted: false, waitingForUser: false, traces: [], messages: [], finalText: '', extractedCode: '', metrics: null, /** B1: chat | shape | task */ intent: null, intentConfidence: null, /** B3: chat | shape | plan | execute | verify */ phase: null, /** B2: Task Brief { goal, scope, format, limits, success_checks[] } */ taskBrief: null, /** * D2 finish gate run log (mutations / successful verify_*). * Runtime assigns createRunToolLog() from tools.js. */ runToolLog: null, /** @type {'not_required'|'verified'|'unverified'|null} */ finishVerification: null }; } export function truncateText(text, maxChars = MAX_OBSERVATION_CHARS) { if (text == null) return ''; const s = typeof text === 'string' ? text : JSON.stringify(text); if (s.length <= maxChars) return s; return s.slice(0, maxChars) + `\n…[truncated ${s.length - maxChars} chars]`; } export function compactHistoryMessages(messages, maxTurns = MAX_HISTORY_TURNS) { if (!Array.isArray(messages) || messages.length === 0) return []; // Keep system separate; trim oldest user/assistant pairs const system = messages.filter((m) => m.role === 'system'); const rest = messages.filter((m) => m.role !== 'system'); const capped = rest.slice(-Math.max(2, maxTurns * 2)); return [...system, ...capped]; } export function serializeTraceForStorage(traces, maxSteps = 20) { if (!Array.isArray(traces)) return []; return traces.slice(-maxSteps).map((t) => ({ step: t.step, thought: truncateText(t.thought || '', 2000), toolCalls: (t.toolCalls || []).map((c) => ({ name: c.name, arguments: c.arguments })), observations: (t.observations || []).map((o) => ({ name: o.name, result: truncateText(o.result || '', 1500) })), assistantText: truncateText(t.assistantText || '', 2000), ts: t.ts })); } - src/agent/trajectory.js (21577 bytes)
/** * BrowserBot agent trajectory — record / serialize / redact for dev export. * * Human owns completion: conversation.humanStatus is only set by the user * (export modal or sticky control). Model `finish` ≠ human completed. */ import { truncateText, MAX_OBSERVATION_CHARS } from './state.js'; import { BEHAVIOR_TRAJECTORY_SCHEMA, serializeBehaviorTrajectory, MAX_TURN_THOUGHT_CHARS } from './vnext/sessionWorkspace/behaviorPath.js'; /** Schema id for exported JSON — behavior path (tools + host), not a chat dump. */ export const TRAJECTORY_SCHEMA = BEHAVIOR_TRAJECTORY_SCHEMA; export { serializeBehaviorTrajectory, BEHAVIOR_TRAJECTORY_SCHEMA }; /** chrome.storage.local key: enable dev trajectory download UI (default true when unset) */ export const DEV_TRAJECTORY_STORAGE_KEY = 'browserbot_dev_trajectory'; /** Per tool result / observation cap for trajectory export (debug-friendly) */ export const TRAJECTORY_OBS_MAX_CHARS = 8000; /** * Cap for step `reasoning` when closing a step (storage + export). * Long-thinking models can emit tens of KB of chain-of-thought; keep enough for * abort/debug export. Trajectory is dev-oriented — large runs can grow * chrome.storage.local; MAX_RUN_STEPS_STORED / MAX_SESSION_TRAJECTORY_RUNS still * bound overall size. Raise further only if downloads still look truncated. */ export const TRAJECTORY_REASONING_MAX_CHARS = 48000; /** Cap for step assistantText when closing a step (not model reasoning) */ export const TRAJECTORY_ASSISTANT_TEXT_MAX_CHARS = 6000; /** Cap runs stored on a session to limit chrome.storage growth */ export const MAX_SESSION_TRAJECTORY_RUNS = 30; /** Cap steps kept per run in storage */ export const MAX_RUN_STEPS_STORED = 64; /** Cap chat messages included in export */ export const MAX_EXPORT_MESSAGES = 200; /** Cap message content chars in export */ export const MAX_EXPORT_MESSAGE_CHARS = 12000; /** Secret-like key name patterns (redacted in args/results deep walk) */ const SECRET_KEY_RE = /^(api[_-]?key|apikey|authorization|auth|password|passwd|secret|token|access[_-]?token|refresh[_-]?token|bearer|private[_-]?key|sk[-_])/i; /** Ordinary arrays share this cap; events / wire / audit keep their persist length. */ export const REDACT_ARRAY_CAP = 100; const PRESERVE_ARRAY_KEYS = new Set(['events', 'wire']); const HUMAN_STATUS_VALUES = new Set([ 'unknown', 'in_progress', 'completed', 'failed', 'partial', 'cancelled' ]); const RUN_STATUS_VALUES = new Set([ 'running', 'finished', 'aborted', 'error', 'waiting_user' ]); /** * Host only from base URL — never full URL with credentials, never API key. * @param {string} [baseURL] * @returns {string} */ export function baseURLHost(baseURL) { if (!baseURL || typeof baseURL !== 'string') return ''; try { const u = new URL(baseURL.includes('://') ? baseURL : `https://${baseURL}`); return u.host || ''; } catch { // Fallback: strip path-ish noise return String(baseURL) .replace(/^https?:\/\//i, '') .split('/')[0] .split('?')[0] .slice(0, 200); } } /** * Truncate text for trajectory (reuse state.truncateText with higher default). * @param {unknown} text * @param {number} [maxChars] */ export function truncateForTrajectory(text, maxChars = TRAJECTORY_OBS_MAX_CHARS) { return truncateText(text, maxChars); } /** * Deep-clone-ish redact of objects: strip secret keys, truncate long strings. * Never leave raw API keys in exported JSON. * Turn `events`, `wire`, and `sessionAudit.events` keep their full persist * length. Other arrays may still cap; the cut is marked. * @param {unknown} value * @param {{ maxString?: number, depth?: number, skipClip?: boolean, preserveArray?: boolean, arrayCap?: number }} [opts] * @returns {unknown} */ export function redactSecrets(value, opts = {}) { const maxString = opts.maxString ?? TRAJECTORY_OBS_MAX_CHARS; const depth = opts.depth ?? 0; const skipClip = opts.skipClip === true; const preserveArray = opts.preserveArray === true; const arrayCap = Number.isFinite(opts.arrayCap) ? opts.arrayCap : REDACT_ARRAY_CAP; if (depth > 12) return '[max depth]'; if (value == null) return value; if (typeof value === 'string') { // Heuristic: long sk- tokens if (/sk-[a-zA-Z0-9]{16,}/.test(value) || /Bearer\s+[A-Za-z0-9._-]{20,}/i.test(value)) { return '[REDACTED]'; } if (skipClip) return value; return truncateForTrajectory(value, maxString); } if (typeof value === 'number' || typeof value === 'boolean') return value; if (Array.isArray(value)) { const childOpts = { maxString, depth: depth + 1, skipClip, arrayCap }; if (preserveArray) { return value.map((v) => redactSecrets(v, childOpts)); } const mapped = value.slice(0, arrayCap).map((v) => redactSecrets(v, childOpts)); if (value.length > arrayCap) { mapped.push({ truncated: true, omitted: value.length - arrayCap }); } return mapped; } if (typeof value === 'object') { /** @type {Record<string, unknown>} */ const out = {}; const eventSkip = value.type === 'thought' || value.type === 'text'; for (const [k, v] of Object.entries(value)) { if (SECRET_KEY_RE.test(k)) { out[k] = '[REDACTED]'; continue; } // Never export nested provider key blobs if (k === 'apiKey' || k === 'api_key') { out[k] = '[REDACTED]'; continue; } const childSkip = skipClip || k === 'thought' || k === 'text' || eventSkip; out[k] = redactSecrets(v, { maxString, depth: depth + 1, skipClip: childSkip, preserveArray: PRESERVE_ARRAY_KEYS.has(k), arrayCap }); } return out; } return String(value).slice(0, 200); } /** * Infer tool success/fail from observation string/object. * @param {unknown} result * @returns {{ ok: boolean, errorCode: string }} */ export function inferToolResultMeta(result) { let ok = true; let errorCode = ''; let parsed = null; if (typeof result === 'string') { try { parsed = JSON.parse(result); } catch { // plain string — treat as ok unless it looks like an error blob if (/^\s*\{/.test(result) && /"status"\s*:\s*"(error|cancelled|failed)"/.test(result)) { ok = false; } return { ok, errorCode }; } } else if (result && typeof result === 'object') { parsed = result; } if (parsed && typeof parsed === 'object') { const st = String(parsed.status || '').toLowerCase(); if ( st === 'error' || st === 'cancelled' || st === 'failed' || st === 'denied' || st === 'timeout' ) { ok = false; } if (parsed.code) errorCode = String(parsed.code); else if (!ok && parsed.message) errorCode = 'ERROR'; } return { ok, errorCode }; } /** * Normalize human status enum. * @param {string} [status] * @returns {string} */ export function normalizeHumanStatus(status) { const s = String(status || 'unknown').toLowerCase().trim(); return HUMAN_STATUS_VALUES.has(s) ? s : 'unknown'; } /** * Ensure session has a trajectory bag (mutates session). * @param {object} session * @returns {object} session.trajectory */ export function ensureSessionTrajectory(session) { if (!session || typeof session !== 'object') { return { humanStatus: 'unknown', humanStatusNote: '', humanStatusSetAt: null, runs: [] }; } if (!session.trajectory || typeof session.trajectory !== 'object') { session.trajectory = { humanStatus: 'unknown', humanStatusNote: '', humanStatusSetAt: null, runs: [] }; } if (!Array.isArray(session.trajectory.runs)) session.trajectory.runs = []; if (!session.trajectory.humanStatus) session.trajectory.humanStatus = 'unknown'; if (session.trajectory.humanStatusNote == null) session.trajectory.humanStatusNote = ''; return session.trajectory; } /** * User sets conversation-level human status (never auto from model finish). * @param {object} session * @param {string} status * @param {string} [note] */ export function setSessionHumanStatus(session, status, note = '') { const bag = ensureSessionTrajectory(session); bag.humanStatus = normalizeHumanStatus(status); bag.humanStatusNote = note != null ? String(note).slice(0, 2000) : ''; bag.humanStatusSetAt = new Date().toISOString(); return bag; } /** * Create a mutable trajectory run recorder (one agent invocation). * @param {object} opts */ export function createTrajectoryRun(opts = {}) { const page = opts.pageMeta || null; let domain = ''; let url = ''; let title = ''; if (page && typeof page === 'object') { url = String(page.url || '').slice(0, 2000); title = String(page.title || '').slice(0, 500); domain = String(page.domain || '').slice(0, 200); if (!domain && url) { try { domain = new URL(url).hostname; } catch { /* ignore */ } } } return { runId: opts.runId || `run_${Date.now()}_${Math.random().toString(36).slice(2, 8)}`, startedAt: opts.startedAt || new Date().toISOString(), endedAt: null, status: 'running', finishReason: null, error: null, agentFinished: false, finishSummary: '', config: { model: String(opts.model || '').slice(0, 200), providerId: opts.providerId ? String(opts.providerId).slice(0, 120) : '', providerName: opts.providerName ? String(opts.providerName).slice(0, 120) : '', baseURLHost: baseURLHost(opts.baseURL || opts.apiBase || ''), planMode: !!opts.planMode, reasoning: !!opts.reasoning, lang: opts.lang || 'zh', maxSteps: opts.maxSteps || null }, context: { userPrompt: truncateForTrajectory(opts.prompt || opts.userPrompt || '', 8000), page: { url, title, domain }, selectionSummary: String(opts.selectionSummary || '').slice(0, 500), elementCount: Number(opts.elementCount) || 0, skillName: opts.skillName || null }, steps: [], events: [], metrics: {}, /** @private live step buffer */ _currentStep: null }; } /** * Append a run-level event (script_confirm, ask_user, user_abort, submit_confirm, …). * @param {object} run * @param {object} event */ export function appendTrajectoryEvent(run, event) { if (!run || !event) return; if (!Array.isArray(run.events)) run.events = []; const ts = event.ts || Date.now(); const type = String(event.type || 'event'); /** @type {Record<string, unknown>} */ const safe = { type, ts }; for (const [k, v] of Object.entries(event)) { if (k === 'type' || k === 'ts') continue; if (SECRET_KEY_RE.test(k)) { safe[k] = '[REDACTED]'; continue; } if (typeof v === 'string') { safe[k] = truncateForTrajectory(v, k === 'code' ? 4000 : 4000); } else if (typeof v === 'object' && v != null) { safe[k] = redactSecrets(v, { maxString: 2000 }); } else { safe[k] = v; } } run.events.push(safe); // Cap event list if (run.events.length > 200) { run.events = run.events.slice(-200); } } /** * Start a new step buffer on the run. * @param {object} run * @param {number} step */ export function trajectoryStartStep(run, step) { if (!run) return null; // Flush previous open step if any if (run._currentStep) { trajectoryEndStep(run); } run._currentStep = { step: step, ts: Date.now(), reasoning: '', assistantText: '', toolCalls: [], toolResults: [] }; return run._currentStep; } /** * Append deltas / tool data into current step. * @param {object} run * @param {'reasoning'|'text'|'tool_call'|'tool_result'} kind * @param {object} payload */ export function trajectoryStepPart(run, kind, payload = {}) { if (!run) return; if (!run._currentStep) { trajectoryStartStep(run, (run.steps?.length || 0) + 1); } const cur = run._currentStep; if (kind === 'reasoning') { cur.reasoning += String(payload.text || ''); } else if (kind === 'text') { cur.assistantText += String(payload.text || payload.chunk || ''); } else if (kind === 'tool_call') { cur.toolCalls.push({ id: payload.id || payload.toolCallId || '', name: String(payload.name || ''), arguments: redactSecrets(payload.arguments || payload.args || {}, { maxString: 4000 }) }); } else if (kind === 'tool_result') { const raw = typeof payload.result === 'string' ? payload.result : truncateForTrajectory(JSON.stringify(payload.result ?? ''), TRAJECTORY_OBS_MAX_CHARS); const meta = inferToolResultMeta(raw); const ok = payload.ok != null ? !!payload.ok : meta.ok; const errorCode = payload.errorCode || meta.errorCode || ''; cur.toolResults.push({ toolCallId: payload.toolCallId || payload.id || '', name: String(payload.name || ''), result: truncateForTrajectory(raw, TRAJECTORY_OBS_MAX_CHARS), ok, errorCode }); // Infer submit_confirm from tool observation (content_script gate) tryInferSubmitConfirmEvent(run, payload.name, raw); } } /** * Detect in-page submit confirm from invoke tool results. * @param {object} run * @param {string} name * @param {string} raw */ function tryInferSubmitConfirmEvent(run, name, raw) { if (!raw || typeof raw !== 'string') return; if (name && name !== 'invoke' && name !== 'click') { // still scan if code present } let parsed = null; try { parsed = JSON.parse(raw); } catch { return; } if (!parsed || typeof parsed !== 'object') return; if (parsed.code === 'SUBMIT_CONFIRM_REQUIRED' || parsed.needsUserConfirm) { appendTrajectoryEvent(run, { type: 'submit_confirm', allowed: false, reason: parsed.reason || 'cancelled', ts: Date.now() }); return; } if (parsed.userConfirmedSubmit === true) { appendTrajectoryEvent(run, { type: 'submit_confirm', allowed: true, reason: 'confirmed', ts: Date.now() }); } } /** * Close current step into run.steps. * @param {object} run */ export function trajectoryEndStep(run) { if (!run || !run._currentStep) return; const cur = run._currentStep; run._currentStep = null; // Truncate bulky fields for storage / session attach. // Reasoning uses TRAJECTORY_REASONING_MAX_CHARS so abort/export keeps long CoT // (not the old 6k debug kill). Trajectory remains dev-oriented for storage size. const stored = { step: cur.step, ts: cur.ts, reasoning: truncateForTrajectory( cur.reasoning || '', TRAJECTORY_REASONING_MAX_CHARS ), assistantText: truncateForTrajectory( cur.assistantText || '', TRAJECTORY_ASSISTANT_TEXT_MAX_CHARS ), toolCalls: (cur.toolCalls || []).map((c) => ({ id: c.id || '', name: c.name || '', arguments: redactSecrets(c.arguments || {}, { maxString: 4000 }) })), toolResults: (cur.toolResults - src/agent/vnext/adapters/codeRuntime.js (46915 bytes)
/** * Code runtime adapter (T8/T9) — QuickJS-WASM isolation (primary). * * Isolation guarantees: * - Guest code runs in a separate QuickJS WASM heap (no host globalThis). * - No chrome / window / document (including via globalThis / Function tricks). * - Injected: console + async fs RPC + sys (browser ABI via host). * - timeout + AbortSignal interrupt. * * Soft AsyncFunction path is emergency-only: * opts.runtime === 'soft' OR process.env.BROWSERBOT_SOFT_SANDBOX=1 * * Interface: * runCode({ code, entry?, signal?, timeoutMs?, fs, sys, runtime? }) → RunResult * codeRuntimeKind() → 'quickjs' | 'function-sandbox' */ /** * @typedef {object} RunResult * @property {number} exitStatus * @property {string} stdout * @property {string} stderr * @property {string[]} writtenFiles * @property {number} duration * @property {string} [error] * @property {unknown} [value] * @property {string} [runtime] // 'quickjs' | 'soft' */ import { SYS_HELP } from '../sessionWorkspace/browserSys.js'; import { createRunDeadline } from '../sessionWorkspace/runDeadline.js'; /** @type {'quickjs' | 'function-sandbox'} */ let _activeKind = 'quickjs'; /** * Marker for replaceable adapter identity. * Default is QuickJS; soft path returns 'function-sandbox' only while active. * @returns {'quickjs' | 'function-sandbox'} */ export function codeRuntimeKind() { return _activeKind; } /** * @param {{ * code: string, * entry?: string, * entryFile?: string, * files?: Record<string, string>, * signal?: AbortSignal, * timeoutMs?: number, * fs?: object, * globals?: Record<string, unknown>, * runtime?: 'quickjs' | 'soft' * }} opts * @returns {Promise<RunResult>} */ export async function runCode(opts = {}) { const code = String(opts.code || ''); const start = Date.now(); const hasVirtualProject = opts.files && typeof opts.files === 'object' && Object.keys(opts.files).length > 0; if (!code.trim() && !hasVirtualProject) { return { exitStatus: 1, stdout: '', stderr: '', writtenFiles: [], duration: Date.now() - start, error: 'code required' }; } if (opts.signal?.aborted) { return { exitStatus: 1, stdout: '', stderr: '', writtenFiles: [], duration: Date.now() - start, error: 'aborted before start' }; } // Product extension path: execute generated source in a manifest-sandboxed // page that has no extension APIs. Node tests and the sandbox page itself // use the local QuickJS adapter below. const sandboxRunner = globalThis.__BROWSERBOT_CODE_SANDBOX_RUN__; if (typeof sandboxRunner === 'function' && opts.runtime !== 'local-quickjs' && opts.runtime !== 'soft') { try { const result = await sandboxRunner(opts); return { ...result, runtime: result?.runtime || 'quickjs-sandbox' }; } catch (error) { const message = error instanceof Error ? error.message : String(error); return { exitStatus: 1, stdout: '', stderr: message, writtenFiles: [], duration: Date.now() - start, error: `sandboxed code runtime failed: ${message}`, runtime: 'quickjs-sandbox' }; } } const wantSoft = opts.runtime === 'soft' || (typeof process !== 'undefined' && process.env && (process.env.BROWSERBOT_SOFT_SANDBOX === '1' || process.env.BROWSERBOT_SOFT_SANDBOX === 'true')); if (wantSoft) { _activeKind = 'function-sandbox'; const r = await runSoftSandbox(opts); r.runtime = 'soft'; return r; } try { _activeKind = 'quickjs'; const r = await runQuickJS(opts); r.runtime = 'quickjs'; return r; } catch (e) { const msg = e instanceof Error ? e.message : String(e); await writeBlockLog(e); return { exitStatus: 1, stdout: '', stderr: msg, writtenFiles: [], duration: Date.now() - start, error: `QuickJS runtime failed: ${msg}`, runtime: 'quickjs' }; } } // ── QuickJS primary path ──────────────────────────────────────────────────── /** * Load QuickJS for the product host (Chrome MV3 extension) or Node tests. * * Order: * 1. Extension-relative vendored module (built by `npm run build:agent`) * — works in sidepanel without node_modules resolution. * 2. Bare `quickjs-emscripten` — Node/dev only. * 3. Total failure → write honest t8-block.log and rethrow (fail closed). * Soft sandbox is NOT a product fallback. * * @returns {Promise<{ * QuickJS: import('quickjs-emscripten-core').QuickJSWASMModule, * shouldInterruptAfterDeadline: (deadline: Date|number) => () => boolean * }>} */ async function loadQuickJSApi() { /** @type {unknown} */ let extensionErr = null; /** @type {unknown} */ let nodeErr = null; // 1) Extension-shipped relative module (must not use bare package specifier) try { const v = await import('./vendor/quickjs-loader.mjs'); const QuickJS = await v.getQuickJS(); const shouldInterruptAfterDeadline = typeof v.shouldInterruptAfterDeadline === 'function' ? v.shouldInterruptAfterDeadline : null; return { QuickJS, shouldInterruptAfterDeadline }; } catch (e1) { extensionErr = e1; } // 2) Node dev/test fallback (package resolution available) try { const q = await import('quickjs-emscripten'); const QuickJS = await q.getQuickJS(); return { QuickJS, shouldInterruptAfterDeadline: q.shouldInterruptAfterDeadline }; } catch (e2) { nodeErr = e2; } // 3) Fail closed — do not soft-sandbox as product default const combined = new Error( `QuickJS load failed (extension vendor + node). ` + `extension: ${extensionErr instanceof Error ? extensionErr.message : String(extensionErr)}; ` + `node: ${nodeErr instanceof Error ? nodeErr.message : String(nodeErr)}` ); await writeBlockLog({ extension: extensionErr, node: nodeErr, combined: combined.message }); throw combined; } /** * @param {object} opts * @returns {Promise<RunResult>} */ async function runQuickJS(opts) { const codeIn = String(opts.code || ''); const entry = opts.entry != null ? String(opts.entry) : null; const timeoutMs = clampTimeout(opts.timeoutMs); const deadlineClock = opts.deadline && typeof opts.deadline.expiresAt === 'number' ? opts.deadline : createRunDeadline(timeoutMs); const signal = opts.signal; const start = Date.now(); /** @type {string[]} */ const stdout = []; /** @type {string[]} */ const stderr = []; /** @type {string[]} */ const writtenFiles = []; const sandboxFs = bindFs(opts.fs, writtenFiles); // Optional TS / modern syntax transpile let code = codeIn; try { code = await compileGuestSource({ code: codeIn, files: opts.files, entryFile: opts.entryFile }); } catch (te) { const message = te instanceof Error ? te.message : String(te); return { exitStatus: 1, stdout: '', stderr: `compile failed: ${message}`, writtenFiles: [], duration: Date.now() - start, error: `compile failed: ${message}`, runtime: 'quickjs' }; } const { QuickJS, shouldInterruptAfterDeadline } = await loadQuickJSApi(); const runtime = QuickJS.newRuntime(); runtime.setMemoryLimit(clampMemoryLimit(opts.memoryLimitBytes)); runtime.setMaxStackSize(1024 * 512); runtime.setInterruptHandler(() => Boolean(signal?.aborted) || Date.now() > deadlineClock.expiresAt); const vm = runtime.newContext(); const hostDeferreds = createHostDeferredBag(); try { injectConsole(vm, stdout, stderr); injectFs(vm, sandboxFs, hostDeferreds); injectSys(vm, opts.sys, hostDeferreds); injectSleep(vm, signal, hostDeferreds, deadlineClock); // Harden: privileged host APIs are absent. Access throws so adversarial // probes fail closed (exitStatus !== 0) rather than silently returning host data. // (QuickJS has its own globalThis — host chrome/window/document never leak.) vm.unwrapResult( vm.evalCode(` (function () { function deny(name) { Object.defineProperty(globalThis, name, { configurable: true, enumerable: false, get: function () { throw new Error(name + ' is not available in sandbox'); }, set: function () { throw new Error(name + ' is not available in sandbox'); } }); } var banned = [ 'chrome', 'window', 'document', 'self', 'frames', 'parent', 'top', 'indexedDB', 'localStorage', 'sessionStorage', 'XMLHttpRequest', 'fetch', 'caches', 'cookieStore', 'navigator', 'location' ]; for (var i = 0; i < banned.length; i++) deny(banned[i]); })(); `) ).dispose(); const wrapped = wrapUserCode(code, entry); const evalResult = vm.evalCode(wrapped, 'task.js', { type: 'global' }); if (evalResult.error) { const errVal = vm.dump(evalResult.error); evalResult.error.dispose(); const msg = formatVmError(errVal); return { exitStatus: 1, stdout: stdout.join('\n'), stderr: (stderr.length ? stderr.join('\n') + '\n' : '') + msg, writtenFiles: unique(writtenFiles), duration: Date.now() - start, error: msg, runtime: 'quickjs' }; } const promiseHandle = evalResult.value; // User code returns a Promise (async IIFE). Resolve it on the host. const settled = await raceVmPromise(vm, promiseHandle, timeoutMs, signal, deadlineClock); promiseHandle.dispose(); if (settled.aborted) { return { exitStatus: 1, stdout: stdout.join('\n'), stderr: stderr.join('\n'), writtenFiles: unique(writtenFiles), duration: Date.now() - start, error: settled.error || 'aborted', runtime: 'quickjs' }; } if (settled.error) { const msg = settled.error; return { exitStatus: 1, stdout: stdout.join('\n'), stderr: (stderr.length ? stderr.join('\n') + '\n' : '') + msg, writtenFiles: unique(writtenFiles), duration: Date.now() - start, error: msg, runtime: 'quickjs' }; } return { exitStatus: 0, stdout: stdout.join('\n'), stderr: stderr.join('\n'), writtenFiles: unique(writtenFiles), duration: Date.now() - start, value: settled.value, runtime: 'quickjs' }; } finally { // newPromise() holds resolve/reject until dispose. In-flight fs/sys RPCs // must drop those handles before JS_FreeRuntime or WASM aborts on // list_empty(&rt->gc_obj_list) and can take the offscreen renderer with it. hostDeferreds.disposeAll(); try { vm.dispose(); } catch { /* ignore dispose races */ } try { runtime.dispose(); } catch { /* ignore */ } } } /** * Wrap guest source so it always yields a Promise of the entry result. * @param {string} code * @param {string|null} entry */ function wrapUserCode(code, entry) { const entryJson = JSON.stringify(entry); return ` "use strict"; var module = { exports: {} }; var exports = module.exports; (async function __pw_main() { ${code} if (typeof __pw_task_promise !== "undefined") { await __pw_task_promise; } var __entryName = ${entryJson}; var __fn = null; if (typeof module.exports === "function") { __fn = module.exports; } else if (module.exports && __entryName && typeof module.exports[__entryName] === "function") { __fn = module.exports[__entryName]; } else if (module.exports && typeof module.exports.default === "function") { __fn = module.exports.default; } else if (module.exports && typeof module.exports.main === "function") { __fn = module.exports.main; } if (typeof __fn === "function") { return await __fn(); } return module.exports; })(); `; } /** * @param {import('quickjs-emscripten').QuickJSContext} vm * @param {string[]} stdout * @param {string[]} stderr */ function injectConsole(vm, stdout, stderr) { const logFn = vm.newFunction('log', (...args) => { const parts = args.map((h) => { try { return stringify(vm.dump(h)); } catch { return String(h); } }); stdout.push(parts.join(' ')); }); const infoFn = vm.newFunction('info', (...args) => { const parts = args.map((h) => stringify(safeDump(vm, h))); stdout.push(parts.join(' ')); }); const warnFn = vm.newFunction('warn', (...args) => { const parts = args.map((h) => stringify(safeDump(vm, h))); stderr.push(parts.join(' ')); }); const errorFn = vm.newFunction('error', (...args) => { const parts = args.map((h) => stringify(safeDump(vm, h))); stderr.push(parts.join(' ')); }); const debugFn = vm.newFunction('debug', (...args) => { const parts = args.map((h) => stringify(safeDump(vm, h))); stdout.push(parts.join(' ')); }); const consoleHandle = vm.newObject(); vm.setProp(consoleHandle, 'log', logFn); vm.setProp(consoleHandle, 'info', infoFn); vm.setProp(consoleHandle, 'warn', warnFn); vm.setProp(consoleHandle, 'error', errorFn); vm.setProp(consoleHandle, 'debug', debugFn); vm.setProp(vm.global, 'console', consoleHandle); logFn.dispose(); infoFn.dispose(); warnFn.dispose(); errorFn.dispose(); debugFn.dispose(); consoleHandle.dispose(); } /** Track QuickJS deferreds from host fs/sys so teardown can free them. */ function createHostDeferredBag() { /** @type {Set<{ dispose?: () => void }>} */ const pending = new Set(); return { track(deferred) { if (deferred) pending.add(deferred); return deferred; }, forget(deferred) { pending.delete(deferred); }, disposeAll() { for (const deferred of pending) { try { deferred.dispose?.(); } catch { /* already freed */ } } pending.clear(); } }; } function releaseHostDeferred(handles, deferred) { handles?.forget?.(deferred); try { deferred?.dispose?.(); } catch { /* already freed */ } } /** * Inject async fs object (RPC-style promises). * @param {import('quickjs-emscripten').QuickJSContext} vm * @param {object} sandboxFs * @param {ReturnType<typeof createHostDeferredBag>} handles */ function injectFs(vm, sandboxFs, handles) { const fsHandle = vm.newObject(); const methods = ['readFile', 'writeFile', 'readdir', 'mkdir', 'stat', 'exists', 'remove', 'rm']; for (const name of methods) { const hostFn = typeof sandboxFs[name] === 'function' ? sandboxFs[name].bind(sandboxFs) : name === 'rm' && typeof sandboxFs.remove === 'function' ? sandboxFs.remove.bind(sandboxFs) : null; const fnHandle = vm.newFunction(name, (...argHandles) => { const args = argHa - src/agent/vnext/adapters/sandboxClient.js (7360 bytes)
/** * Host-side client for the manifest sandboxed code runtime. * * Generated source is executed only in the sandboxed extension page, which has * no extension APIs. Filesystem and sys calls are relayed to the offscreen host. * The sandbox never receives chrome.*, DOM handles, or a reference to Workspace storage. */ import { SANDBOX_CHANNEL } from '../../../brand.js'; import { createRunDeadline, clampRunTimeout } from '../sessionWorkspace/runDeadline.js'; const CHANNEL = SANDBOX_CHANNEL; export function createSandboxCodeClient(iframe) { if (!iframe) throw new Error('sandbox iframe required'); const runs = new Map(); let readyResolved = false; let readyResolve; let readyReject; const ready = new Promise((resolve, reject) => { readyResolve = resolve; readyReject = reject; }); const readyTimer = setTimeout(() => { if (readyResolved) return; console.warn('[sandbox] guest did not become ready within 15s — code run will fail until handshake'); readyReject(new Error('code sandbox did not become ready')); }, 15000); function guestWindow() { return iframe.contentWindow || null; } async function onMessage(event) { if (event.source !== guestWindow()) return; const msg = event.data; if (!msg || msg.channel !== CHANNEL) return; if (msg.type === 'ready') { try { event.source?.postMessage({ channel: CHANNEL, type: 'ready-ack' }, '*'); } catch (_) {} if (!readyResolved) { readyResolved = true; clearTimeout(readyTimer); readyResolve(true); } return; } if (msg.type === 'fs-request') { const run = runs.get(msg.runId); const target = guestWindow(); if (!run) { rejectMissingRun(target, msg, 'fs-response'); return; } if (!target) return; try { const method = normalizeFsMethod(msg.method); const fn = run.fs?.[method]; if (typeof fn !== 'function') throw new Error(`fs.${method} unavailable`); const value = await fn.apply(run.fs, Array.isArray(msg.args) ? msg.args : []); target.postMessage({ channel: CHANNEL, type: 'fs-response', runId: msg.runId, requestId: msg.requestId, ok: true, value }, '*'); } catch (error) { target.postMessage({ channel: CHANNEL, type: 'fs-response', runId: msg.runId, requestId: msg.requestId, ok: false, error: error instanceof Error ? error.message : String(error) }, '*'); } return; } if (msg.type === 'sys-request') { const run = runs.get(msg.runId); const target = guestWindow(); if (!run) { rejectMissingRun(target, msg, 'sys-response'); return; } if (!target) return; try { const call = run.sys?.call; if (typeof call !== 'function') throw new Error('SYS_DENIED: browser sys has no host in this runtime'); const value = await call.call(run.sys, msg.op, msg.params || {}); target.postMessage({ channel: CHANNEL, type: 'sys-response', runId: msg.runId, requestId: msg.requestId, ok: true, value }, '*'); } catch (error) { target.postMessage({ channel: CHANNEL, type: 'sys-response', runId: msg.runId, requestId: msg.requestId, ok: false, code: typeof error?.code === 'string' ? error.code : 'SYS_FAILED', error: error instanceof Error ? error.message : String(error) }, '*'); } return; } if (msg.type === 'run-result') { const run = runs.get(msg.runId); if (!run) return; cleanupRun(msg.runId, run); if (msg.ok) run.resolve(msg.result); else run.reject(new Error(msg.error || 'sandbox run failed')); } } function cleanupRun(runId, run) { runs.delete(runId); if (run.timer) clearTimeout(run.timer); if (typeof run.unsub === 'function') run.unsub(); if (run.signal && run.onAbort) run.signal.removeEventListener('abort', run.onAbort); } window.addEventListener('message', onMessage); return { ready, async run(opts = {}) { await ready; const target = guestWindow(); if (!target) throw new Error('code sandbox frame missing'); if (opts.signal?.aborted) throw new DOMException('Aborted', 'AbortError'); const runId = crypto.randomUUID ? crypto.randomUUID() : `run_${Date.now()}_${Math.random().toString(36).slice(2)}`; return new Promise((resolve, reject) => { const timeoutMs = clampRunTimeout(opts.timeoutMs); const deadline = opts.deadline && typeof opts.deadline.remaining === 'function' ? opts.deadline : createRunDeadline(timeoutMs); const onAbort = () => { target.postMessage({ channel: CHANNEL, type: 'abort', runId }, '*'); }; const armHostTimer = () => { const run = runs.get(runId); if (!run) return; if (run.timer) clearTimeout(run.timer); const remain = Math.max(1, deadline.remaining()) + 2000; run.timer = setTimeout(() => { const live = runs.get(runId); if (!live) return; target.postMessage({ channel: CHANNEL, type: 'abort', runId }, '*'); cleanupRun(runId, live); reject(new Error(`sandbox host timeout after ${timeoutMs + 2000}ms`)); }, remain); }; const onDeadline = (expiresAt) => { target.postMessage({ channel: CHANNEL, type: 'extend-deadline', runId, expiresAt }, '*'); armHostTimer(); }; const unsub = deadline.subscribe(onDeadline); runs.set(runId, { resolve, reject, fs: opts.fs, sys: opts.sys, signal: opts.signal, onAbort, timer: null, unsub }); armHostTimer(); opts.signal?.addEventListener('abort', onAbort, { once: true }); target.postMessage({ channel: CHANNEL, type: 'run', runId, payload: { code: String(opts.code || ''), entry: opts.entry == null ? null : String(opts.entry), entryFile: opts.entryFile == null ? null : String(opts.entryFile), files: opts.files && typeof opts.files === 'object' ? opts.files : null, timeoutMs, expiresAt: deadline.expiresAt, memoryLimitBytes: opts.memoryLimitBytes } }, '*'); }); }, dispose() { window.removeEventListener('message', onMessage); clearTimeout(readyTimer); for (const [runId, run] of runs) { cleanupRun(runId, run); run.reject(new Error('sandbox client disposed')); } } }; } function rejectMissingRun(target, msg, type) { if (!target) return; try { target.postMessage({ channel: CHANNEL, type, runId: msg.runId, requestId: msg.requestId, ok: false, code: 'SYS_ABORTED', error: 'run is no longer active' }, '*'); } catch { /* frame gone */ } } function normalizeFsMethod(name) { const value = String(name || ''); if (value === 'rm') return 'remove'; const allowed = new Set(['readFile', 'writeFile', 'readdir', 'mkdir', 'stat', 'exists', 'remove']); if (!allowed.has(value)) throw new Error(`unsupported fs method: ${value}`); return value; } - src/agent/vnext/adapters/stdlib.js (3320 bytes)
/** Bundled, dependency-free BrowserBot standard-library source modules. */ export const BROWSERBOT_STDLIB_MODULES = Object.freeze({ 'browserbot:stdlib/csv': ` export function escapeCsv(value) { const text = value == null ? '' : String(value); const quote = text.includes(',') || text.includes('\"') || text.includes(String.fromCharCode(10)) || text.includes(String.fromCharCode(13)); return quote ? '"' + text.replace(/"/g, '""') + '"' : text; } export function rowsToCsv(rows, columns) { const list = Array.isArray(rows) ? rows : []; const cols = Array.isArray(columns) && columns.length ? columns : Array.from(new Set(list.flatMap(row => row && typeof row === 'object' ? Object.keys(row) : []))); return [cols.map(escapeCsv).join(','), ...list.map(row => cols.map(col => escapeCsv(row?.[col])).join(','))].join(String.fromCharCode(10)); } `, 'browserbot:stdlib/html': ` export function escapeHtml(value) { return String(value == null ? '' : value) .replace(/&/g, '&').replace(/</g, '<').replace(/>/g, '>') .replace(/"/g, '"').replace(/'/g, '''); } export function htmlDocument({ title = 'Browser Kitten Output', body = '', styles = '' } = {}) { return '<!doctype html><html><head><meta charset="utf-8"><meta name="viewport" content="width=device-width,initial-scale=1">' + '<title>' + escapeHtml(title) + '</title><style>' + styles + '</style></head><body>' + body + '</body></html>'; } export function tableHtml(rows, columns) { const list = Array.isArray(rows) ? rows : []; const cols = Array.isArray(columns) && columns.length ? columns : Array.from(new Set(list.flatMap(row => Object.keys(row || {})))); return '<table><thead><tr>' + cols.map(c => '<th>' + escapeHtml(c) + '</th>').join('') + '</tr></thead><tbody>' + list.map(row => '<tr>' + cols.map(c => '<td>' + escapeHtml(row?.[c]) + '</td>').join('') + '</tr>').join('') + '</tbody></table>'; } `, 'browserbot:stdlib/table': ` export function select(rows, columns) { return (Array.isArray(rows) ? rows : []).map(row => Object.fromEntries(columns.map(column => [column, row?.[column]]))); } export function groupBy(rows, key) { const out = Object.create(null); for (const row of Array.isArray(rows) ? rows : []) { const value = typeof key === 'function' ? key(row) : row?.[key]; const name = String(value ?? ''); (out[name] ||= []).push(row); } return out; } export function sortBy(rows, key, direction = 'asc') { const factor = direction === 'desc' ? -1 : 1; return [...(Array.isArray(rows) ? rows : [])].sort((a, b) => { const av = typeof key === 'function' ? key(a) : a?.[key]; const bv = typeof key === 'function' ? key(b) : b?.[key]; return av < bv ? -factor : av > bv ? factor : 0; }); } `, 'browserbot:stdlib/runtime': ` export const fs = globalThis.fs; export const sys = globalThis.sys; export async function writeJson(path, value, spacing = 2) { await fs.writeFile(path, JSON.stringify(value, null, spacing)); } export async function readJson(path) { return JSON.parse(await fs.readFile(path)); } ` }); - src/agent/vnext/adapters/vendor/esbuild-loader.mjs (150401 bytes)
var __create = Object.create; var __defProp = Object.defineProperty; var __getOwnPropDesc = Object.getOwnPropertyDescriptor; var __getOwnPropNames = Object.getOwnPropertyNames; var __getProtoOf = Object.getPrototypeOf; var __hasOwnProp = Object.prototype.hasOwnProperty; var __commonJS = (cb, mod) => function __require() { try { return mod || (0, cb[__getOwnPropNames(cb)[0]])((mod = { exports: {} }).exports, mod), mod.exports; } catch (e) { throw mod = 0, e; } }; var __copyProps = (to, from, except, desc) => { if (from && typeof from === "object" || typeof from === "function") { for (let key of __getOwnPropNames(from)) if (!__hasOwnProp.call(to, key) && key !== except) __defProp(to, key, { get: () => from[key], enumerable: !(desc = __getOwnPropDesc(from, key)) || desc.enumerable }); } return to; }; var __toESM = (mod, isNodeMode, target) => (target = mod != null ? __create(__getProtoOf(mod)) : {}, __copyProps( // If the importer is in node compatibility mode or this is not an ESM // file that has been converted to a CommonJS file using a Babel- // compatible transform (i.e. "__esModule" has not been set), then set // "default" to the CommonJS "module.exports" for node compatibility. isNodeMode || !mod || !mod.__esModule ? __defProp(target, "default", { value: mod, enumerable: true }) : target, mod )); // node_modules/esbuild-wasm/lib/browser.js var require_browser = __commonJS({ "node_modules/esbuild-wasm/lib/browser.js"(exports, module) { ((module2) => { "use strict"; var __defProp2 = Object.defineProperty; var __getOwnPropDesc2 = Object.getOwnPropertyDescriptor; var __getOwnPropNames2 = Object.getOwnPropertyNames; var __hasOwnProp2 = Object.prototype.hasOwnProperty; var __export = (target, all) => { for (var name in all) __defProp2(target, name, { get: all[name], enumerable: true }); }; var __copyProps2 = (to, from, except, desc) => { if (from && typeof from === "object" || typeof from === "function") { for (let key of __getOwnPropNames2(from)) if (!__hasOwnProp2.call(to, key) && key !== except) __defProp2(to, key, { get: () => from[key], enumerable: !(desc = __getOwnPropDesc2(from, key)) || desc.enumerable }); } return to; }; var __toCommonJS = (mod) => __copyProps2(__defProp2({}, "__esModule", { value: true }), mod); var __async = (__this, __arguments, generator) => { return new Promise((resolve, reject) => { var fulfilled = (value) => { try { step(generator.next(value)); } catch (e) { reject(e); } }; var rejected = (value) => { try { step(generator.throw(value)); } catch (e) { reject(e); } }; var step = (x) => x.done ? resolve(x.value) : Promise.resolve(x.value).then(fulfilled, rejected); step((generator = generator.apply(__this, __arguments)).next()); }); }; var browser_exports = {}; __export(browser_exports, { analyzeMetafile: () => analyzeMetafile, analyzeMetafileSync: () => analyzeMetafileSync, build: () => build2, buildSync: () => buildSync, context: () => context, default: () => browser_default, formatMessages: () => formatMessages, formatMessagesSync: () => formatMessagesSync, initialize: () => initialize2, stop: () => stop, transform: () => transform2, transformSync: () => transformSync, version: () => version2 }); module2.exports = __toCommonJS(browser_exports); function encodePacket(packet) { let visit = (value) => { if (value === null) { bb.write8(0); } else if (typeof value === "boolean") { bb.write8(1); bb.write8(+value); } else if (typeof value === "number") { bb.write8(2); bb.write32(value | 0); } else if (typeof value === "string") { bb.write8(3); bb.write(encodeUTF8(value)); } else if (value instanceof Uint8Array) { bb.write8(4); bb.write(value); } else if (value instanceof Array) { bb.write8(5); bb.write32(value.length); for (let item of value) { visit(item); } } else { let keys = Object.keys(value); bb.write8(6); bb.write32(keys.length); for (let key of keys) { bb.write(encodeUTF8(key)); visit(value[key]); } } }; let bb = new ByteBuffer(); bb.write32(0); bb.write32(packet.id << 1 | +!packet.isRequest); visit(packet.value); writeUInt32LE(bb.buf, bb.len - 4, 0); return bb.buf.subarray(0, bb.len); } function decodePacket(bytes) { let visit = () => { switch (bb.read8()) { case 0: return null; case 1: return !!bb.read8(); case 2: return bb.read32(); case 3: return decodeUTF8(bb.read()); case 4: return bb.read(); case 5: { let count = bb.read32(); let value2 = []; for (let i = 0; i < count; i++) { value2.push(visit()); } return value2; } case 6: { let count = bb.read32(); let value2 = {}; for (let i = 0; i < count; i++) { value2[decodeUTF8(bb.read())] = visit(); } return value2; } default: throw new Error("Invalid packet"); } }; let bb = new ByteBuffer(bytes); let id = bb.read32(); let isRequest = (id & 1) === 0; id >>>= 1; let value = visit(); if (bb.ptr !== bytes.length) { throw new Error("Invalid packet"); } return { id, isRequest, value }; } var ByteBuffer = class { constructor(buf = new Uint8Array(1024)) { this.buf = buf; this.len = 0; this.ptr = 0; } _write(delta) { if (this.len + delta > this.buf.length) { let clone = new Uint8Array((this.len + delta) * 2); clone.set(this.buf); this.buf = clone; } this.len += delta; return this.len - delta; } write8(value) { let offset = this._write(1); this.buf[offset] = value; } write32(value) { let offset = this._write(4); writeUInt32LE(this.buf, value, offset); } write(bytes) { let offset = this._write(4 + bytes.length); writeUInt32LE(this.buf, bytes.length, offset); this.buf.set(bytes, offset + 4); } _read(delta) { if (this.ptr + delta > this.buf.length) { throw new Error("Invalid packet"); } this.ptr += delta; return this.ptr - delta; } read8() { return this.buf[this._read(1)]; } read32() { return readUInt32LE(this.buf, this._read(4)); } read() { let length = this.read32(); let bytes = new Uint8Array(length); let ptr = this._read(bytes.length); bytes.set(this.buf.subarray(ptr, ptr + length)); return bytes; } }; var encodeUTF8; var decodeUTF8; var encodeInvariant; if (typeof TextEncoder !== "undefined" && typeof TextDecoder !== "undefined") { let encoder = new TextEncoder(); let decoder = new TextDecoder(); encodeUTF8 = (text) => encoder.encode(text); decodeUTF8 = (bytes) => decoder.decode(bytes); encodeInvariant = 'new TextEncoder().encode("")'; } else if (typeof Buffer !== "undefined") { encodeUTF8 = (text) => Buffer.from(text); decodeUTF8 = (bytes) => { let { buffer, byteOffset, byteLength } = bytes; return Buffer.from(buffer, byteOffset, byteLength).toString(); }; encodeInvariant = 'Buffer.from("")'; } else { throw new Error("No UTF-8 codec found"); } if (!(encodeUTF8("") instanceof Uint8Array)) throw new Error(`Invariant violation: "${encodeInvariant} instanceof Uint8Array" is incorrectly false This indicates that your JavaScript environment is broken. You cannot use esbuild in this environment because esbuild relies on this invariant. This is not a problem with esbuild. You need to fix your environment instead. `); function readUInt32LE(buffer, offset) { return (buffer[offset++] | buffer[offset++] << 8 | buffer[offset++] << 16 | buffer[offset++] << 24) >>> 0; } function writeUInt32LE(buffer, value, offset) { buffer[offset++] = value; buffer[offset++] = value >> 8; buffer[offset++] = value >> 16; buffer[offset++] = value >> 24; } var fromCharCode = String.fromCharCode; function throwSyntaxError(bytes, index, message) { const c = bytes[index]; let line = 1; let column = 0; for (let i = 0; i < index; i++) { if (bytes[i] === 10) { line++; column = 0; } else { column++; } } throw new SyntaxError( message ? message : index === bytes.length ? "Unexpected end of input while parsing JSON" : c >= 32 && c <= 126 ? `Unexpected character ${fromCharCode(c)} in JSON at position ${index} (line ${line}, column ${column})` : `Unexpected byte 0x${c.toString(16)} in JSON at position ${index} (line ${line}, column ${column})` ); } function JSON_parse(bytes) { if (!(bytes instanceof Uint8Array)) { throw new Error(`JSON input must be a Uint8Array`); } const propertyStack = []; const objectStack = []; const stateStack = []; const length = bytes.length; let property = null; let state = 0; let object; let i = 0; while (i < length) { let c = bytes[i++]; if (c <= 32) { continue; } let value; if (state === 2 && property === null && c !== 34 && c !== 125) { throwSyntaxError(bytes, --i); } switch (c) { // True case 116: { if (bytes[i++] !== 114 || bytes[i++] !== 117 || bytes[i++] !== 101) { throwSyntaxError(bytes, --i); } value = true; break; } // False case 102: { if (bytes[i++] !== 97 || bytes[i++] !== 108 || bytes[i++] !== 115 || bytes[i++] !== 101) { throwSyntaxError(bytes, --i); } value = false; break; } // Null case 110: { if (bytes[i++] !== 117 || bytes[i++] !== 108 || bytes[i++] !== 108) { throwSyntaxError(bytes, --i); } value = null; break; } // Number begin case 45: case 46: case 48: case 49: case 50: case 51: case 52: case 53: case 54: case 55: case 56: case 57: { let index = i; value = fromCharCode(c); c = bytes[i]; while (true) { switch (c) { case 43: case 45: case 46: case 48: case 49: case 50: case 51: case 52: case 53: case 54: case 55: case 56: case 57: case 101: case 69: { value += fromCharCode(c); c = bytes[++i]; continue; } } break; } value = +value; if (isNaN(value)) { throwSyntaxError(bytes, --index, "Invalid number"); } break; } // String begin case 34: { value = ""; while (true) { if (i >= length) { throwSyntaxError(bytes, length); } c = bytes[i++]; if (c === 34) { break; } else if (c === 92) { switch (bytes[i++]) { // Normal escape sequence case 34: value += '"'; break; case 47: value += "/"; break; case 92: value += "\\"; break; case 98: value += "\b"; break; case 102: value += "\f"; break; case 110: value += "\n"; break; case 114: value += "\r"; break; case 116: value += " "; break; // Unicode escape sequence case 117: { let code = 0; for (let j = 0; j < 4; j++) { c = bytes[i++]; code <<= 4; if (c >= 48 && c <= 57) code |= c - 48; else if (c >= 97 && c <= 102) code |= c + (10 - 97); else if (c >= 65 && c <= 70) code |= c + (10 - 65); else throwSyntaxError(bytes, --i); } value += fromCharCode(code); break; } // Invalid escape sequence default: throwSyntaxError(bytes, --i); break; } } else if (c <= 127) { value += fromCharCode(c); } else if ((c & 224) === 192) { value += fromCharCode((c & 31) << 6 | bytes[i++] & 63); } - src/agent/vnext/host/accessPolicy.js (1468 bytes)
/** * One access mode. There is no user setting to switch. * * Every mutating page action and sys call still passes the host gate * (operationGate → dispatchTicket), whose rules do not depend on a mode: * known payment → never dispatched * known delete → one-shot host approval * everything else, including page scripts and CDP → dispatched (best effort; the * classifier is heuristic and raw code can reach anything the page can) * * The former Guarded / Full storage keys are deleted on startup (clearLegacyAccessPolicy). * Journal rows keep `accessMode` for compatibility; it is always "full". */ import { STORAGE_ACCESS_POLICY, STORAGE_ACCESS_POLICY_SESSION } from '../../../brand.js'; export const ACCESS_MODE = 'full'; const POLICY = Object.freeze({ mode: ACCESS_MODE, source: 'fixed' }); export async function readEffectiveAccessPolicy() { return POLICY; } export function snapshotAccessPolicy(effective = POLICY, now = Date.now()) { return { ...effective, frozenAt: now }; } /** Remove the old per-profile and per-session mode keys. Safe to call repeatedly. */ export async function clearLegacyAccessPolicy(areas) { const local = areas?.local ?? globalThis.chrome?.storage?.local; const session = areas?.session ?? globalThis.chrome?.storage?.session; await Promise.all([ local?.remove?.(STORAGE_ACCESS_POLICY)?.catch?.(() => {}), session?.remove?.(STORAGE_ACCESS_POLICY_SESSION)?.catch?.(() => {}) ]); } - src/agent/vnext/host/browserExecution.js (2208 bytes)
/** Browser resources are scoped to an execution, not to a sidepanel or a model call. */ import { listTabLeases, revokeTabLeaseOwner, releaseOwnedTabLeases } from './tabLease.js'; import { releaseBrowserSysResources } from './browserSysHost.js'; import { clearVisualReplay } from './visualClick.js'; const releases = new Map(); export function releaseBrowserExecution(sessionId, executionId) { if (!sessionId || !executionId) return Promise.resolve([]); const key = JSON.stringify([sessionId, executionId]); if (releases.has(key)) return releases.get(key); const pending = (async () => { // Block queued / late calls BEFORE detaching, and keep other owners out until cleanup is done. await revokeTabLeaseOwner(sessionId, executionId); const owned = (await listTabLeases()).filter(row => row.sessionId === sessionId && row.executionId === executionId); await releaseBrowserSysResources(sessionId, executionId, owned); clearVisualReplay(executionId); return releaseOwnedTabLeases(sessionId, executionId); })(); releases.set(key, pending); void pending.finally(() => { if (releases.get(key) === pending) releases.delete(key); }).catch(() => {}); return pending; } /** Compare only pre-read lease candidates with the authoritative offscreen runtime. */ export async function reconcileBrowserExecutions(rpc) { const candidates = await listTabLeases(); if (!candidates.length) return []; const response = await rpc({ method: 'getBrowserRuntimeState', params: {} }); if (!response?.ok || !Array.isArray(response.result?.activeExecutions)) { throw Object.assign(new Error('Cannot confirm browser execution owners; leases retained.'), { code: 'LEASE_RECONCILE_FAILED' }); } const active = response.result.activeExecutions; const owners = new Map(); for (const row of candidates) { if (!active.some(slot => slot.sessionId === row.sessionId && (!slot.executionId || slot.executionId === row.executionId))) { owners.set(JSON.stringify([row.sessionId, row.executionId]), row); } } const released = []; for (const row of owners.values()) released.push(...await releaseBrowserExecution(row.sessionId, row.executionId)); return released; } - src/agent/vnext/host/browserSysHost.js (51415 bytes)
/** * Service-worker side of browserbot-sys-v1. * Guest never sees chrome.*. This file is the syscall implementation. */ import { isInjectableTabUrl } from '../sessionWorkspace/pageContext.js'; import { SYS_EVAL_JSON_MAX, SYS_EVAL_SOURCE_MAX, SYS_FETCH_BYTES_MAX, SYS_HELP, SYS_UPLOAD_BYTES_MAX, SYS_WAIT_TIMEOUT_MS, clampWaitTimeout } from '../sessionWorkspace/browserSys.js'; import { acquireTabLease, assertTabLeaseOwnerActive } from './tabLease.js'; import { readDocumentTarget, targetError } from './documentTarget.js'; import { consumeDispatchTicket } from './dispatchTicket.js'; import { classifyRisk, needsDispatchTicket } from './riskClassify.js'; import { hashOperationPayload } from './payloadHash.js'; import { cdpCommandRunsScript } from './cdpScriptGate.js'; import { applyUploadToTab, assertUploadGuestPath, coerceUploadBytes, hashUploadPayloadBytes, putUploadStageChunk, takeUploadStage, uploadOwnerKey, UPLOAD_BYTES_MAX } from './uploadChannel.js'; function classifySysRequest(request, extras = {}) { const params = request.params && typeof request.params === 'object' ? request.params : {}; const isUpload = request.op === 'upload'; return { channel: 'sys', op: request.op, sysOp: request.op, url: extras.url || extras.frameUrl || params.url, frameUrl: extras.frameUrl || extras.url || params.url, method: isUpload ? '' : (extras.method || params.init?.method || params.method), uploadMethod: extras.uploadMethod || (isUpload ? (params.method || extras.method) : undefined), path: extras.path || params.path, bytesHash: extras.bytesHash, itemId: extras.itemId || params.itemId, artifactId: extras.artifactId || params.artifactId, code: params.code ?? params.source, action: params.action, tabId: extras.tabId ?? params.tabId, documentId: extras.documentId || params.documentId, init: params.init, bodyText: typeof params.init?.body === 'string' ? params.init.body : '', filename: extras.filename || params.filename }; } async function authorizeSysDispatch(request, extras = {}) { const classifyInput = classifySysRequest(request, extras); const classified = classifyRisk(classifyInput); if (!needsDispatchTicket(classified)) return { ok: true, classified, skipped: true }; const params = request.params && typeof request.params === 'object' ? request.params : {}; const isUpload = request.op === 'upload'; const payloadHash = await hashOperationPayload(isUpload ? { channel: 'sys', op: request.op, sysOp: request.op, url: extras.frameUrl || extras.url || params.url || '', frameUrl: extras.frameUrl || extras.url || '', method: '', path: extras.path || params.path || '', bytesHash: extras.bytesHash || '', itemId: extras.itemId || params.itemId || '', artifactId: extras.artifactId || params.artifactId || '', tabId: extras.tabId ?? params.tabId, documentId: extras.documentId || params.documentId || '' } : { channel: 'sys', op: request.op, sysOp: request.op, url: params.url || '', method: params.init?.method || params.method || '', code: params.code ?? params.source, tabId: extras.tabId ?? params.tabId, documentId: params.documentId || '' }); return consumeDispatchTicket({ ...request, payloadHash, tabId: extras.tabId ?? params.tabId, documentId: isUpload ? (extras.documentId || params.documentId || '') : (params.documentId || '') }, { ...classifyInput, params }); } const SYS_TIMEOUT_MS = 20000; const SYS_UPLOAD_TIMEOUT_MS = 120000; const SYS_CDP_TIMEOUT_MS = 60000; const SYS_CDP_RESULT_MAX = 6_000_000; const SYS_CDP_EVENT_CAP = 300; const SYS_CDP_EVENT_JSON_MAX = 8000; const CDP_PROTOCOL = '1.3'; /** @type {Set<string>} */ const cdpAttached = new Set(); const cdpAttaching = new Map(); const cdpOwners = new Map(); /** @type {Map<string, Array<{ method: string, params: unknown, ts: number }>>} */ const cdpEventBuf = new Map(); installCdpHooks(); const sysCalls = new Map(); const cancelledCalls = new Map(); export async function handleWorkspaceSys(request = {}) { const now = Date.now(); for (const [id, expiry] of cancelledCalls) if (expiry <= now) cancelledCalls.delete(id); const callId = String(request.callId || crypto.randomUUID()); const owner = `${request.sessionId || ''}:${request.executionId || ''}`; if (request.op === 'cancel') { const active = sysCalls.get(callId); if (active && active.owner !== owner) return { ok: false, code: 'SYS_DENIED', error: 'call owner mismatch' }; cancelledCalls.set(`${owner}:${callId}`, now + 120000); if (cancelledCalls.size > 512) cancelledCalls.delete(cancelledCalls.keys().next().value); active?.controller.abort(); return { ok: true, result: { callId, cancelled: true, outcome: 'unknown' } }; } if (cancelledCalls.has(`${owner}:${callId}`)) return { ok: false, code: 'SYS_ABORTED', error: 'call cancelled before dispatch' }; if (sysCalls.has(callId)) return { ok: false, code: 'SYS_BUSY', error: 'call already in flight' }; if (sysCalls.size >= 128) return { ok: false, code: 'SYS_BUSY', error: 'too many browser calls' }; const controller = new AbortController(); const maximum = request.op === 'cdp' ? SYS_CDP_TIMEOUT_MS : request.op === 'waitFor' ? SYS_WAIT_TIMEOUT_MS : request.op === 'upload' ? SYS_UPLOAD_TIMEOUT_MS : SYS_TIMEOUT_MS; const remaining = Number.isFinite(request.deadline) ? request.deadline - now : maximum; if (remaining <= 0) return { ok: false, code: 'SYS_TIMEOUT', error: 'call deadline expired before dispatch' }; let timedOut = false; const timer = setTimeout(() => { timedOut = true; controller.abort(); }, Math.min(maximum, remaining)); sysCalls.set(callId, { owner, controller }); const aborted = new Promise((resolve) => controller.signal.addEventListener('abort', () => resolve({ ok: false, code: timedOut ? 'SYS_TIMEOUT' : 'SYS_ABORTED', error: 'Browser call stopped; already dispatched page actions may have completed. Verify state before retrying.', callId, outcome: 'unknown' }), { once: true })); try { const result = await Promise.race([ dispatchWorkspaceSys({ ...request, params: { ...(request.params || {}), _signal: controller.signal } }), aborted ]); return { ...result, callId }; } finally { clearTimeout(timer); sysCalls.delete(callId); controller.abort(); } } async function dispatchWorkspaceSys(request = {}) { const op = String(request.op || ''); const params = request.params && typeof request.params === 'object' ? request.params : {}; try { if (op === 'help') return { ok: true, result: SYS_HELP }; if (op === 'capabilities') { const denied = await userScriptsDenied(); return { ok: true, result: { abi: SYS_HELP.abi, userScripts: { available: !denied, reason: denied?.error || null }, debugger: typeof chrome.debugger?.attach === 'function', screenshot: typeof chrome.tabs?.captureVisibleTab === 'function', download: typeof chrome.downloads?.download === 'function', limits: { fetchBytes: SYS_FETCH_BYTES_MAX, evalChars: SYS_EVAL_JSON_MAX, cdpChars: SYS_CDP_RESULT_MAX, uploadBytes: SYS_UPLOAD_BYTES_MAX }, cancellation: 'Extension fetch is abortable; dispatched page/CDP side effects may have completed. Verify state before retrying.' } }; } if (op === 'tabs.list') return { ok: true, result: await sysTabsList() }; if (op === 'tabs.current') return { ok: true, result: await sysTabsCurrent(params) }; if (op === 'tabs.frames') return { ok: true, result: await sysTabsFrames(params) }; // Reads above do not dispatch effects. Fence every other execution-owned call. await assertTabLeaseOwnerActive(request.sessionId, request.executionId); params._signal?.throwIfAborted(); if (op === 'upload.stage') return await sysUploadStage(params, request); if (op === 'eval') return await sysEval(params, request); if (op === 'waitFor') return await sysWaitFor(params, request); if (op === 'fetch') return await sysFetch(params, request); if (op === 'cdp') return await sysCdp(params, request); if (op === 'download') return await sysDownload(params, request); if (op === 'screenshot') return await sysScreenshot(params, request); if (op === 'upload') return await sysUpload(params, request); if (op === 'tabs.open') return await sysTabsOpen(params, request); if (op === 'tabs.navigate') return await sysTabsNavigate(params, request); if (op === 'tabs.reload') return await sysTabsReload(params, request); if (op === 'tabs.close') return await sysTabsClose(params, request); if (op === 'tabs.focus') return await sysTabsFocus(params, request); return { ok: false, code: 'BAD_INPUT', error: `unknown sys op: ${op}` }; } catch (error) { return { ok: false, code: sysFailureCode(error), error: error instanceof Error ? error.message : String(error), ...(error?.outcome ? { outcome: error.outcome } : {}) }; } } async function sysTabsList() { const tabs = await chrome.tabs.query({}); return tabs.map((tab) => publicTab(tab)); } async function sysTabsCurrent(params) { const tab = await resolveTab(params); return publicTab(tab); } async function sysTabsFrames(params) { const tab = await resolveTab(params); if (typeof chrome.webNavigation?.getAllFrames !== 'function') { return [{ frameId: 0, parentFrameId: -1, url: tab.url || '' }]; } const frames = (await chrome.webNavigation.getAllFrames({ tabId: tab.id })) || []; return frames.map((f) => ({ frameId: f.frameId, parentFrameId: f.parentFrameId, url: f.url || '', documentId: f.documentId || null, documentLifecycle: f.documentLifecycle || null, injectable: isSysInjectableUrl(f.url) })); } async function sysTabsOpen(params, request = {}) { const url = String(params.url || 'about:blank').trim(); if (!isNavigableUrl(url)) { return { ok: false, code: 'BAD_INPUT', error: 'sys.tabs.open only allows http(s) or about:blank' }; } await assertTabLeaseOwnerActive(request.sessionId, request.executionId); params._signal?.throwIfAborted(); const navGate = await authorizeSysDispatch(request, { url }); if (!navGate.ok) return navGate; const tab = await chrome.tabs.create({ url, active: params.active !== false }); if (tab?.id) { const lease = await acquireTabLease(tab.id, request.sessionId, request.executionId, 'tabs.open'); if (!lease.ok) return { ...lease, outcome: 'unknown' }; } return { ok: true, result: publicTab(tab) }; } async function sysTabsNavigate(params, request = {}) { const url = String(params.url || '').trim(); if (!isNavigableUrl(url)) { return { ok: false, code: 'BAD_INPUT', error: 'sys.tabs.navigate only allows http(s) or about:blank' }; } const tab = await resolveTab(params); const denied = await acquireSysTab(request, tab.id, 'tabs.navigate'); if (denied) return denied; params._signal?.throwIfAborted(); const navGate = await authorizeSysDispatch(request, { url, tabId: tab.id }); if (!navGate.ok) return navGate; const updated = await chrome.tabs.update(tab.id, { url }); return { ok: true, result: publicTab(updated || tab) }; } async function sysTabsReload(params, request = {}) { const tab = await resolveTab(params); const denied = await acquireSysTab(request, tab.id, 'tabs.reload'); if (denied) return denied; params._signal?.throwIfAborted(); const navGate = await authorizeSysDispatch(request, { tabId: tab.id }); if (!navGate.ok) return navGate; await chrome.tabs.reload(tab.id, { bypassCache: params.bypassCache === true }); return { ok: true, result: publicTab(await chrome.tabs.get(tab.id)) }; } async function sysTabsClose(params, request = {}) { const tab = await resolveTab(params); const denied = await acquireSysTab(request, tab.id, 'tabs.close'); if (denied) return denied; params._signal?.throwIfAborted(); const gate = await authorizeSysDispatch(request, { tabId: tab.id }); if (!gate.ok) return gate; await chrome.tabs.remove(tab.id); return { ok: true, result: { closed: tab.id } }; } async function sysTabsFocus(params, request = {}) { const tab = await resolveTab(params); const denied = await acquireSysTab(request, tab.id, 'tabs.focus'); if (denied) return denied; params._signal?.throwIfAborted(); const gate = await authorizeSysDispatch(request, { tabId: tab.id }); if (!gate.ok) return gate; if (tab.windowId != null) await chrome.windows.update(tab.windowId, { focused: true }); const updated = await chrome.tabs.update(tab.id, { active: true }); return { ok: true, result: publicTab(updated || tab) }; } /** chrome.downloads.download: this profile's cookie jar + this-machine IP. No tab Referer. Not credentials:omit. */ async function sysDownload(params, request = {}) { let url = String(params.url || '').trim(); if (params.base64 != null && String(params.base64)) { const mime = String(params.mimeType || 'application/octet-stream').replace(/[^\w.+/-]/g, '') || 'application/octet-stream'; url = `data:${mime};base64,${String(params.base64).replace(/\s+/g, '')}`; } if (!url) return { ok: false, code: 'BAD_INPUT', error: 'sys.download requires url or base64' }; if (!/^https?:/i.test(url) && !url.startsWith('data:')) { return { ok: false, code: 'BAD_INPUT', error: 'sys.download only allows http(s) or data: url' }; } const filename = sanitizeDownloadName(params.filename); const gate = await authorizeSysDispatch(request, { url, filename }); if (!gate.ok) return gate; const downloadId = await chrome.downloads.download({ url, filename: filename || undefined, saveAs: params.saveAs === true, conflictAction: 'uniquify' }); return { ok: true, result: { downloadId, filename: filename || null } }; } async function sysScreenshot(params, request = {}) { const tab = await resolveTab(params); const [active] = await chrome.tabs.query({ active: true, windowId: tab.windowId }); if (active?.id !== tab.id) return { ok: false, code: 'TAB_NOT_VISIBLE', error: 'Target tab is not visible. Focus it first or use CDP Page.captureScreenshot.' }; params._signal?.throwIfAborted(); const shotGate = await authorizeSysDispatch(request, { tabId: tab.id, url: tab.url }); if (!shotGate.ok) return shotGate; const format = String(params.format || 'png').toLowerCase() === 'jpeg' ? 'jpeg' : 'png'; const dataUrl = await chrome.tabs.captureVisibleTab(tab.windowId, { format, quality: format === 'jpeg' ? 90 : undefined }); const parsed = splitDataUrl(dataUrl); const [after] = await chrome.tabs.query({ active: true, windowId: tab.windowId }); if (after?.id !== tab.id) return { ok: false, code: 'TARGET_CHANGED', error: 'Active tab changed du - src/agent/vnext/host/browserTabGroups.js (4802 bytes)
/** * Work-tab policy — pure helpers (silent open + one Chrome tab group per session). * Chrome I/O stays in background.js. */ export const WORK_PAGE_FILES = Object.freeze([ 'docs.html', 'site.html', 'artifactPreview.html', 'preview.html' ]); /** Live office canvases the agent may lock. Never live web / picker / artifactPreview. */ export const LOCKABLE_PAGE_FILES = Object.freeze([ 'docs.html', 'site.html' ]); export const TAB_GROUP_COLORS = Object.freeze([ 'blue', 'cyan', 'green', 'purple', 'orange', 'pink', 'yellow', 'red' ]); const GROUP_TITLE_MAX = 50; export function workPageFile(url) { const raw = String(url || '').trim(); if (!raw) return ''; let pathname = raw.split('#')[0].split('?')[0]; try { pathname = new URL(raw).pathname; } catch { /* keep stripped path */ } const file = pathname.split('/').pop() || ''; return WORK_PAGE_FILES.includes(file) ? file : ''; } export function isBrowserBotPageUrl(url) { if (!workPageFile(url)) return false; const raw = String(url || ''); return /^chrome-extension:/i.test(raw) || /\/src\/preview\//.test(raw); } export function isLockableWorkPageUrl(url) { const file = workPageFile(url); if (!LOCKABLE_PAGE_FILES.includes(file)) return false; return isBrowserBotPageUrl(url); } /** * Session-scoped work-tab lock. Live web / huaban / picker / preview.html never lock. * Empty sessionId or a mismatched lockedSessionId stays unlocked. */ export function shouldLockWorkTab(opts = {}) { const lockedSid = String(opts.lockedSessionId || '').trim(); if (!lockedSid) return false; const url = String(opts.url || ''); if (!isLockableWorkPageUrl(url)) return false; const sid = String(opts.sessionId || sessionIdFromBrowserBotUrl(url) || '').trim(); return sid === lockedSid; } export function sessionIdFromBrowserBotUrl(url) { const raw = String(url || ''); try { return String(new URL(raw).searchParams.get('sessionId') || '').trim(); } catch { const m = raw.match(/[?&]sessionId=([^&#]*)/); if (!m) return ''; try { return decodeURIComponent(m[1]).trim(); } catch { return String(m[1] || '').trim(); } } } /** * Agent / preview / canvas writes default to silent. * User sidepanel open and reveal-on-page stay focused. * Explicit `focus` always wins. */ export function shouldFocusBrowserBotTab(opts = {}) { if (opts.focus === true) return true; if (opts.focus === false) return false; const reason = String(opts.reason || opts.source || '').toLowerCase(); return reason === 'user' || reason === 'reveal'; } /** * Capture-page reveal never joins. Empty sessionId never groups. * Live-web URLs stay out of the session group. */ export function shouldJoinSessionGroup(opts = {}) { const reason = String(opts.reason || opts.source || '').toLowerCase(); if (reason === 'reveal') return false; const sid = String(opts.sessionId || sessionIdFromBrowserBotUrl(opts.url) || '').trim(); if (!sid) return false; const url = String(opts.url || ''); if (url && !isBrowserBotPageUrl(url)) return false; return true; } export function tabCreateProps({ url, focus } = {}) { return { url: String(url || ''), active: focus === true }; } /** Agent/preview/canvas writes must never call chrome.windows.update({ focused: true }). */ export function shouldFocusChromeWindow() { return false; } export function sessionTaskIndex(opts = {}) { const n = Number(opts.index); if (Number.isFinite(n) && n > 0) return Math.floor(n); const fromTitle = String(opts.title || opts.name || '').match(/(?:任务|Task|会话|Session)\s*(\d+)/i); if (fromTitle) return Number(fromTitle[1]); const fromSid = String(opts.sessionId || '').match(/(\d+)\s*$/); if (fromSid) return Number(fromSid[1]); return 1; } export function sessionGroupTitle(opts = {}) { const sid = String(opts.sessionId || '').trim(); const explicit = String(opts.title || opts.name || '') .replace(/^💬\s*/, '') .replace(/\s+/g, ' ') .trim(); if (explicit && explicit !== sid) return clipGroupTitle(explicit); const n = sessionTaskIndex(opts); return opts.lang === 'en' ? `Task ${n}` : `任务 ${n}`; } export function sessionGroupColor(sessionId) { const s = String(sessionId || ''); let h = 0; for (let i = 0; i < s.length; i++) h = (Math.imul(31, h) + s.charCodeAt(i)) >>> 0; return TAB_GROUP_COLORS[h % TAB_GROUP_COLORS.length]; } export function sessionGroupUpdate(opts = {}) { return { title: sessionGroupTitle(opts), color: sessionGroupColor(opts.sessionId), collapsed: opts.collapsed !== false }; } function clipGroupTitle(title) { const t = String(title || '').trim(); if (t.length <= GROUP_TITLE_MAX) return t; return `${t.slice(0, GROUP_TITLE_MAX - 1)}…`; } - src/agent/vnext/host/callJournal.js (19637 bytes)
/** * Durable call journal. Not task progress, not sessionAudit, not SW memory. * Write-ahead: prepared+authorized must succeed before SW dispatch. * Exactly-once for arbitrary web pages does not exist; never blind-replay. */ import { JOURNAL_DB_NAME, JOURNAL_SCHEMA } from '../../../brand.js'; export { JOURNAL_SCHEMA }; export const JOURNAL_DB = JOURNAL_DB_NAME; export const JOURNAL_DB_VERSION = 1; export const JOURNAL_STORE = 'operations'; export const JOURNAL_APPROVALS_STORE = 'approvals'; export const JOURNAL_SESSION_CAP = 200; export const JOURNAL_ANOMALY_CAP = 50; function journalUnavailable(message) { return Object.assign(new Error(message), { code: 'JOURNAL_UNAVAILABLE' }); } function hasJournalStores(db) { const names = db?.objectStoreNames; return !!(names && names.contains(JOURNAL_STORE) && names.contains(JOURNAL_APPROVALS_STORE)); } function ensureJournalSchema(db) { if (!db.objectStoreNames.contains(JOURNAL_STORE)) { const store = db.createObjectStore(JOURNAL_STORE, { keyPath: 'operationId' }); store.createIndex('sessionId', 'sessionId', { unique: false }); store.createIndex('state', 'state', { unique: false }); } if (!db.objectStoreNames.contains(JOURNAL_APPROVALS_STORE)) { const store = db.createObjectStore(JOURNAL_APPROVALS_STORE, { keyPath: 'approvalId' }); store.createIndex('sessionId', 'sessionId', { unique: false }); } } /** * Attach IDBRequest handlers only when `req` is a real request. * Undefined/closed/aborted requests must be JOURNAL_UNAVAILABLE, never TypeError. */ export function requestValue(req, transaction) { return new Promise((resolve, reject) => { let settled = false; const ok = (value) => { if (settled) return; settled = true; resolve(value); }; const fail = (message = 'journal idb request failed') => { if (settled) return; settled = true; reject(journalUnavailable(message)); }; if (!req || typeof req !== 'object') { fail(); return; } try { req.onsuccess = () => ok(req.result); req.onerror = () => fail(); } catch { fail(); return; } if (!transaction) return; const prevAbort = transaction.onabort; transaction.onabort = (event) => { try { if (typeof prevAbort === 'function') prevAbort.call(transaction, event); } finally { fail('journal idb write failed'); } }; }); } export const JOURNAL_STATES = Object.freeze([ 'prepared', 'awaiting_approval', 'authorized', 'dispatched', 'succeeded', 'failed', 'unknown', 'verified', 'needs_human' ]); const ALLOWED = { prepared: new Set(['awaiting_approval', 'authorized', 'failed']), awaiting_approval: new Set(['authorized', 'failed']), authorized: new Set(['dispatched', 'failed']), dispatched: new Set(['succeeded', 'failed', 'unknown']), succeeded: new Set(['verified', 'needs_human']), failed: new Set(['verified', 'needs_human']), unknown: new Set(['verified', 'needs_human']), verified: new Set(), needs_human: new Set() }; function nowTs() { return Date.now(); } function clone(row) { return row ? JSON.parse(JSON.stringify(row)) : null; } function transitionError(from, to) { const err = new Error(`illegal journal transition ${from} → ${to}`); err.code = 'JOURNAL_ILLEGAL'; return err; } export function assertJournalTransition(from, to) { if (from === to) return true; if (!ALLOWED[from] || !ALLOWED[from].has(to)) throw transitionError(from, to); return true; } function applyStateTimes(row, state, ts) { row.ts = row.ts && typeof row.ts === 'object' ? { ...row.ts } : {}; if (state === 'prepared') row.ts.prepared = ts; if (state === 'awaiting_approval') row.ts.awaitingApproval = ts; if (state === 'authorized') row.ts.authorized = ts; if (state === 'dispatched') row.ts.dispatched = ts; if (state === 'succeeded' || state === 'failed' || state === 'unknown') row.ts.result = ts; if (state === 'verified' || state === 'needs_human') row.ts.verified = ts; } export function createMemoryCallJournal() { const operations = new Map(); const approvals = new Map(); let revision = 0; let failWrites = false; function pruneSession(sessionId) { const sid = String(sessionId || ''); const rows = [...operations.values()].filter((row) => row.sessionId === sid); if (rows.length <= JOURNAL_SESSION_CAP) return; rows.sort((a, b) => (a.updatedAt || 0) - (b.updatedAt || 0)); const drop = rows.length - JOURNAL_SESSION_CAP; for (let i = 0; i < drop; i += 1) operations.delete(rows[i].operationId); } return { kind: 'memory', _operations: operations, _approvals: approvals, failNextWrite() { failWrites = true; }, async put(row) { if (failWrites) { failWrites = false; const err = new Error('journal write failed'); err.code = 'JOURNAL_UNAVAILABLE'; throw err; } const next = clone(row); next.revision = ++revision; next.updatedAt = nowTs(); operations.set(next.operationId, next); pruneSession(next.sessionId); return clone(next); }, async get(operationId) { return clone(operations.get(String(operationId || ''))); }, async listBySession(sessionId) { const sid = String(sessionId || ''); return [...operations.values()] .filter((row) => row.sessionId === sid) .sort((a, b) => (a.updatedAt || 0) - (b.updatedAt || 0)) .map(clone); }, async deleteSession(sessionId) { const sid = String(sessionId || ''); for (const [id, row] of [...operations.entries()]) { if (row.sessionId === sid) operations.delete(id); } for (const [id, rec] of [...approvals.entries()]) { if (rec.sessionId === sid) approvals.delete(id); } }, async putApproval(rec) { if (failWrites) { failWrites = false; const err = new Error('journal write failed'); err.code = 'JOURNAL_UNAVAILABLE'; throw err; } approvals.set(rec.approvalId, clone(rec)); return clone(rec); }, async getApproval(approvalId) { return clone(approvals.get(String(approvalId || ''))); }, async listApprovals(sessionId) { const sid = String(sessionId || ''); return [...approvals.values()].filter((rec) => !sid || rec.sessionId === sid).map(clone); }, async scan(predicate) { return [...operations.values()].filter((row) => predicate(row)).map(clone); } }; } function openJournalDatabase(idb, version) { return new Promise((resolve, reject) => { let req; try { req = idb.open(JOURNAL_DB, version); } catch { reject(journalUnavailable('journal idb open failed')); return; } if (!req) { reject(journalUnavailable('journal idb open failed')); return; } req.onupgradeneeded = () => { try { ensureJournalSchema(req.result); } catch { reject(journalUnavailable('journal idb open failed')); } }; req.onsuccess = () => resolve(req.result); req.onerror = () => reject(journalUnavailable('journal idb open failed')); }); } export async function openIndexedDbCallJournal(indexedDBImpl) { const idb = indexedDBImpl || globalThis.indexedDB; if (!idb) { throw journalUnavailable('indexedDB unavailable'); } let db = await openJournalDatabase(idb, JOURNAL_DB_VERSION); if (!hasJournalStores(db)) { const nextVersion = Math.max(Number(db.version) || JOURNAL_DB_VERSION, JOURNAL_DB_VERSION) + 1; try { db.close(); } catch { /* ignore */ } db = await openJournalDatabase(idb, nextVersion); } if (!hasJournalStores(db)) { try { db.close(); } catch { /* ignore */ } throw journalUnavailable('journal idb open failed'); } function tx(storeName, mode, fn) { return new Promise((resolve, reject) => { let settled = false; const finishOk = (value) => { if (settled) return; settled = true; resolve(value); }; const finishErr = (error) => { if (settled) return; settled = true; reject( error?.code === 'JOURNAL_UNAVAILABLE' ? error : journalUnavailable(error?.message || 'journal idb write failed') ); }; let transaction; try { transaction = db.transaction(storeName, mode); } catch { finishErr(journalUnavailable('journal idb write failed')); return; } if (!transaction) { finishErr(journalUnavailable('journal idb write failed')); return; } let store; try { store = transaction.objectStore(storeName); } catch { finishErr(journalUnavailable('journal idb write failed')); return; } let pending; try { pending = fn(store); } catch (error) { finishErr(error); return; } const prevAbort = transaction.onabort; transaction.onabort = (event) => { try { if (typeof prevAbort === 'function') prevAbort.call(transaction, event); } finally { finishErr(journalUnavailable('journal idb write failed')); } }; transaction.onerror = () => finishErr(journalUnavailable('journal idb write failed')); if (pending && typeof pending.then === 'function') { pending.then(undefined, finishErr); transaction.oncomplete = () => pending.then(finishOk, finishErr); return; } const value = requestValue(pending); // Handle reject in this turn so Chrome does not report uncaught TypeError / JOURNAL_UNAVAILABLE. value.then(undefined, finishErr); transaction.oncomplete = () => value.then(finishOk, finishErr); }); } return { kind: 'idb', async put(row) { await tx(JOURNAL_STORE, 'readwrite', (store) => store.put(row)); return row; }, async get(operationId) { return tx(JOURNAL_STORE, 'readonly', (store) => store.get(String(operationId || ''))); }, async listBySession(sessionId) { return tx(JOURNAL_STORE, 'readonly', (store) => { const index = store.index('sessionId'); return index.getAll(String(sessionId || '')); }); }, async deleteSession(sessionId) { const rows = await this.listBySession(sessionId); if (rows?.length) { await tx(JOURNAL_STORE, 'readwrite', (store) => { let last = null; for (const row of rows) last = store.delete(row.operationId); return last; }); } const approvals = await this.listApprovals(sessionId); if (approvals?.length) { await tx(JOURNAL_APPROVALS_STORE, 'readwrite', (store) => { let last = null; for (const rec of approvals) last = store.delete(rec.approvalId); return last; }); } }, async putApproval(rec) { await tx(JOURNAL_APPROVALS_STORE, 'readwrite', (store) => store.put(rec)); return rec; }, async getApproval(approvalId) { return tx(JOURNAL_APPROVALS_STORE, 'readonly', (store) => store.get(String(approvalId || ''))); }, async listApprovals(sessionId) { const sid = String(sessionId || ''); const all = await tx(JOURNAL_APPROVALS_STORE, 'readonly', (store) => store.getAll()); return (all || []).filter((rec) => !sid || rec.sessionId === sid); }, async scan(predicate) { const all = await tx(JOURNAL_STORE, 'readonly', (store) => store.getAll()); return (all || []).filter((row) => predicate(row)); } }; } export function createUnavailableCallJournal() { const fail = async () => { const err = new Error('Call journal is unavailable'); err.code = 'JOURNAL_UNAVAILABLE'; throw err; }; return { kind: 'unavailable', commit: fail, update: fail, get: async () => null, listBySession: async () => [], deleteSession: async () => {}, putApproval: fail, getApproval: async () => null, listApprovals: async () => [], findDuplicate: async () => null, recoverOnStartup: async () => [], listAnomalies: async () => [], exportSession: fail }; } export function createCallJournal(backend) { const store = backend || createMemoryCallJournal(); async function commit(partial) { try { const ts = nowTs(); const row = { schema: JOURNAL_SCHEMA, operationId: String(partial.operationId || ''), requestId: String(partial.requestId || ''), sessionId: String(partial.sessionId || ''), executionId: String(partial.executionId || ''), taskId: partial.taskId || null, intent: partial.intent || { channel: 'action', op: '' }, risk: partial.risk, confidence: partial.confidence, accessMode: partial.accessMode, rawEscape: partial.rawEscape || 'n/a', reason: partial.reason || '', target: partial.target || {}, payloadHash: String(partial.payloadHash || ''), payloadRef: partial.payloadRef, preconditions: Array.isArray(partial.preconditions) ? partial.preconditions : [], postconditions: Array.isArray(partial.postconditions) ? partial.postconditions : [], approvalId: partial.approvalId, approvalReceipt: partial.approvalReceipt, policySnapshot: partial.policySnapshot || null, state: partial.state || 'prepared', ts: {}, receipt: partial.receipt, error: partial.error, outcome: partial.outcome, revision: 0, updatedAt: ts }; applyStateTimes(row, row.state, ts); if (!row.operationId) { const err = new Error('operationId required'); err.code = 'JOURNAL_UNAVAILABLE'; throw err; } return await store.put(row); } catch (error) { if (error?.code === 'JOURNAL_UNAVAILABLE') throw error; throw journalUnavailable(error?.message || 'journal write failed'); } } async function update(operationId, patch = {}) { const current = await store.get(operationId); if (!current) { const err = new Error('journal row missing'); err.code = 'JOURNAL_UNAVAILABLE'; throw err; } const nextState = patch.state || current.state; if (patch.state) assertJournalTransition(current.state, patch.state); const ts = nowTs(); const next = { ...current, ...patch, state: nextState, ts: { ...current.ts, ...(patch.ts || {}) }, error: patch.error !== undefined ? patch.error : current.error, receipt: patch.receipt !== undefined ? patch.receipt : current.receipt, outcome: patch.outcome !== undefined ? patch.outcome : current.outcome }; applyStateTimes(next, nextState, ts); try { return await store.put(next); } catch (err - src/agent/vnext/host/cdpScriptGate.js (3749 bytes)
/** * Which guest sys.cdp commands can run script in the page. * * Those commands get the same opt-in as sys.eval: the optional userScripts permission, * granted and usable. Without it the host refuses them with PERMISSION_REQUIRED, so * "page scripts need the user's opt-in" stays true when the debugger pipe is open. * * Allowlist, not denylist. CDP has hundreds of methods and new ones ship with Chrome; * many reach script indirectly (breakpoint conditions, rewritten responses, HTML with * event handlers, child-target sessions). A denylist would quietly let the next one * through. So a method passes without userScripts only if its domain is listed below * as observation / trusted input, and it is not one of the listed script-capable * exceptions inside those domains. Everything else (Runtime, Debugger, Fetch, * unknown domains) is treated as script. * * Only guest-originated sys.cdp sends go through this. The host's own debugger calls * (pointer clicks, layout metrics, CDP upload, the attach-reclaim probe) never do. */ const NON_SCRIPT_DOMAINS = new Set([ 'Accessibility', 'Audits', 'CSS', 'CacheStorage', 'DOM', 'DOMSnapshot', 'DOMStorage', 'Emulation', 'IO', 'IndexedDB', 'Input', 'LayerTree', 'Log', 'Network', 'Overlay', 'Page', 'Performance', 'Security', 'Storage', 'Target' ]); /** Runtime calls that read or tidy state and run no page code. */ const NON_SCRIPT_RUNTIME = new Set([ 'Runtime.enable', 'Runtime.disable', 'Runtime.getProperties', 'Runtime.releaseObject', 'Runtime.releaseObjectGroup', 'Runtime.getHeapUsage', 'Runtime.getIsolateId', 'Runtime.discardConsoleEntries' ]); /** Inside an allowed domain, these run or plant script whatever their params. */ const SCRIPT_METHODS = new Set([ 'Page.addScriptToEvaluateOnNewDocument', 'Page.addScriptToEvaluateOnLoad', 'Page.createIsolatedWorld', 'Page.setDocumentContent', 'Page.setBypassCSP', 'DOM.setOuterHTML', 'Network.setRequestInterception', 'Network.continueInterceptedRequest', 'Target.attachToTarget', 'Target.attachToBrowserTarget', 'Target.setAutoAttach', 'Target.sendMessageToTarget', 'Target.exposeDevToolsProtocol' ]); const HANDLER_ATTRIBUTE = /^\s*(on[a-z]|srcdoc\b)/i; const SCRIPT_TEXT = /(^|[\s"'/])(on[a-z]+|srcdoc)\s*=|javascript\s*:/i; /** * @param {string} method CDP method, e.g. "Runtime.evaluate" * @param {object} [params] CDP params as the guest sent them * @returns {boolean} true when the command needs the userScripts opt-in */ export function cdpCommandRunsScript(method, params) { const name = String(method || ''); const p = params && typeof params === 'object' ? params : {}; const domain = name.split('.')[0]; if (domain === 'Runtime') return !NON_SCRIPT_RUNTIME.has(name); if (!NON_SCRIPT_DOMAINS.has(domain)) return true; if (SCRIPT_METHODS.has(name)) return true; if (name === 'Page.navigate' || name === 'Target.createTarget') return !isPlainPageUrl(p.url); if (name === 'DOM.setAttributeValue') { return HANDLER_ATTRIBUTE.test(String(p.name || '')) || /javascript\s*:/i.test(stripControls(p.value)); } if (name === 'DOM.setAttributesAsText') { return HANDLER_ATTRIBUTE.test(String(p.name || '')) || SCRIPT_TEXT.test(stripControls(p.text)); } return false; } /** http(s) and about:blank only; javascript:, data:, blob: and the rest count as script. */ function isPlainPageUrl(url) { const s = stripControls(url).trim(); if (s === 'about:blank') return true; try { const u = new URL(s); return u.protocol === 'http:' || u.protocol === 'https:'; } catch { return false; } } function stripControls(value) { return String(value ?? '').replace(/[\x00-\x1f\x7f]/g, ''); } - src/agent/vnext/host/dispatchTicket.js (8611 bytes)
/** * One-shot dispatch tickets in chrome.storage.session. * SW consumes; offscreen issues. Not a journal. * SW recomputes payload hash from the resolved operation and never trusts the message hash. */ import { classifyRisk, needsDispatchTicket, ticketBindingFields, RISK_PAYMENT, CONF_KNOWN, RISK_DELETE } from './riskClassify.js'; export const DISPATCH_TICKETS_KEY = 'browserbot_dispatch_tickets_v1'; let installed = null; let mapQueue = Promise.resolve(); function withTicketMap(fn) { const work = mapQueue.then(fn, fn); mapQueue = work.catch(() => {}); return work; } export function installDispatchTicketStorage(area) { installed = area || null; } export function dispatchTicketStorageInstalled() { return !!installed; } export function resetDispatchTicketStorage() { installed = null; } function sessionArea() { if (installed) return installed; try { return typeof chrome !== 'undefined' ? chrome.storage?.session : null; } catch { return null; } } async function readMap() { const area = sessionArea(); if (!area || typeof area.get !== 'function') { const err = new Error('dispatch ticket storage unavailable'); err.code = 'TICKET_REQUIRED'; throw err; } const bag = await area.get(DISPATCH_TICKETS_KEY); const raw = bag?.[DISPATCH_TICKETS_KEY] ?? bag; return raw && typeof raw === 'object' && !Array.isArray(raw) ? { ...raw } : {}; } async function writeMap(map) { const area = sessionArea(); if (!area || typeof area.set !== 'function') { const err = new Error('dispatch ticket storage unavailable'); err.code = 'TICKET_REQUIRED'; throw err; } await area.set({ [DISPATCH_TICKETS_KEY]: map }); } export function createMemoryTicketStorage(seed = {}) { const data = { [DISPATCH_TICKETS_KEY]: { ...seed } }; return { get: async (key) => ({ [key]: data[key] }), set: async (values) => { Object.assign(data, values); }, remove: async (key) => { delete data[key]; }, _data: data }; } function requireTicketFields(ticket) { const missing = []; if (!String(ticket?.operationId || '')) missing.push('operationId'); if (!String(ticket?.sessionId || '')) missing.push('sessionId'); if (!String(ticket?.executionId || '')) missing.push('executionId'); if (!String(ticket?.payloadHash || '')) missing.push('payloadHash'); if (!String(ticket?.nonce || '')) missing.push('nonce'); if (missing.length) { const err = new Error(`ticket missing ${missing.join(', ')}`); err.code = 'TICKET_REQUIRED'; throw err; } } export async function putDispatchTicket(ticket) { requireTicketFields(ticket); const operationId = String(ticket.operationId); return withTicketMap(async () => { const map = await readMap(); const row = { operationId, nonce: String(ticket.nonce), sessionId: String(ticket.sessionId), executionId: String(ticket.executionId), risk: String(ticket.risk || ''), payloadHash: String(ticket.payloadHash), exp: Number(ticket.exp) || 0 }; if (ticket.tabId != null && ticket.tabId !== '') row.tabId = Number(ticket.tabId); if (ticket.documentId) row.documentId = String(ticket.documentId); map[operationId] = row; await writeMap(map); return map[operationId]; }); } export async function dropDispatchTicket(operationId) { const id = String(operationId || ''); if (!id) return false; try { return await withTicketMap(async () => { const map = await readMap(); if (!map[id]) return false; delete map[id]; await writeMap(map); return true; }); } catch { return false; } } export async function dropTicketsForExecution(sessionId, executionId) { const sid = String(sessionId || ''); const eid = String(executionId || ''); if (!sid || !eid) return 0; try { return await withTicketMap(async () => { const map = await readMap(); let n = 0; for (const [id, ticket] of Object.entries(map)) { if (String(ticket?.sessionId || '') === sid && String(ticket?.executionId || '') === eid) { delete map[id]; n += 1; } } if (n) await writeMap(map); return n; }); } catch { return 0; } } export async function listDispatchTickets() { try { const map = await readMap(); return Object.values(map); } catch { return []; } } export async function expireDispatchTickets(now = Date.now()) { try { await withTicketMap(async () => { const map = await readMap(); let changed = false; for (const [id, ticket] of Object.entries(map)) { if (!ticket || Number(ticket.exp) <= now) { delete map[id]; changed = true; } } if (changed) await writeMap(map); }); } catch { /* ignore */ } } function mismatch(code, message) { return { ok: false, code, error: message }; } function requiredPresent(value) { if (value == null) return false; if (typeof value === 'string' && !value.trim()) return false; return true; } /** * SW last door. Reclassifies. Known payment always denied. * Hash is always the host-computed hash of the resolved operation. */ export async function consumeDispatchTicket(request, classifyInput, now = Date.now()) { const classified = classifyRisk(classifyInput || request || {}); if (classified.risk === RISK_PAYMENT && classified.confidence === CONF_KNOWN) { if (request?.operationId) await dropDispatchTicket(request.operationId); return mismatch('PAYMENT_DENIED', 'Known payment is never dispatched.'); } if (!needsDispatchTicket(classified)) { return { ok: true, classified, ticket: null, skipped: true }; } const operationId = String(request?.operationId || ''); if (!operationId) { return mismatch('TICKET_REQUIRED', 'Mutating dispatch requires a one-shot ticket.'); } const computedHash = String(request?.payloadHash || ''); if (!computedHash) { return mismatch('TICKET_REQUIRED', 'Host must recompute the payload hash before consume.'); } let ticket; try { ticket = await withTicketMap(async () => { const map = await readMap(); const found = map[operationId]; if (!found) return null; delete map[operationId]; await writeMap(map); return found; }); } catch (error) { return mismatch(error?.code || 'TICKET_REQUIRED', error?.message || 'ticket storage unavailable'); } if (!ticket) return mismatch('TICKET_REQUIRED', 'Dispatch ticket is missing or already used.'); if (Number(ticket.exp) <= now) return mismatch('APPROVAL_EXPIRED', 'Dispatch ticket expired.'); const bind = ticketBindingFields(classifyInput || request || {}); if (bind.sessionId && String(ticket.sessionId || '') !== String(request.sessionId || '')) { return mismatch('APPROVAL_MISMATCH', 'Ticket session does not match.'); } if (bind.executionId && String(ticket.executionId || '') !== String(request.executionId || '')) { return mismatch('APPROVAL_MISMATCH', 'Ticket execution does not match.'); } if (bind.payloadHash) { if (!requiredPresent(ticket.payloadHash) || ticket.payloadHash !== computedHash) { return mismatch('APPROVAL_MISMATCH', 'Ticket payload hash does not match.'); } } if (bind.nonce) { if (!requiredPresent(ticket.nonce) || !requiredPresent(request.ticketNonce) || String(ticket.nonce) !== String(request.ticketNonce)) { return mismatch('APPROVAL_MISMATCH', 'Ticket nonce does not match.'); } } if (bind.tabId) { if (!requiredPresent(ticket.tabId) || request.tabId == null || Number(ticket.tabId) !== Number(request.tabId)) { return mismatch('APPROVAL_MISMATCH', 'Ticket tab does not match.'); } } if (bind.documentId) { if (!requiredPresent(ticket.documentId) || !requiredPresent(request.documentId) || String(ticket.documentId) !== String(request.documentId)) { return mismatch('APPROVAL_MISMATCH', 'Ticket document does not match.'); } } if (ticket.risk && classified.risk && String(ticket.risk) !== String(classified.risk)) { return mismatch('APPROVAL_MISMATCH', 'Ticket risk does not match host classification.'); } if (classified.risk === RISK_DELETE && classified.confidence === CONF_KNOWN && ticket.risk !== RISK_DELETE) { return mismatch('APPROVAL_MISMATCH', 'Known delete requires a delete approval ticket.'); } return { ok: true, classified, ticket, consumed: true }; } export async function peekDispatchTicket(operationId) { const map = await readMap(); return map[String(operationId || '')] || null; } - src/agent/vnext/host/documentTarget.js (2228 bytes)
/** Browser-owned document identity. URL equality alone cannot identify a document. */ export function targetError(code, message) { return Object.assign(new Error(message), { code }); } export async function readDocumentTarget(chromeApi, tabId, frameId = 0, expected = {}) { const id = Number(tabId); const frame = Number(frameId); if (!Number.isInteger(id) || id <= 0 || !Number.isInteger(frame) || frame < 0) { throw targetError('NEED_PAGE', 'An explicit tabId and valid frameId are required.'); } if (typeof chromeApi.webNavigation?.getFrame !== 'function') { throw targetError('DOCUMENT_UNAVAILABLE', 'Cannot identify the current document.'); } const live = await chromeApi.webNavigation.getFrame({ tabId: id, frameId: frame }); if (!live?.documentId || (live.documentLifecycle && live.documentLifecycle !== 'active')) { throw targetError('DOCUMENT_UNAVAILABLE', 'The target document is unavailable or not active.'); } if ((expected.documentId && live.documentId !== expected.documentId) || (expected.url && live.url !== expected.url)) { throw targetError('TARGET_CHANGED', 'The document or route changed. Observe it again before acting.'); } return { tabId: id, frameId: frame, documentId: live.documentId, url: String(live.url || '') }; } /** Revisions are opaque and never repeat after SW restart (unlike t1, t2, ...). */ export function createPageSnapshotRegistry({ token = () => crypto.randomUUID() } = {}) { const snapshots = new Map(); return { capture(tabId, frames) { const snapshot = { rev: token(), frames: frames.map(f => ({ frameId: f.frameId, documentId: f.documentId, url: f.url || '' })) }; snapshots.set(Number(tabId), snapshot); return structuredClone(snapshot); }, get(tabId) { return structuredClone(snapshots.get(Number(tabId)) || null); }, invalidate(tabId) { snapshots.delete(Number(tabId)); }, require(tabId, rev) { const snapshot = snapshots.get(Number(tabId)); if (!snapshot || !rev || snapshot.rev !== String(rev)) { throw targetError('STALE_REF', 'Snapshot first; use the latest rev for every page mutation.'); } return structuredClone(snapshot); } }; } - src/agent/vnext/skills/page-restyle/index.js (391 bytes)
import { parseSkillMd } from '../parseSkillMd.js'; import { SKILL_MD } from './skillSource.js'; const { meta, body } = parseSkillMd(SKILL_MD); export const skillId = 'page-restyle'; export const skill = { id: skillId, name: meta.name || 'Page Restyle', description: meta.description, instructions: body, libraries: [], resources: {}, templates: {} }; export default skill; - src/agent/vnext/skills/page-restyle/SKILL.md (357 bytes)
--- name: Page Restyle description: User wants the current http(s) tab restyled in place — hide named clutter, a calmer reading layout, contrast, or a floating TOC. --- The live document is the outcome. One tagged `<style>`; undo = remove that node. Idempotent id (`#paw-restyle`). After navigate or SPA remount, inject again. This is not a saved clone.
- src/agent/vnext/skills/page-restyle/skillSource.js (443 bytes)
// Generated from SKILL.md — do not edit by hand. export const SKILL_MD = "---\nname: Page Restyle\ndescription: User wants the current http(s) tab restyled in place — hide named clutter, a calmer reading layout, contrast, or a floating TOC.\n---\n\nThe live document is the outcome. One tagged `<style>`; undo = remove that node. Idempotent id (`#paw-restyle`). After navigate or SPA remount, inject again. This is not a saved clone.\n";
- src/agent/vnext/skills/parseSkillMd.js (974 bytes)
/** * Parse industry-style SKILL.md: YAML-ish frontmatter + markdown body. * Minimal parser (no deps) for extension + Node. * * --- * name: ... * description: ... * libraries: a, b * --- * instructions body */ /** * @param {string} raw * @returns {{ meta: Record<string, string>, body: string }} */ export function parseSkillMd(raw) { const text = String(raw || '').replace(/^\uFEFF/, ''); if (!text.startsWith('---')) { return { meta: {}, body: text.trim() }; } const end = text.indexOf('\n---', 3); if (end < 0) { return { meta: {}, body: text.trim() }; } const fm = text.slice(3, end).trim(); const body = text.slice(end + 4).replace(/^\r?\n/, '').trim(); /** @type {Record<string, string>} */ const meta = {}; for (const line of fm.split(/\r?\n/)) { const m = /^([A-Za-z0-9_-]+)\s*:\s*(.*)$/.exec(line.trim()); if (!m) continue; meta[m[1]] = m[2].trim().replace(/^["']|["']$/g, ''); } return { meta, body }; } - src/agent/vnext/skills/registry.js (7373 bytes)
/** * Skill registry — loads **folder packages** (industry layout). * * Each skill is a directory: * skills/<id>/ * SKILL.md # name + description (semantic when-to-use) + playbook * skillSource.js # bundled SKILL.md text for Chrome (no .md import) * index.js # package entry → { skill } * templates/ # reusable assets * scripts/ # reusable helpers / hints * * Skills are playbooks, NOT tools. Model surface stays inspect / acquire / run. * Host never keyword-matches user text; description is for model semantic routing. */ import { skill as pageRestyle } from './page-restyle/index.js'; import { skill as siteToolReuse } from './site-tool-reuse/index.js'; /** Permanent id aliases so inspect / skillStore overlays keyed by old ids still resolve. */ export const SKILL_ID_ALIASES = Object.freeze({}); /** @type {Map<string, SkillDef>} */ const SKILLS = new Map(); /** * @param {string} id * @returns {string} */ export function resolveSkillId(id) { const raw = String(id || ''); return SKILL_ID_ALIASES[raw] || raw; } /** * Alias ids that point at a canonical packaged skill. * @param {string} canonicalId * @returns {string[]} */ export function skillIdAliases(canonicalId) { const want = String(canonicalId || ''); return Object.entries(SKILL_ID_ALIASES) .filter(([, canonical]) => canonical === want) .map(([alias]) => alias); } /** * @typedef {object} SkillDef * @property {string} id * @property {string} [name] * @property {string} description * @property {string|((ctx?: object) => string)} instructions * @property {Record<string, string>} [templates] * @property {Record<string, string>} [resources] * @property {string[]} [resourcePaths] inspect listing when files live on disk * @property {string} [packagePrefix] allow inspect of any safe path under this prefix * @property {string[]} [libraries] * @property {string} [root] package folder id */ /** * @param {SkillDef} def */ export function registerSkill(def) { if (!def || !def.id) throw new Error('registerSkill: id required'); if (def.instructions == null) throw new Error('registerSkill: instructions required'); const description = String(def.description || '').trim(); if (!description) { throw new Error(`registerSkill: description required for semantic routing (${def.id})`); } SKILLS.set(String(def.id), { id: String(def.id), name: def.name || def.id, description, instructions: def.instructions, templates: def.templates && typeof def.templates === 'object' ? { ...def.templates } : {}, resources: def.resources && typeof def.resources === 'object' ? { ...def.resources } : {}, resourcePaths: Array.isArray(def.resourcePaths) ? def.resourcePaths.slice() : [], packagePrefix: String(def.packagePrefix || ''), libraries: Array.isArray(def.libraries) ? def.libraries.slice() : [], root: def.root || def.id }); return SKILLS.get(String(def.id)); } /** * @param {string} id * @returns {SkillDef|null} */ export function getSkill(id) { return SKILLS.get(resolveSkillId(id)) || null; } /** * @returns {string[]} */ export function listSkills() { return [...SKILLS.keys()]; } /** * Catalog for model-facing semantic routing. * @returns {Array<{id:string,name:string,description:string,resourcePaths:string[]}>} */ export function listPackagedSkillCatalog() { return [...SKILLS.values()].map((s) => ({ id: s.id, name: s.name || s.id, description: s.description, resourcePaths: skillResourcePaths(s), origin: 'packaged' })); } export function listSkillCatalog() { return listPackagedSkillCatalog(); } /** * @param {string} id * @param {object} [ctx] * @returns {string} */ export function loadSkillInstructions(id, ctx = {}) { const skill = getSkill(id); if (!skill) return ''; if (typeof skill.instructions === 'function') { return String(skill.instructions(ctx) || ''); } return String(skill.instructions || ''); } /** * @param {SkillDef} skill * @returns {string[]} */ export function skillResourcePaths(skill) { if (!skill) return []; if (Array.isArray(skill.resourcePaths) && skill.resourcePaths.length) { return skill.resourcePaths.slice(); } return Object.keys(skill.resources || {}); } /** * @param {string} raw * @returns {string} */ function safeSkillRelPath(raw) { const n = String(raw || '').replace(/\\/g, '/').replace(/^\/+/, ''); if (!n || n.includes('\0')) return ''; const parts = n.split('/'); if (parts.some((p) => !p || p === '.' || p === '..')) return ''; return parts.join('/'); } /** * @param {string} root skill folder id * @param {string} rel * @returns {Promise<string|null>} */ async function readPackagedSkillFile(root, rel) { if (typeof chrome === 'undefined' || !chrome.runtime?.getURL) return null; const url = chrome.runtime.getURL(`src/agent/vnext/skills/${root}/${rel}`); try { const res = await fetch(url); if (!res.ok) return null; return await res.text(); } catch { return null; } } /** * @param {string} skillId * @param {string} resourcePath logical path e.g. templates/report.html * @returns {Promise<string|null>} */ export async function loadSkillResource(skillId, resourcePath) { const skill = getSkill(skillId); if (!skill) return null; const rel = safeSkillRelPath(resourcePath); if (!rel) return null; const packed = skill.resources?.[rel]; if (packed != null && String(packed) !== '') return String(packed); const prefix = String(skill.packagePrefix || ''); if (prefix && (rel === prefix.replace(/\/$/, '') || rel.startsWith(prefix))) { return await readPackagedSkillFile(skill.root || skill.id, rel); } return packed == null ? null : String(packed); } /** * Industry progressive disclosure: system gets catalog only (id/name/description). * Playbooks and resources load later via inspect view=skill. * * @param {object} [_ctx] * @returns {string} */ export function formatSkillsForSystemPrompt(_ctx = {}) { const catalog = (Array.isArray(_ctx.catalog) ? _ctx.catalog : listSkillCatalog()) .slice() .sort((a, b) => a.id.localeCompare(b.id)); if (!catalog.length) return ''; return [ 'Skills are optional folder packages (playbooks + templates/scripts), not tools and not modes.', 'Match the user intent to a skill description using semantic understanding — never by host keyword lists.', 'If no skill description fits, do not load any skill. Use the machine (inspect / run / action / acquire).', 'If a description fits, load that playbook with inspect view=skill and that skillId before following it.', 'If the loaded playbook lists a resource path, load it with inspect view=skill, the same skillId, and path.', 'Create, import, or delete a skill only when the user explicitly asked to. Never invent a skill unprompted.', '', '### Skill catalog', ...catalog.map((s) => `- id: ${s.id}\n name: ${s.name}\n description: ${s.description}`) ].join('\n'); } export function clearSkills() { SKILLS.clear(); } // ── Register built-in folder packages ─────────────────────────────────────── for (const pack of [pageRestyle, siteToolReuse]) { registerSkill({ ...pack, root: pack.id }); } - src/agent/vnext/skills/site-tool-reuse/index.js (397 bytes)
import { parseSkillMd } from '../parseSkillMd.js'; import { SKILL_MD } from './skillSource.js'; const { meta, body } = parseSkillMd(SKILL_MD); export const skillId = 'site-tool-reuse'; export const skill = { id: skillId, name: meta.name || 'Site Tool Reuse', description: meta.description, instructions: body, libraries: [], resources: {}, templates: {} }; export default skill; - src/agent/vnext/skills/site-tool-reuse/SKILL.md (1953 bytes)
--- name: Site Tool Reuse description: User needs an outcome a live website or already-open logged-in tab already provides — edit, export, generate, convert, upload, send — including how to discover and compare those capabilities with other channels. --- # When this applies The result may already exist as an in-page control, an open tab, or a service the user is logged into. This playbook is a discovery method, not a required path. # Discover 1. Read this turn's world: focusPage / activeTab, and openTabs (tabCount + domains only). 2. Snapshot or screenshot the current page if you need controls or pixels. 3. If a matching domain is already open, you can focus that tab. Opening a parallel editor is optional. 4. Note logged-in services the user named. # Compare then choose Write one comparison line before acting: - Ready-made site tool: what it already does, and the visible success check. - action (ref / name / pointer), run + sys, or a self-built canvas: what each can do here. Pick the channel that best fits quality, reliability, and cost for this task. Host codes (NO_TARGET, CDP_BUSY, TAB_NOT_VISIBLE) are facts for switching, not a prescribed order. # Operate Use whichever live-tab or guest-sys channel you chose. After a mutate, check the page (url / controls / screenshot if attached). # Check afterward Confirm a host-visible postcondition (new filename, canvas object, download, URL change). Tool `ok` is not task success. Upload `siteAccepted` stays unknown — look at the page. # If the preferred route fails Re-enter discovery. Try another control, another open domain, another channel, or another logged-in service the user already has. Then compare again. A failed preferred route is not a lock; discovery stays allowed. # Parameterize Keep task class, intents, and locate strategies (role, accessible name, region, or viewport-relative pointer). Do not persist `ref`, `rev`, coordinates, or fragile CSS.
- src/agent/vnext/skills/site-tool-reuse/skillSource.js (2042 bytes)
// Bundled SKILL.md text for Chrome (no .md import). export const SKILL_MD = `--- name: Site Tool Reuse description: User needs an outcome a live website or already-open logged-in tab already provides — edit, export, generate, convert, upload, send — including how to discover and compare those capabilities with other channels. --- # When this applies The result may already exist as an in-page control, an open tab, or a service the user is logged into. This playbook is a discovery method, not a required path. # Discover 1. Read this turn's world: focusPage / activeTab, and openTabs (tabCount + domains only). 2. Snapshot or screenshot the current page if you need controls or pixels. 3. If a matching domain is already open, you can focus that tab. Opening a parallel editor is optional. 4. Note logged-in services the user named. # Compare then choose Write one comparison line before acting: - Ready-made site tool: what it already does, and the visible success check. - action (ref / name / pointer), run + sys, or a self-built canvas: what each can do here. Pick the channel that best fits quality, reliability, and cost for this task. Host codes (NO_TARGET, CDP_BUSY, TAB_NOT_VISIBLE) are facts for switching, not a prescribed order. # Operate Use whichever live-tab or guest-sys channel you chose. After a mutate, check the page (url / controls / screenshot if attached). # Check afterward Confirm a host-visible postcondition (new filename, canvas object, download, URL change). Tool \`ok\` is not task success. Upload \`siteAccepted\` stays unknown — look at the page. # If the preferred route fails Re-enter discovery. Try another control, another open domain, another channel, or another logged-in service the user already has. Then compare again. A failed preferred route is not a lock; discovery stays allowed. # Parameterize Keep task class, intents, and locate strategies (role, accessible name, region, or viewport-relative pointer). Do not persist \`ref\`, \`rev\`, coordinates, or fragile CSS. `;
- src/AGENTS.md (14232 bytes)
# src — Chrome 宿主面 本目录是扩展的可执行面。进程分层见根 [AGENTS.md](../AGENTS.md)。模型循环与工具契约见 [agent/AGENTS.md](agent/AGENTS.md)。画布页见 [preview/AGENTS.md](preview/AGENTS.md)。侧栏模块见 [sidepanel/README.md](sidepanel/README.md)。 Service worker 会被 Chrome 杀掉。会话仓库、AI SDK、`run` 沙箱客户端都放在 **offscreen**,不放 SW。 ## 进程入口 | 面 | 路径 | 现状 | |----|------|------| | Service worker | `background.js` | `manifest.background.type=module`。标签 / 下载 / 预览页 / office RPC / offscreen 转发 / **`workspace_page_action` 跨 frame 扇出** / **`workspace_sys`** / **tab 租约(storage.session)** / **`chrome.alarms` 唤醒 due task / 定时 Workflow** | | Offscreen | `offscreen/runtime.html` + `runtime.js` | `SessionWorkspaceService.create()`;收 `workspace_rpc_execute`。内嵌 sandbox iframe,**不**把 workspace boot 闸在 handshake 上 | | Content script | `content_script.js` | `matches: <all_urls>`,`run_at: document_idle`,**`all_frames: true`**。点选选区 + 每 frame 的 action。经典脚本(IIFE),`executeScript` 可再注入 | | Sidepanel | `sidepanel.html` + `sidepanel.js` | 对话 UI。`workspaceRpc` → background。编排器仍是 `sidepanel.js` | | Sandbox | `sandbox/runtime.html` | `manifest.sandbox`;无 `chrome.*`。`run` 的访客代码 + FS / `sys` postMessage | | Preview | `preview/` | 扩展页标签:docs / site / artifactPreview / print | `icons/` 在仓库根,不在 `src/`。`web_accessible_resources` 目前只有 `src/agent/vnext/sessionWorkspace/pickContext.js`(content script 动态 `import`)。 CSP:`extension_pages` 允许 `'wasm-unsafe-eval'`(QuickJS);sandbox 另加 `'unsafe-eval'`。 ## 消息总线 约定:`target` + `action`。Offscreen / SW 用 `browserbot-*`;部分截图/选区消息仍用 `browserbot_*`。 ```text 侧栏 / 预览页 chrome.runtime.sendMessage { target: browserbot-background, action, … } workspace_rpc → ensureBrowserBotOffscreen + 契约检查(仅只读调用最多重试 8 次;写入不重试) workspace_page_action → handleWorkspacePageAction → 各 frame content_script sheet_host → 工作簿 RPC(文件,不另开表格引擎页) canvas_host → 无设计画布,返回 NO_CANVAS workspace_fetch / workspace_sys / workspace_capture_* / workspace_find_tab / … workspace_tab_lease_peek|release / workspace_task_resolve_page storage_local_get|set → chrome.storage.local 代理 workspace_get_llm_settings / workspace_get_active_page offscreen → SW session_workspace_event → 侧栏刷新(execution-start/end、流式、artifact_preview、task-updated、task-schedule-changed、…) content_script → SW 点选选区、browserbot_region_selected、截图、toast ``` Offscreen URL:`src/offscreen/runtime.html`。SW `ensureBrowserBotOffscreen()`。 `workspace_sys` 只接受本扩展 offscreen URL 的 sender;调用携带 callId/sessionId/executionId/deadline,取消走同一消息入口。`workspace_rpc` 只在接收端连接尚未建立时重试;响应丢失不代表未执行,不自动重放可能有副作用的请求。 ### `workspace_rpc` ```text sidepanel workspaceRpc(method, params) # vnext/host/workspaceClient.js → { target: browserbot-background, action: workspace_rpc } → background.forwardWorkspaceRpc → { target: browserbot-offscreen, action: workspace_rpc_execute } → SessionWorkspaceService[method] ``` `method` 必须在 `workspaceRpcContract.js` 的显式清单内;按 Chrome 原生 sender 区分侧栏、预览与内部调度。`getTaskSchedule` / `runDueTasks` / `getBrowserRuntimeState` 只给 SW;offscreen 拒绝侧栏绕过 SW 直调。新增公开 async 方法不自动暴露。侧栏和四种预览共用 `workspaceClient.js`,完整参数语义仍由领域方法验证。`rewindSession` 是侧栏写入:先停掉该会话当前执行,再截 durable 对话记录。文件、交付物、页面副作用和未完成的 durable task 不回滚。截断规则见 [agent/AGENTS.md](agent/AGENTS.md)「回滚用户消息」。 ## Selector 与对话引用 点选是网页内容入口,同时支持直接下载/复制/导出和 Bot 引用。侧栏抽屉「收集到」选择新内容的资料组;输入框「参考」管理本会话绑定,两者独立。SelectionGroup 保存资料,既不是 live `action` 的前置条件,也不代表网页操作授权。文字先收进扩展剪贴板,再由用户点击复制到系统剪贴板。展示命名调整不改变 Group/WebItem/sessionBindings 的 RPC 或存储合同。 点选候选只从指针 hit stack 派生,不能用整页媒体矩形越过实际遮挡。默认一击采集首项;悬停边缘的可点击范围标签可直接采集图片、链接、卡片、单元格/行/表等关联范围,标签自身不参与点选,Esc 仍退出。新捕获携带 `scope`、`facets`、保留段落和缩进的 `textRaw`、严格 `locatorFingerprint`,以及 `source.tabId/frameId/documentId/url`;恢复必须按 frame/document 定向并核验身份,不能把新项静默重定到相似节点。 卡片 `facets` 保存其本体文字以及最多 8 个媒体、8 个链接的关系,可靠字段才写入;它不是完整页面抓取。点选保存内容与来源,不授予 live `action` 权限。`locatorFingerprint` 只用于恢复或采集时拒绝错误重定;live action 仍必须走既有 opaque ref/rev 和宿主权限链。 ## `action` 运输 ```text SessionWorkspaceService.sendMessage hostPageAction(payload) → { target: browserbot-background, action: workspace_page_action } → handleWorkspacePageAction → chrome.tabs.sendMessage({ action: workspace_page_action, … }, { frameId }) → content_script 本 frame 执行 ``` 模型参数与错误码在 [agent/AGENTS.md](agent/AGENTS.md)「`action` 契约」。宿主行为: Background: - `chrome.webNavigation.getAllFrames` 列 frame;跳过 `chrome://` / `chrome-extension://` / `edge://` / `devtools://` / `view-source:`。 - 缺脚本的 frame:`chrome.scripting.executeScript` 注入 `src/content_script.js`(回退 `content_script.js`)。 - 各 frame 本地 `aN` 编成不透明 `f{frameId}.aN`。 - `pageActionHost.js` 从 background 抽出;同 tab 的 action 排队。`documentTarget.js` 为快照保存 frameId/documentId/URL,用随机不透明 rev,SW 重启后必须重新 snapshot。 - `action` 的全部 mutate(包括 name-only 和 bare press)需要 rev;每次派发前核对文档,消息用 `tabs.sendMessage` 的 documentId 定向。导航、history/fragment 变化使旧快照失效。 - 页面操作先成功写入 storage.session 的租约;`tabs.close` 同样占锁。`sys.cdp` 的 targetId 先归一到 page tabId,不能绕过锁。`sys.tabs.current` 不再 fallback 当前焦点;缺省只允许已注入的本轮 defaultTabId。 - `execution-end`、finally、abort 通过 `browserExecution.js` 精确释放 sessionId+executionId:先撤销后续派发,再取消 sys/断开 CDP,最后放锁。SW 首次执行前与 offscreen 活动执行对账,恢复失败不继续派发。浏览器重启/扩展重载会清空 storage.session 租约与 session 政策覆盖。 - `action` mutate 与关键 `sys` 副作用走双门:offscreen `gatedDispatch`(分类 + journal + 审批 + one-shot ticket)后,SW `consumeDispatchTicket` 再分类一次。已知付款即使有 ticket 也不派发。通用存储桥(`storage_*_get|set`)禁止读写派发工单键,工单只经 `workspace_ticket_put|drop`。 - `sys.eval/waitFor/fetch(as:page)` 使用 documentIds 定向,可以传 documentId/expectedUrl。`sys.upload` / `action op=upload` 共用 SW `uploadChannel.js`:字节经 `chrome.scripting.executeScript({ world:'MAIN', func, args })` 分块注入,默认 `DataTransfer` + `input.files` setter + 手派 `input`/`change`,找不到 file input 再脚本 drop。`auto` 不 attach CDP。回执带 `methodUsed`、`trusted:false`、`siteAccepted:'unknown'`,不声称站点已接受。未知写入回执要先观察再决定,不能自动重放;确认动作后的观察失败单列 observationError。 - 预览页 workLock 仍是同 session 画布 UI 锁,不与 live tab 租约合并。 - 合并 snapshot 后 `controls` 再截到 80 条。 - `fill_form` 按 `frameId` 拆开发送,出现失败/未知回执停止后续 frame;返回已知的部分结果,不伪装为原子事务。 - `wait` + `text`(无 `ref`)对所有 frame 并行等,任一命中即停。 - `name` 回退:内部 `op: resolve_name`(不在模型 schema 里)。 - 限制页(`chrome://`、`edge://`、`about:`、`devtools://`、`view-source:`、Web Store、扩展页)→ `NEED_PAGE`。 Content script: - 本 frame snapshot:可交互节点 cap **80**(`ACTION_SNAPSHOT_CAP`);本地 ref 为 `a1`…。 - 控件名(`computeActionName`):aria-labelledby → aria-label → 字段取 label/placeholder/title;按钮、链接、menuitem 等取自身文本(含 img alt、submit 的 value)再 title;附近文本只在都为空时兜底。无 role/tabindex 的 div/span/li 菜单:可见、短文本、自身 `cursor:pointer` / `onclick` / `data-action` / 交互 role、或位于 nav/menu 容器内时,以 role `generic-clickable` 补入(每 frame 最多 40,嵌套取最内层)。纯逻辑在 content_script 的 `action-controls:pure` 标记块,`tests/action-controls.test.mjs` 直接求值测试。 - 公开 op:`snapshot` / `fill_form` / `click` / `fill` / `select` / `press` / `scroll` / `wait`;`upload` 不在 content script 里赋文件(SW MAIN-world 注入)。另有内部 `resolve_name` / `mark_upload`。`fill` 对 file input 仍 `FILE_INPUT`;`click` 打到 file input 回 `FILE_CHOOSER`。 - `resolveActionTarget` 另有 `css` 路径;模型工具参数未暴露该字段。 - `wait`:无 `text`/`ref` 时睡 `ms`(默认 300,上限 5000)。 - `press` 无目标时打到主 frame(`frameId` 0)的 `activeElement`。 ## 预览标签(SW 打开) `background.js` 按 artifact 打开 `chrome-extension://…/src/preview/*.html?sessionId&artifactId`,并用 tab 缓存 + `*_tab_ready` 握手。`sheet_host` 是工作簿文件 RPC,不另开表格引擎页。`canvas_host` 返回 `NO_CANVAS`。种类 → 页面映射见 [preview/AGENTS.md](preview/AGENTS.md)。 会话相关预览标签可进 `tabGroups`(`browserTabGroups` / `attachTabToSessionGroup`)。 ## 持久化(宿主键) | 存哪 | 键 / 库 | 用途 | |------|---------|------| | IndexedDB + OPFS | `browserbot-session-workspace-v1` | 会话 / group / artifact / fsNodes / `meta`(含可选 `task:*`)(offscreen) | | `chrome.storage.local` | `browserbot_providers`、`browserbot_active_provider_id` | BYOK | | 同上 | `browserbot_chat_media_evidence_v1`、`browserbot_chat_media_auto_check` | 连接范围的媒体校验证据与自动校验开关;offscreen 调度,SW 监听配置变更触发,普通发送只读 | | 同上 | `browserbot_context_window_evidence_v1`、`browserbot_context_window_auto_check` | 精确 base+model 的上下文窗口探测结果与自动探测开关;目录已有窗口则不 chat-probe;发送只读、不 await | | 同上 | `browserbot_web_acquire` | acquire 搜索/抓取设置 | | 同上 | `browserbot_cloud_account` | Browser Kitten 账户:设备令牌、登录时代填了哪些 web 槽位(`cloudAccount.js`)。不含套餐/积分,那由服务端每次判定 | | 同上 | `browserbot_theme_mode`(兼 `browserbot_theme`) | 侧栏主题 | | 同上 | `browserbot_user_skills` | 用户固化 skill(`agent/skills.js`,侧栏) | ## 其它宿主事实 - 权限与命令见根 AGENTS.md / `manifest.json`。`userScripts` 给 `sys.eval` / page `fetch`,也是 guest `sys.cdp` 脚本类命令(`Runtime.evaluate` 等,见 `cdpScriptGate.js`)的开关;`debugger` 给 `sys.cdp`。Chrome 135+ 的 `userScripts.execute` 需要扩展卡片 **允许运行用户脚本**(或更早的开发者模式)。CDP 挂上时 Chrome 会显示调试横幅;F12 已打开会 `CDP_BUSY`。`eval` / page `fetch` / `cdp` 不注入扩展预览页。`sys` ABI 目录见([agent/AGENTS.md](agent/AGENTS.md))。 - 区域截图:Alt+Shift+C → content script 框选 → SW `captureVisibleTab` 裁剪 → 剪贴板 + 对话附件。 - `llm_proxy_fetch`:侧栏/设置探测模型时走 SW,避免页面 CORS。 ### 媒体观察 `action snapshot` 附 `mediaElements`(最多 6 项,含 frame/document 的媒体播放和源线索);它只提示可用证据,不自动选择路径。`action op=record` 经 SW tabCapture → offscreen 录制可见标签画面+声音(默认 8 秒,最大 30 秒),只在当前模型可接收原生视频时执行。`browserbot_record_grant:{sessionId}:{tabId}` 是独立侧栏视频授权,听音 grant 不覆盖它。两种采集共用生命周期清理,但 listen 保持 `video:false`。详见 [CHAT_MEDIA_INPUT.md](../docs/CHAT_MEDIA_INPUT.md)。 ## 顺序 Workflow 宿主 `workflowHost.js` 接收仅 offscreen 可用的 `workspace_workflow_host`;该入口只提供录制运输、标签/下载读取及 UI 高亮,不增加回放执行通道。真实内容事件由 SW 验证 sender 的 tab/frame/document 身份,再转内部 RPC;网页与 preview 不能发布或运行流程。显式录制范围包含已加入标签及 opener 子标签。重启/运输或存储失败暂停录制,不能静默遗漏后声称完整。页面回放依旧走 pageActionHost;针对 Workflow 的窄目标 snapshot 保持原 ref/rev/documentId 校验,滚动距离也参与派发哈希。详见 [Workflow 合同](../docs/WORKFLOW_V1.md)。 ## 当前事件入口 网页侧已有点选、截图和 Workflow 录制事件;Workflow 录制由 SW 核验原生 sender 的 tab/frame/document 身份后转内部 RPC。Task 与已发布 Workflow 的后台启动来自 alarms。当前没有通用网页事件订阅、聊天新消息队列或自动回复入口;这些不能由既有 workflow_record_event 直接启动。 - src/preview/AGENTS.md (2481 bytes)
# src/preview — 画布与预览标签 扩展页,由 SW 打开(`chrome.runtime.getURL('src/preview/…')`)。**不是** content script,也不是 agent 循环。读写经 `sheet_host` / artifact RPC 回到 SW → offscreen store。`canvas_host` 返回 `NO_CANVAS`。 分类仍由 `openClassify.js` 做 kind / MIME / 魔数。打开面、家族与徽标走 `artifactCapability.js`(`previewEntryForItem` / `previewViewForItem` 是它的再导出)。shelf 与预览不得另写一套 if-else。 ## 页面 | 页 | 引擎 | 打开时机 | |----|------|----------| | `docs.html` | 标记 HTML 正文 | docx / json-document / html-document / html-plates | | `site.html` | 静态 HTML+CSS 宿主 + postMessage bridge | `data-browserbot-kind=site`。**不是**任意 guest JS 运行时。用户 `<script>` / on* 被 `siteSanitize` 剥掉。bridge(ready/click/serialize/nudge)跑在 `sandbox.pages` 的 `siteFrame.html` + 外部 `siteFrame.js`,无 chrome.*、无 extension-origin 同源。禁止把内联 `<script>` 塞进 srcdoc。动态行为另立安全 runtime | | `artifactPreview.html` | 通用查看 | 图片/svg、PDF 只读重建、音视频、text-like 转义只读、未知检查器。未知 HTML 不 `srcdoc` 执行。文案:`?lang=en\|zh` 优先,**不**把页面上写死的 `zh-CN` 当用户语言 | | `print.html` | 系统打印 | PDF 交付:`delivery: browser_print` → Save as PDF | | `preview.html` | 旧草稿预览 | 遗留 `open_draft_preview` | 共享:`workLock.js`(**同一 session** 在预览画布上的执行中 UI 锁,`execution-start/end` →「编排中」;**不是** SW 里跨 session 的 live-page tab 租约 / `TAB_LEASED`)、`officeHelp*` / `officeShortcuts` / `officeSelBubble`、`durableImage.js`、`host-bar.css`。表编解码:`sheetCodec.js` / `sheetModel.js`(agent office 工具也会 import)。 ## Vendor `vendor/fflate.js` 已跟踪,打开 docx 时解压用。本仓库日常无 `package.json`。 ## 宿主约定 - Query:`sessionId`、`artifactId`;预览语言可选 `lang=en|zh`。 - 页 load 后发 `sheet_tab_ready` / `html_tab_ready` / `docs_tab_ready`。 - 同一 session+artifact 复用标签(SW 里 `sheetTabByKey` / `htmlTabByKey`)。 - **预览/检查器**走 `readArtifactPreview`(默认 64K,硬顶 8M)。**下载/编辑器载入**走 `downloadArtifact` / `readArtifactChunk`,必须原字节完整。截断只能显式拒绝,不能把 truncated 前缀当完整文件。 - SUPPORT.md (1888 bytes)
# Support / 获取帮助 ## Start with the guide - [Getting started](docs/GETTING_STARTED.md) · [中文安装指南](docs/GETTING_STARTED.zh-CN.md) - [Troubleshooting](docs/GETTING_STARTED.md#troubleshooting) · [常见问题](docs/GETTING_STARTED.zh-CN.md#常见问题与反馈) - [Documentation index](docs/README.md) Inference and System1 need separate configurations. Ordinary HTTP(S) pages work differently from restricted Chrome pages. Local follow-ups require Chrome to remain running and the computer awake. ## Ask or report Search [existing issues](https://github.com/Player-YN/BrowserKitten/issues). If your problem is new, use the [bug report form](https://github.com/Player-YN/BrowserKitten/issues/new?template=bug_report.yml); for a capability proposal, use [feature requests](https://github.com/Player-YN/BrowserKitten/issues/new?template=feature_request.yml). Other questions can use a blank issue. Chinese and English are welcome. Include the extension/source version, Chrome version, operating system, a small reproduction, and the exact error. A redacted screenshot can help. Never post keys, tokens, cookies, or entire private conversation exports. Screenshots and task traces can contain sensitive page content even when no key is visible. Suspected vulnerabilities belong in [private security reporting](SECURITY.md). For model-provider billing, quotas, or service availability, contact that provider. Browser Kitten account-hosted services depend on a separate backend. This is a community project with no guaranteed response time or support SLA. **中文:**遇到问题请先看安装指南和常见问题,再用中文或英文提交 Issue,附版本、复现步骤与错误码。截图先脱敏;不要提交密钥、登录令牌、Cookie 或整份私人任务轨迹。漏洞走私密入口,服务商计费和额度问题联系对应服务商。
- THIRD_PARTY_NOTICES.md (1470 bytes)
# Third-party notices Browser Kitten's own code is MIT-licensed (see [LICENSE](LICENSE)). This repository **does** commit and redistribute the third-party runtime bundles listed below — an unpacked MV3 extension has no install step, so every dependency must already be in the load root. | Component | License | How it ships | |-----------|---------|--------------| | [Lucide](https://lucide.dev) | ISC | Icon path data inlined in `src/sidepanel/icons.js` | | [esbuild](https://esbuild.github.io) (`esbuild-wasm`) | MIT | Committed as `src/agent/vnext/adapters/vendor/esbuild.wasm` + `esbuild-loader.mjs` | | [QuickJS](https://bellard.org/quickjs/) via [quickjs-emscripten](https://github.com/justjake/quickjs-emscripten) | MIT (both) | Committed as `src/agent/vnext/adapters/vendor/quickjs-loader.mjs` | | [Vercel AI SDK](https://ai-sdk.dev) (`ai`, `@ai-sdk/openai-compatible`) | Apache-2.0 | Committed as `src/agent/vnext/adapters/vendor/ai-sdk-loader.mjs` | | [Zod](https://zod.dev) | MIT | Bundled inside the committed `ai-sdk-loader.mjs` | | [fflate](https://github.com/101arrowz/fflate) | MIT | Committed as `src/preview/vendor/fflate.js` | | [Playwright](https://playwright.dev) | Apache-2.0 | Dev-only, supplied by the machine running `tests/browser-smoke.cjs`; not committed, never packaged | Upstream license texts are at the linked project pages. If you redistribute builds of this extension, you are responsible for complying with each upstream license.